We ask the user the upload a shapefile (.shp). I cannot however figure out how to read that shapefile with Fiona because Viktor gives me a Viktor File object of the shape file. To use fiona, you use the code snipped below where path/to/file = URI(strorpathlib.Path), orfile-like object (documentation)
with fiona.open(path/to/file, "r") as file:
# Do something
On a local file system you would say something like “C/path/to/shapefile.shp”. However, on Viktor you get a Viktor File object.
If I use shape_variable.file.getvalue_binary()) and pass that into fiona, you get the error TypeError: cannot use a string pattern on a bytes-like object.
If I use BytesIO(cadastre_parcels.file.getvalue_binary()) and pass that into fiona, I get the error: fiona.errors.DriverError: '/vsimem/699e1fedd47743dfba620fb0dbc27f5a/699e1fedd47743dfba620fb0dbc27f5a' not recognized as a supported file format
Is there somebody who knows how access the shapefile in a way that Fiona accepts it?
With file-like object, the fiona documentation indeed refers to a BytesIO or StringIO object. from the io-package. These are also called streams, and are used to represent any source of data (such as files but also memory arrays … ). Using streams allows you to work with all types of data in the same way, no matter if it is actually file saved locally, saved somewhere else, an object in memory or something else
You can get a BytesIO from a viktor File object directly with open_binary(). So no need to start with getvalue_binary() and create it yourself.
file = File(...) # viktor-file
with fiona.open(file.open_binary(), mode="r") as file:
# Do something
Thanks for the reply. Fiona however does still not accept the file. The following is my input for Fiona cadastre_parcels.file.open_binary() where cadastre_parcels is the variable from params. When I pass this into Fiona, I get the following error.
fiona.errors.DriverError: ‘/vsimem/752b65b8e4f2468abe81b898d43eabc1/752b65b8e4f2468abe81b898d43eabc1’ not recognized as a supported file format.
I also cannot get it to work unfortunately. I tried to convert it to a BytesIO object first, but it still doesn’t work. Something like this. Hope to hear from you soon! We are using Fiona==1.8.22 and Gdal==3.4.3. I did try to update Fiona to the newest version. For Gdal this is harder since I cannot find a wheel for the newest Gdal version.
with fiona.open(BytesIO(cadastre_parcels.file.getvalue_binary()), "r") as geometry_reader:
I’ve taken some time to dive into your issue, and it seems to be that the package fiona is doing some format checking before opening the file for reading. I haven’t looked into too much detail to find all alternatives, but here is at least one that I think should work:
# get file from a `FileField`
shapefile: FileResource = params.shapefile
# Create a temporary file in memory using a `NamedTemporaryFile` object
tmp = tempfile.NamedTemporaryFile(delete=False)
# Make sure to give it a name with the correct format. Assuming you have uploaded the right format, this should not give any errors later on
tmp.name = shapefile.filename
# write the uploaded file to the `NamedTemporaryFile` object
tmp.write(shapefile.file.getvalue_binary())
# Continue with your logic
with fiona.open(tmp) as src:
...
# Close your file
tmp.close()
Thanks for the reply. We also discovered a similar solution. A shape file always has helper files so we needed to upload a zip file. We use the code below
The shape_file variable can be used when calling Fiona. We haven’t tested this on the Viktor servers however, only locally.
import os
from tempfile import NamedTemporaryFile, gettempdir
from zipfile import ZipFile
# VIKTOR imports
from viktor.api_v1 import FileResource
temp_zipfile = NamedTemporaryFile(suffix=".zip", delete=False, mode="wb")
temp_zipfile.write(cadastre_parcels.file.getvalue_binary())
temp_zipfile.close()
# Extract the zipfile in the temp folder!
zipfile_name = os.path.basename(temp_zipfile.name).split(".")[0]
zipfile_path = os.path.join(gettempdir(), zipfile_name)
ZipFile(temp_zipfile.name).extractall(path=zipfile_path)
for file_ in os.listdir(zipfile_path):
if file_.endswith(".shp"):
shp_file = os.path.join(zipfile_path, file_)
break
Great to see that you have made progress. If you finally get to a working version, maybe it is an idea to share with the community what you have created with a snippet and image in the “Automate the boring, Engineer the Awesome” channel. With that we can help one another by inspiring and showing what is possible.
Paulien made a little mistake here. In her snippet, a new empty File Object is created. But in the question of OP, there is already an uploaded file available as VIKTOR file object. So no need to create a new File.
If you want to open a file in fiona from a Viktor File object, from a FileField or an FileEntity. The best way is as follows:
with shape_file.open_binary() as fp:
with fiona.open(fp, "r") as fiona_src:
#.. Do something with fiona_src
Here shape_file is the VIKTOR file object from the FileField. In this example, everything is kept in memory and no files are written to the local file system (whicg can cause problems in production).
Thanks for this thread.
I am trying to open a shape file uploaded to a FileField using geopandas(gpd). However, I get the following error “‘/vsimem/0054ecffb9d44597aa81bace08a79a53’ not recognized as a supported file format.” at the gpd.read_file(fp) line. What could be missing?
Thank you for your help in advance.
def site_area_calculation(params, **kwargs):
"""
Computes the area of a site defined by a list of WGS84 coordinates.
points: List of tuples containing WGS84 coordinates [(lon1,lat1), (lon2,lat2), ...]
Returns: Area in square meters
"""
# List to store the site map polygon features.
site_features = []
if params.tab_2.section_2.site_shape_file:
# Assign the shape file to a variable.
shape_file = (params.tab_2.section_2.site_shape_file).file
# Read the shape file.
with shape_file.open_binary() as fp:
shape = gpd.read_file(fp)
# Ensure the CRS of the shapefile is WG84.
wg84_shape_file = shape.to_crs(epsg=4326)