Reading zipped gpkg files (i.e. `*.gpkg.zip`) with Python

Question

I have many large gpkg.zip files that I would like to read directly into Python (to then upload to BigQuery). I can unzip each, read with geopandas, then upload -- however, since unzipping triples the space the file takes up, it starts to really clog up my computer's relatively small hard drive.
Is there a way to read a compressed file in directly?

Spacedman · Accepted Answer

You can use the GDAL /vsizip system.
I have this geopackage with one layer:
$ ogrinfo ottawa.gpkg 
INFO: Open of `ottawa.gpkg'
      using driver `GPKG' successful.
1: ottawa (Multi Polygon)

And I can create a zip file containing just it:
$ zip ottawa.gpkg.zip ottawa.gpkg 
  adding: ottawa.gpkg (deflated 38%)

And its nice and smaller than the source:
$ ls -sh ottawa.gpkg*
872K ottawa.gpkg  544K ottawa.gpkg.zip

Then I can access that file with /vsizip with the zip file name and the name of the file in the zip file:
$ ogrinfo /vsizip/ottawa.gpkg.zip/ottawa.gpkg 
INFO: Open of `/vsizip/ottawa.gpkg.zip/ottawa.gpkg'
      using driver `GPKG' successful.
1: ottawa (Multi Polygon)

The downside is this will be slower.

Reading zipped gpkg files (i.e. `*.gpkg.zip`) with Python

One Answer

Add your own answers!

Ask a Question