TransWikia.com

Downloading file using URL value in shapefile field accessed via ArcPy cursor

Geographic Information Systems Asked by Thomas Zuberbuehler on June 23, 2021

How do I download files through Python using URLs located in a shapefile with ArcPy?

The script needs to read a shapefile, create folders and download images using URLs in more than 1 field/record (all starting with url*).

One Answer

import os
import arcpy
import shutil
import requests

# since I have no access to INDEX_utm20_PEI_Charlottetown_2008.shp, I am using 
# a test shapefile with 5 fields: 
# id (int), name (string), url1 (string), url2 (string), notes (string)
shapefile = r"dataphotos.shp"

photo_location = r"dataphotos"

fields = [field.name for field in arcpy.ListFields(shapefile, "url*", "string")]

with arcpy.da.SearchCursor(shapefile, ["id", "name"] + fields) as cursor:

    for row in cursor:
        
        id = row[0] # unique
        name = row[1]

        destination = os.path.join(photo_location, name)
        os.makedirs(destination, exist_ok=True)
        
        for url, field_name in zip(row[2:], fields):
            
            file_path = os.path.join(destination, f"{id}-{name}-{field_name}.jpg")
            
            response = requests.get(url, stream=True)
            if response.status_code == 200:
                with open(file_path, 'wb') as file:
                    response.raw.decode_content = True
                    shutil.copyfileobj(response.raw, file)

See also:


The above example does not use arcpy.env.workspace. I avoid using that whenever possible, especially in a a standalone script, or a complex script (using more than one path/workspace). I think this makes the script more explicit and easier to read.

Correct answer by Thomas Zuberbuehler on June 23, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP