TransWikia.com

Getting arcpy.da.walk to continue walking after encountering corrupt/problem file

Geographic Information Systems Asked by JRam on June 25, 2021

I am relatively new to python. I am trying to iterate through all DWG polylines in a directory with multiple (hundreds) of subdirectories. I would like the code to skip any DWG files that it cannot do something to and continue iterating.

The following code works fine, until it comes across any DWG files that are possibly corrupted. Python stalls and I cannot get it to throw an exception.

I have isolated the problem to the arcpy.da.walk. It appears that the walk function can open the corrupted DWG to find the polyline, but it cannot get out of the DWG to continue the walk.

I have tried multiple exception types. I am not sure how the onerror function call works in arcpy.da.walk,or even if this is an avenue of exploration (I cannot find any examples where onerror is anything other than “NONE”).

import arcpy  
import os  
import sys  
from arcpy import env  
import pdb  

##def WalkErr(onerror):  
##    try:  
##        import errno          
##        print errno  
##        if errno!=errno.ENOENT: raise  
##             
##        elif errno == errno.EACCES:  
##            print filename,"cannot open"           
##        else:  
##            print errno  
##            print filename, "Not Found....Continuing"  
##    except:  
##        print "Something went wrong"  


WorkspaceRoot="W:/GIS_Scratch"  

env.workspace=WorkspaceRoot  

#List workspaces in root directory  
workspaces=arcpy.ListWorkspaces ("*","Folder")  
for workspace in workspaces:  
    env.workspace=workspace  
    print workspace  

#Find workspaces named Drawings  
    workspaces1=arcpy.ListWorkspaces("*rawing*","Folder")  
    print workspaces1  

    for workspaceNext in workspaces1:          
        arcpy.env.workspace = workspaceNext  
        print workspaceNext  

#start stepping through code      
#    pdb.set_trace()  

# Walk through directory to find CAD polylines (Note: "CadDrawing" datatype does not return any polylines, so had to use "FeatureClass")  
        try:  
            for dirpath, dirnames, filenames in arcpy.da.Walk(workspaceNext,topdown=True, datatype='FeatureClass',onerror=None,type='Polyline'):  
                try:  
                    print dirpath  
                    for filename in filenames:  

                        try:  

                            arcpy.env.workspace=filename  
                            desc = arcpy.Describe(dirpath)  
                            featureTest=(os.path.join(dirpath,filename))  
                            print filename  
                            print featureTest  
                            print desc.extent.XMin  
                            filename.close  

                        except arcpy.ExecuteError:  
                            arcpy.AddError(arcpy.GetMessages())  
                        except:  
                            arcpy.AddError("Non-tool error occurred")  
                except OSError as e:  
                    print e.errno  
                    print e.filename  
                    print e.strerror  
                    continue  


                except Exception as e:  
                    print "An error has occurred"  
                    print e  
                    continue  

        except OSError as e:  
            print e.errno  
            print e.filename  
            print e.strerror  
            continue  
        except Exception as e:  
            print "An error has occurred"  
            print e  
            continue  

2 Answers

I suggest to use function, something like

def Get_Xmin(desc):
 try:
  return desc.extent.XMin
except:
  return -1

instead of continue

Answered by FelixIP on June 25, 2021

Below is the code that I used to finally solve my issue. Multiprocessing and os.walk allowed me to iterate all the DWGs within the directories.Thank you everyone for your help!

import os, arcpy, multiprocessing, csv, sys
outputCSV=r'W:GIS_ScratchMultiProcessing.csv'

#Get spatial extent of dwg files and spatial reference
def GetExtent(ds):
    desc=arcpy.Describe(ds)


    with open(outputCSV, 'a') as csvfile:
       csvwriter = csv.writer(csvfile)
       csvwriter.writerow([ds,desc.extent.XMin,desc.extent.YMin,desc.extent.XMax,desc.extent.YMax,desc.spatialReference.name])

#Find all .dwg files in directories
def main(workspace):
    arcpy.env.workspace = workspace
    dataset1 = arcpy.ListDatasets("*.dwg", "Feature")
    ds_list=[os.path.join(workspace,dataset) for dataset in dataset1]
    pool = multiprocessing.Pool()
    pool.map(GetExtent,ds_list)    
    pool.close()
    pool.join()

#iterate directories
arcpy.env.workspace = "F:/Metro Design/_Rev 8, jobs"
workspaces = arcpy.ListWorkspaces ("*","Folder")
for workspace in workspaces:
    arcpy.env.workspace = workspace
    workspaces = arcpy.ListWorkspaces ("*rawing*","Folder")
    for workspace2 in workspaces:
        for dirname, dirnames, filenames in os.walk(workspace2):
            for subdirname in dirnames:
                WorkspacePass = os.path.join(dirname, subdirname)
                if __name__=='__main__':
                    main(WorkspacePass)

Answered by JRam on June 25, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP