Releasing PyQGIS file locks?

Sorry to keep answering my own questions, but I think I found a solution.

As it turns out, it works well if you add the layer to the map registry, and then remove it again. The map registry takes ownership of the layer, so when it is deleted from the registry, the locks are freed. Note that you have to add the layer to the legend (.addMapLayer(layer, addToLegend = False) won't work).

Still not sure whether to call this a solution or a workaround, but it does the job.

# ...

# Replace the following code (note: should do error checking on map registry functions):

# Load layer 1
layer1 = QgsVectorLayer(layer1_path, "lyr1", "ogr")
if not layer1.isValid(): raise Exception("Failed to load layer 1")
QgsMapLayerRegistry.instance().addMapLayer(layer1) #!!!!

# Use layer 1 to create layer 2  
processing.runalg("qgis:reprojectlayer", layer1, "EPSG:54030", layer2_path)

# Load layer 2
layer2 = QgsVectorLayer(layer2_path, "lyr2", "ogr")
if not layer2.isValid(): raise Exception("Failed to load layer 2")
QgsMapLayerRegistry.instance().addMapLayer(layer2) #!!!!

# Remove layer references
QgsMapLayerRegistry.instance().removeMapLayer(layer1.id()) #!!!!
QgsMapLayerRegistry.instance().removeMapLayer(layer2.id()) #!!!!

# Remove data sources for layers
for f in os.listdir(project_temp_dir):          
    if f.endswith(".shp") and not os.path.isdir(project_temp_dir + f):    
    # ...

If anyone has more info, I'd be happy to learn more about this.

The workaround presented by Oystein didnt work for me, and i was also not able to find an actual solution to the file lock problem. However, i was able to find another general workaround for the problem using the tempfile lib, which is the following:

Instead of having QGIS save layers (both raster and vector) on a predefined folder i used tempfile lib to create a temporary folder for those intermediate layers everytime i call the function that creates those layers (a different temporary folder will be created everytime you call the function). I did it like this:

import tempfile

tempfolder = tempfile.mkdtemp()

You can easily check the path to the temporary folder with:

print tempfolder

Now everytime you create a new Vector or Raster layer either as the result of a processing algorithm, or via the QgsRasterFileWriter or QgsVectorFileWriter you can set the OUTPUT parameter to something similar to:

raster_Path = (tempfolder + "/yourRasterName.tif")
vector_Path = (tempfolder + "/yourVectorName.shp")

If you put those lines into some function you can repeatedly call it without having to worry with excluding intermediate layers.

Another option to avoid creating intermediate layers in the first place is to define the OUTPUT parameter of processing algorithms to None, so that the output stays in the memory, and then define your new layers refering to that output with the "OUTPUT" key. Like so:

                cutProcess = processing.runalg('gdalogr:cliprasterbymasklayer',
                              someRasterLayer,  # INPUT <ParameterRaster>
                              someMaskLayer,  # MASK <ParameterVector>
                              "-9999",  # NO_DATA <ParameterString>
                              False,  # ALPHA_BAND <ParameterBoolean>
                              False,  # CROP_TO_CUTLINE <ParameterBoolean>
                              False,  # KEEP_RESOLUTION <ParameterBoolean>
                              5,  # RTYPE <ParameterSelection>
                              4,  # COMPRESS <ParameterSelection>
                              1,  # JPEGCOMPRESSION <ParameterNumber>
                              6,  # ZLEVEL <ParameterNumber>
                              1,  # PREDICTOR <ParameterNumber>
                              False,  # TILED <ParameterBoolean>
                              0,  # BIGTIFF <ParameterSelection>
                              False,  # TFW <ParameterBoolean>
                              "",  # EXTRA <ParameterString>
                              None) # OUTPUT <OutputRaster>

                 outputRasterLayer = QgsRasterLayer(cutProcess["OUTPUT"],
                                            'newLayer')

Hope it helps. I'm using QGIS 2.18.22 and consequently Python 2.7.