Geographic Information Systems Asked by Rachel Thoms on August 14, 2021
I am writing one of my first scripts in GEE. I have grabbed various code chucks from stack exchange and have created a memory intensive script. My goal is two take the union of two subsets of global river datasets, buffer them depending on whether width information is provided, create a mask, and apply it to a raster.
When trying to export the raster image to an asset, I get the error Error: Execution failed; out of memory.Error: Execution failed; out of memory.
It executes when I use 1 test feature from each dataset. The full dataset fails. How can I reduce the amount of memory used by my script?
Here is a link to my code
// Testing variables
// var test_grwl = grwl.filter(ee.Filter.eq('OBJECTID',13455));
// var test_rivers = rivers.filter(ee.Filter.eq('HYRIV_ID',70629416));
// Get pixels where erosion above threshold (for each year)
var erosion_high = wat_070.toBands().gte(50);
// Filter HydroRivers to 1st order rivers
var first_order = rivers.filter(ee.Filter.eq('ORD_CLAS',1));
// Filter HydroRivers to Exhoreic Rivers
var first_order_ex = first_order.filter(ee.Filter.eq('ENDORHEIC',0));
// Function to buffer filtered rivers
var bufferBy = function(size) {
return function(feature) {
return feature.buffer(size);
};
};
// Buffer Exhoreic Rivers
var riv_bounds = first_order_ex.map(bufferBy(1000));
print('filtering');
// Filter GRWL rivers by the buffer
var grwl_sub = grwl.filter(ee.Filter.bounds(riv_bounds));
// Buffer GRWL rivers by their width
/// Define a variable buffer based on a property
var grwl_buffer = grwl_sub.map(function(feature){
// Set a new property.
var feature = feature.set('ORD_CLAS', 1);
//you need an expression to do math, and if you want to use a variable in an expression you have to wrap it in a dictionary
var width = ee.Dictionary({value:feature.get('width_med_')});
//define the buffer as a new geometry
var buffered_line = feature.geometry().buffer(
ee.Number.expression('(value/2)+7',width)
);
//coerce that geometry to a feature, copy the properties of the input feature, and return it
return(ee.Feature(buffered_line).copyProperties(feature));
});
// Filter HydroRivers to those not included in GRWL
var hydroRiv_sub = first_order_ex.filter((ee.Filter.bounds(grwl_buffer)).not());
// All HydroRivers (without width information) will be assigned a width of 10m
// Buffer Exhoreic Rivers by 1/2 width + 7m
var hydroRiv_buffer = hydroRiv_sub.map(bufferBy(5+7));
// Merge the two buffers
var combinedRiverBuffer = hydroRiv_buffer.merge(grwl_buffer);
// Make an image out of the combined buffer
var riv_image = combinedRiverBuffer
.reduceToImage({
properties: ['ORD_CLAS'],
reducer: ee.Reducer.first()
});
// extract mask from image
var riv_mask = riv_image.mask();
// apply mask
var combination = erosion_high.updateMask(riv_mask);
// rename bands
combination = combination.rename(['Y2002','Y2007','Y2012','Y2017','Y2020']);
// // VISUALIZE DATA
// /// Visualize the Exhoreic, 1st Order Rivers
// first_order_ex = first_order_ex.style({
// color: "0000ff",
// width: 1.0,
// });
// Map.addLayer(first_order_ex, {}, "First Order Exhoreic Rivers");
// /// Visualize the Exhoreic, 1st Order Rivers Buffer
// hydroRiv_buffer = hydroRiv_buffer.style({
// color: "0001ff",
// width: 1.0,
// });
// Map.addLayer(hydroRiv_buffer, {}, "HydroRiv Buffer");
// /// Visualize the GRWL Rivers
// grwl = grwl.style({
// color: "ff0000",
// width: 1.0,
// });
// Map.addLayer(grwl, {}, "GRWL");
// /// Visualize the GRWL Buffer
// grwl_buffer = grwl_buffer.style({
// color: "ff1000",
// width: 1.0,
// });
// Map.addLayer(grwl_buffer, {}, "GRWL Buffer");
// /// Visualize masked soil erosion
// var sld_intervals =
// '<RasterSymbolizer>' +
// '<ColorMap type="intervals" extended="false">' +
// '<ColorMapEntry color="#222222" quantity="0" opacity="0" label="None"/>' +
// '<ColorMapEntry color="#654321" quantity="1" opacity="1" label="Low Integrity"/>' +
// '</ColorMap>' +
// '</RasterSymbolizer>';
// Map.addLayer(combination.select([0]).sldStyle(sld_intervals), {}, 'Sediment Pressure Zones');
// Export the image to an Earth Engine asset.
Export.image.toAsset({
image: combination,
description: 'toAsset_OW_sediment_pressure_zones',
assetId: 'projects/resource-watch-gee/wat_070b_sediment_pressure_zones',
maxPixels: 1e13,
scale: 100,
pyramidingPolicy: {
'Y2002': 'mode',
'Y2007': 'mode',
'Y2012': 'mode',
'Y2017': 'mode',
'Y2020': 'mode',
}
});
I am sure this code needs work.
The collections in your code aren't shared, but one of the most likely candidates for running out of memory is this line:
var grwl_sub = grwl.filter(ee.Filter.bounds(riv_bounds));
If riv_bounds is a large and complex collection (seems likely), then trying to convert it to a single geometry is probably exceeding available memory. You might be able to get this to work using a inner join instead of a filter.
Answered by Noel Gorelick on August 14, 2021
Get help from others!
Recent Answers
Recent Questions
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP