TransWikia.com

Error: Execution failed; out of memory. Google Earth Engine Fails to Export Image to Asset

Geographic Information Systems Asked by Rachel Thoms on August 14, 2021

I am writing one of my first scripts in GEE. I have grabbed various code chucks from stack exchange and have created a memory intensive script. My goal is two take the union of two subsets of global river datasets, buffer them depending on whether width information is provided, create a mask, and apply it to a raster.

When trying to export the raster image to an asset, I get the error Error: Execution failed; out of memory.Error: Execution failed; out of memory.

It executes when I use 1 test feature from each dataset. The full dataset fails. How can I reduce the amount of memory used by my script?

Here is a link to my code

// Testing variables
// var test_grwl = grwl.filter(ee.Filter.eq('OBJECTID',13455));
// var test_rivers = rivers.filter(ee.Filter.eq('HYRIV_ID',70629416));

// Get pixels where erosion above threshold (for each year)
var erosion_high = wat_070.toBands().gte(50);

// Filter HydroRivers to 1st order rivers
var first_order = rivers.filter(ee.Filter.eq('ORD_CLAS',1));
 
// Filter HydroRivers to Exhoreic Rivers
var first_order_ex = first_order.filter(ee.Filter.eq('ENDORHEIC',0));

// Function to buffer filtered rivers 
var bufferBy = function(size) {
  return function(feature) {
    return feature.buffer(size);   
  };
};

// Buffer Exhoreic Rivers
var riv_bounds = first_order_ex.map(bufferBy(1000));

print('filtering');

// Filter GRWL rivers by the buffer
var grwl_sub = grwl.filter(ee.Filter.bounds(riv_bounds));

// Buffer GRWL rivers by their width

/// Define a variable buffer based on a property
var grwl_buffer = grwl_sub.map(function(feature){
  
  // Set a new property.
  var feature = feature.set('ORD_CLAS', 1);
  //you need an expression to do math, and if you want to use a variable in an expression you have to wrap it in a dictionary
  var width = ee.Dictionary({value:feature.get('width_med_')});
  
  //define the buffer as a new geometry
  var buffered_line = feature.geometry().buffer(
    ee.Number.expression('(value/2)+7',width)
    );
  
  //coerce that geometry to a feature, copy the properties of the input feature, and return it
  return(ee.Feature(buffered_line).copyProperties(feature));
});


// Filter HydroRivers to those not included in GRWL
var hydroRiv_sub = first_order_ex.filter((ee.Filter.bounds(grwl_buffer)).not());

// All HydroRivers (without width information) will be assigned a width of 10m

// Buffer Exhoreic Rivers by 1/2 width + 7m
var hydroRiv_buffer = hydroRiv_sub.map(bufferBy(5+7));

// Merge the two buffers
var combinedRiverBuffer = hydroRiv_buffer.merge(grwl_buffer);


// Make an image out of the combined buffer
var riv_image = combinedRiverBuffer
  .reduceToImage({
    properties: ['ORD_CLAS'],
    reducer: ee.Reducer.first()
});

// extract mask from image
var riv_mask = riv_image.mask();

// apply mask
var combination = erosion_high.updateMask(riv_mask);

// rename bands
combination = combination.rename(['Y2002','Y2007','Y2012','Y2017','Y2020']);

// // VISUALIZE DATA
// /// Visualize the Exhoreic, 1st Order Rivers
// first_order_ex = first_order_ex.style({
//   color: "0000ff",
//   width: 1.0,
// });

// Map.addLayer(first_order_ex, {}, "First Order Exhoreic Rivers");

// /// Visualize the Exhoreic, 1st Order Rivers Buffer
// hydroRiv_buffer = hydroRiv_buffer.style({
//   color: "0001ff",
//   width: 1.0,
// });

// Map.addLayer(hydroRiv_buffer, {}, "HydroRiv Buffer");

// /// Visualize the GRWL Rivers
// grwl = grwl.style({
//   color: "ff0000",
//   width: 1.0,
// });

// Map.addLayer(grwl, {}, "GRWL");

// /// Visualize the GRWL Buffer
// grwl_buffer = grwl_buffer.style({
//   color: "ff1000",
//   width: 1.0,
// });

// Map.addLayer(grwl_buffer, {}, "GRWL Buffer");

// /// Visualize masked soil erosion
// var sld_intervals =
// '<RasterSymbolizer>' +
//   '<ColorMap type="intervals" extended="false">' +
//     '<ColorMapEntry color="#222222" quantity="0" opacity="0" label="None"/>' +
//     '<ColorMapEntry color="#654321" quantity="1" opacity="1" label="Low Integrity"/>' +
//   '</ColorMap>' +
// '</RasterSymbolizer>';
// Map.addLayer(combination.select([0]).sldStyle(sld_intervals), {}, 'Sediment Pressure Zones');

// Export the image to an Earth Engine asset.
Export.image.toAsset({
  image: combination,
  description: 'toAsset_OW_sediment_pressure_zones',
  assetId: 'projects/resource-watch-gee/wat_070b_sediment_pressure_zones',
  maxPixels: 1e13,
  scale: 100,
  pyramidingPolicy: {
    'Y2002': 'mode',
    'Y2007': 'mode',
    'Y2012': 'mode',
    'Y2017': 'mode',
    'Y2020': 'mode',
  }
});

I am sure this code needs work.

One Answer

The collections in your code aren't shared, but one of the most likely candidates for running out of memory is this line:

var grwl_sub = grwl.filter(ee.Filter.bounds(riv_bounds));

If riv_bounds is a large and complex collection (seems likely), then trying to convert it to a single geometry is probably exceeding available memory. You might be able to get this to work using a inner join instead of a filter.

Answered by Noel Gorelick on August 14, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP