1  Helpers

2 Overview

The helper module is design to handle features or operations that will be utilized by several modules. Currently, the module handle data output specially raster based data. This operations is utilized by module 1, 5, and 6. Furthermore, there are several options for data output namely, direct download, google drive download, and cloud storage export. The complex export options are wrapped by helpers module for ease integration with front end code

Input

Name of Input Input Type Details
Earth Engine image (ee.Image) Module output magery composite (module 1), predictors (module 5), and classification image (module 6)
Export parameters User’s input File name, destination, projection, and scale (pixel size)

Output

Raster based data (GeoTiff)

3 Process

3.1 Earth Engine Export Utilities

3.1.1 Export Parameters

Prior to export any earth engine image, the user must define several parameters required for export

Luma User Journey

  1. The user select which export option they are going to use

  2. The user define the required parameters for export. These parameters are as follows:

    1. File name (default file name is provided according to each module output)

    2. Spatial resolution (default to 30 m, since Landsat mission is the only one supported)

    3. Coordinate Reference System (CRS). Use WGS 1984 (EPSG:4326) as the default value. If the user wish to use different CRS, they must specified the EPSG code (i.e ‘EPSG:4326’, ‘ESRI:54009’, or ’SR-ORG:6864’)

    4. For module 5, the user select which predictors to be exported. The predictor can be selected one by one, stacked of predictors, or by category

    CautionFeature Note

    Current version of Luma GE only support direct download, which limits the data size to 32 MB. Google Cloud Storage (GCS) and google drive download are not supported in this phase.

Luma Geospatial Engine

  1. The system creates a predefined default names according to the module and dataset exported.

  2. Extract the geometry from the selected AOI, as prerequisites for export

  3. Validate scale and CRS and raise a helpful error if invalid scale or CRS is detected

TipRelated Function

helper.EE_Export_Manager.santize_export_name(): Standardize user input file name according to the system need. Replace invalid characters for GCS, Drive, and local file system

helper.EE_Export_Manager.extract_geometry(): Extract the geometry from ee.featureCollection(AOI) for export purpose

helper.EE_Export_Manager.validate_scale(): Validate user input pixel size value, design to avoid negative value what might cause the system to break. Default value use 30 m

helper.EE_Export_Manager.validate_crs() : Perform semantic validation for user input CRS code. The CRS content itself is not validated. Default values is WGS 1984 (EPSG:4326)

helper.EE_Export_Manager.validate_image() : Validate if the object pass to export manager is an ee.image

helper.EE_Export_Manager.validate_export_params() : validate the entire export parameters, raise helpful errors if detected

3.1.2 Export Process

3.1.2.1 Direct Download

Luma Geospatial Engine

  1. The direct download function in Luma-GE used Earth Engine’s ee.Image.getDownloadURL function, which generate a link for small chunks of image data in GeoTIFF or NumPy format. Maximum request size is 32 MB, maximum grid dimension is 10000.

  2. The parameters used for the export is passed from user input

TipRelated Function

helper.EE_Export_Manager._export_direct_download : Generate a data download link for direct download option

helper.EE_Export_Manager.export_image : Function to initialize earth engine export with input parameters validation while supporting several export option

3.1.2.2 Google Cloud Storage Export

To be developed

3.1.2.3 Google Drive Export

To be developed

3.1.3 Export Task Monitoring

Luma User Journey

  1. The user can monitor the status of their export directly in Luma

  2. If there’s no update within 1 minute, they can refresh the task monitor to acquired the latest update of current task status

  3. The user receive feedback regarding the status of the export task (in progress, complete, failed, etc)

Luma Geospatial Engine

TipRelated Function

helper.EE_Export_Manager.get_task_status(): Return task status using cache to reduce API calls

helper.EE_Export_Manager.get_active_task(): Return activate task information, especially the “ready” and “running” state

helper.EE_Export_Manager.remove_task(): Remove task from local tracking without affecting the task in earth engine task monitor

helper.EE_Export_Manager.remove_completed_task(): Remove all completed task for future monitoring. The removed task include all of the terminal states (complete, failed, or canceled export task)