OpenTopography Services for CyberGIS: Digital Elevation Model (DEM) generation
The OpenTopography Facility uses a Service-oriented Architecture (SOA) to provide access to high resolution LIDAR datasets, and also to expose application level functionalities to our end-users, e.g. for the generation of custom Digital Elevation Models (DEMs). In order to build the Web services, we have leveraged the open-source Opal Toolkit, which enables users to automatically wrap scientific applications running on cluster and Grid resources as Web services, and provides Web-based and programmatic access to them, without writing a single line of Web service code.
OpenTopography users interact with the system via a Web portal (http://opentopo.sdsc.edu/gridsphere/gridsphere?cid=datasets). An interactive Google Map interface showing the extent of available data allows the user to define a rectangular area of interest by drawing a box on the map. The coordinates of this bounding box plus optional attribute type (e.g. ground points only) form the user’s query (implemented by the Data Selection service). Once a query has been defined, the user chooses their processing options. Point cloud data download for the user’s area of interest is available in ASCII or LAS binary formats. The user can also pass the subset of their selected point cloud to a series of geoprocessing services that will process the data on the fly to create custom DEMs and derivatives tailored to their science application. Using a high-performance local gridding algorithm (implemented by the Points2Grid service, described below), OpenTopography will produce a DEM to the user’s grid resolution and algorithm specifications. The user can also choose the format of the grid product that is generated (implemented by the Format Translation service) so that the file is compatible with their preferred analysis software. Finally, the user is given the option to generate common geomorphic metric products such as a slope grid derived from the DEM (implemented by the Derivative Products service). OpenTopography also generates visualization products (implemented by the Visualization service) on the fly for each submitted job and embeds these images on the results page.
A detailed description of the OpenTopography SOA, and more details about the workflow can be found on our SDSC Technical Report.
DEM Generation using Points2Grid
Points2Grid uses a local gridding algorithm to compute grid cell elevation using a circular neighbourhood defined around each grid cell based on a radius provided by the user. This neighbourhood is referred to as a bin, while the grid cell is referred to as a DEM node. Up to four values — minimum, maximum, mean, or inverse distance weighted (IDW) mean — are computed for points that fall within the bin. These values are then assigned to the corresponding DEM node and used to represent the elevation variation over the neighbourhood represented by the bin. If no points are found within a given bin, the DEM node receives a value of null. The P2G local gridding algorithm is illustrated in this figure. Points2Grid also provides a null filing option, which applies an inverse distance weighted focal mean via a square moving window of 3, 5, or 7 pixels to fill cells in the DEM that have null values.
Given that natural topography is not dependant on regional trends (i.e. the elevation at a given point is not influenced by the elevation elsewhere in the region), a local gridding methodology for topographic measurements is appropriate. The nature of LIDAR point cloud data also lend themselves to a local gridding approach. Because these data are collected at 10s to 100s of kilohertz from a low-flying aircraft, they typically sample elevation at a spacing of significantly less than a meter. However, researchers typically perform their analyses on DEMs with resolutions of half a meter or more. Thus, in many research grade LIDAR datasets, each pixel in the DEM may have been sampled several times. Thus, fitting a surface between points to estimate elevation at the DEM node is not required, and induces unnecessary computational burden. A detailed description of the performance of the gridding algorithm can be found in our IEEE CloudCom 2010 paper.
The Points2Grid service has been implemented by wrapping the Points2Grid code (written in C++) using the Opal toolkit. It accepts input parameters such as grid resolution, search radius, input and output formats, and generates a custom DEM.
Points2Grid on CyberGIS VM
The production version of the Points2Grid service is hosted on servers at the San Diego Supercomputer Center (SDSC). For CyberGIS, we have developed a beta/development instance of the Points2Grid service on a CyberGIS Virtual Machine (VM), hosted on its own Opal server (http://cybergis.sdsc.edu:8080/opal2/dashboard). To use the Points2Grid service, follow these steps:
- Prepare an input dataset that you would like to grid. P2G will accept both LAS binary point cloud files and comma delimited ascii data. You can obtain smaple data from OpenTopography by running a custom job to create your own datasets at http://opentopo.sdsc.edu/gridsphere/gridsphere?cid=geonlidar. For instance, you might want to choose the RIT Haiti Earthquake LIDAR Dataset. Alternatively, you could just use the sample LAS data file. ASCII format point cloud data must be one-point-per-line, comma delimited where column one is x (easting), column two is y (northing) and column three is z (elevation). The presence or lack of additional attribute columns is unimportant since Points2Grid only looks at the first three columns of the file. Delimiters other than comma (or comma followed by a space) will cause Points2Grid to crash. A header on the input file is not necessary but if one is present, Points2Grid will not be affected.
- From the Opal dashboard (http://cybergis.sdsc.edu:8080/opal2/dashboard), click on the List of Applications tab. Then select the link for the Points2Grid service.
- Either upload your input file to Points2Grid, or provide a link (e.g. ).
- Set the input format correctly. If you are using the sample file above, set it to las. If you are using an ASCII dataset, set it to ascii.
- Selection search radius and grid resolution - a value of 1 works for both of those fields.
- Submit your job.
After job submission, you can monitor progress and download results via the Opal dashboard.
The Opal Dashboard provides a Web-based interface to access the Points2Grid service. For the next steps, we should use the SOAP API for the Points2Grid service to access the service programmatically. Using this API, the Points2Grid service will be easily accessible from the CyberGIS Dashboard.