CyberGIS Software Component Overview: OpenTopography
Chris Crosby and Sriram Krishnan, San Diego Supercomputer Center
Please provide the information below in order to help bring CyberGIS participants up to speed on the motivations, goals, and technical aspects of your software development project. The information will help to ensure a common level of understanding by CyberGIS project members, and will help to facilitate discussions about use cases and technical issues related to proposed software integration.
OpenTopography facilitates community access to high-resolution Earth science-oriented, topography data, and related tools and resources.
Over the past decade, there has been dramatic growth in the acquisition of publicly funded high-resolution topographic and bathymetric data for scientific, environmental, engineering and planning purposes. Because of the richness of these data sets, they are often extremely valuable beyond the application that drove their acquisition and thus are of interest to a large and varied user community. However, because of the large volumes of data produced by high-resolution mapping technologies such as LiDAR, it is often difficult to distribute these datasets. Furthermore, the data can be technically challenging to work with, requiring software and computing resources not readily available to many users.
Aspects of the OpenTopography Facility related to the CyberGIS project are:
- Democratize online access to high-resolution (meter to sub-meter scale), Earth science-oriented, topography data acquired with LiDAR and other technologies.
- Harness cutting edge cyberinfrastructure to provide Web service-based data access, processing, and analysis capabilities that are scalable, extensible, and innovative.
- Promote discovery of data and software tools through community populated metadata catalogs.
- Partner with public domain data holders to leverage OpenTopography infrastructure for data discovery, hosting and processing.
Scientific use case(s) & user communities
OpenTopography’s primary user communities are geoscientists studying earthquake faulting, geomorphology, etc. Other users include commercial sector consultants, software developers (seeking sample data for testing), and government agency staff. Typically users visit OT to download and process data for a specific area of interest, with a primary interest in high-resolution DEMs derived from lidar point cloud data. Usage is typically a dozen or more jobs per day. See: http://www.opentopography.org/index.php/about/metrics
The technical goal of the OpenTopography Facility is to build a modular system that is flexible enough to support the growing needs of the Earth science LIDAR community. In particular, we strive to host and provide access to datasets as soon as they become available, and also to expose greater application level functionalities to our end-users, e.g. generation of custom Digital Elevation Models (DEMs), and hydrological modeling algorithms for generation of derivative products. Furthermore, we envision enabling direct authenticated access to our back-end functionality through simple Web service APIs, so that users may access our data and compute resources via clients other than Web browsers.
OpenTopography Web services are accessed via simple SOAP APIs. In order to build the Web services, we have leveraged the open-source Opal toolkit (http://www.nbcr.net/software/opal/). Opal is a toolkit that enables users to automatically wrap scientific applications running on cluster and Grid resources as Web services, and provides Web-based and programmatic access to them, without writing a single line of Web service code. Developers simply write an XML-based configuration file for an application, and deploy it as a Web service using Opal’s command-line deployment tool.
Opal installation and subsequent service deployment are straightforward and can be accomplished in a matter of hours for each new application. Opal provides mechanisms to manage and track job submissions, with the help of a back-end database. It allows monitoring of job and system status by providing charting tools out of the box. It also provides a “Dashboard”, which can be used by administrators to track system and usage information, and access automatically generated user interfaces that can be used for prototyping and debugging. Finally, Opal provides a number of ways to ensure that only authorized users are able to access our Web services. In our current system, we have configured Opal to restrict jobs based on IP addresses.
Currently, access to OpenTopography services via the SOAP API is limited to a restricted set of IP addresses. However, the Web services that could be conceivably be integrated with other CyberGIS services include:
- Point Cloud Access
- Access to lidar point cloud data in ASCII or LAS format given a bounding box, dataset ID and other filtering parameters. Depending on user preference, the output point cloud could be in ASCII or LAS formats.
- Generation of DEMs using a local gridding method. The service accepts input parameters such as grid resolution, search radius, input and output formats, and generates a custom DEM.
- Format Translation
- Translation of DEMs from Arc ASCII to GeoTIFF and IMG formats using the GDAL toolkit.
- Derivative Products
- Generation of slope and hill shade grids in GeoTIFF and IMG formats. Uses GDALDEM.
- Generation of hill shades, browser images and Google Earth image overlay files for visualization of data products.
More details at:
Software platform and languages
OpenTopography runs in a Linux environment (with a few small exceptions). Code is primarily written in C++ and Java. OT strives to use open source code libraries when possible, including GDAL, libLAS, etc.
Downloadable code, libraries, SVN (with URLs)
OpenTopography is a hosted software system where we provide our users with data access mechanisms and processing software hosted at SDSC. Our code is currently not available for download, although it could be packaged and made available to CyberGIS partners. Depending anticipated processing load and user demands; OT services could be deployed on CyberGIS compute resources (e.g. CyberGIS Virtual Machines) to enable R&D outside of the production OpenTopography environment.
Key project staff involved in cybergis
- Sriram Krishnan (email@example.com), OpenTopography SOA Architect. Currently SDSC’s CyberGIS Technical Coordinator
- Chris Crosby (firstname.lastname@example.org), OpenTopography Project Manager.
- Choonhan Youn (email@example.com). Will be primary OT developer on CyberGIS (starting in March 2011)