Software:PGIST

From CyberGIS

Jump to: navigation, search

Contents

Participatory GIS Technologies

Participatory technologies are collections of tools that enable participants to express and synthesize ideas to reach a consensus about a specific goal.

Let’s Improve Transportation and Voicing Climate Concerns Overview

Let’s Improve Transportation (LIT) and Voicing Climate Concerns (VCC) are two online participatory decision making experiments conducted at the University of Washington, aimed to enable large-scale, asynchronous participation of a diverse group of actors in a decision making process. Delphi and Technology of Participation heavily influence both processes, but there are significant modifications.

As decision-making methodologies, LIT and VCC employ the maxim of generate ideas, synthesize them, and reach a decision about the best possible scenario. In the context of a participatory, asynchronous methodology, all three steps are altered to enable multiple people to provide their input – either in the generation of ideas, their synthesis or reaching a consensus. The following diagrams illustrate an overview of both LIT and VCC.

PGISTLitWorkflow.png PGISTVccWorkflow.png

The collection of processes shown above is termed an experiment, in which the processes are instantiated and pushed on a workflow engine. The processes are linear, non-simultaneous and asynchronous, depending on input from the previous process to achieve their function. It is possible to feed your own data to each process to avoid dependencies on previous processes.


User Interface

LIT and VCC employ a web interface for all the processes shown in the diagrams above. Each step (box in the diagram) is itself presented through a web interface. This is achieved using web forms for the participants’ input, and simple, HTML presentation of their contributions, either in aggregate or individual. Some enhancing of the presentation is achieved using Javascript (e.g. to enable sorting), through the use of jQuery and Scriptaculous. Some of the functionality, like categorizing ideas and prioritizing them are innovative designs that emulate desktop applications to achieve their function, through the use of Javascript and direct manipulation of the Document Object Model (DOM).

API and Libraries

As dynamic web applications, LIT and VCC define their functionality through JavaServer Pages (JSP). Extensive use of Java and the Java Standard Libraries provides functionality for each process, including the input, processing and storage. Some external libraries are used, like Taglibs and Direct Web Remoting (DWR) that enables the client (browser), through the use of Java and Javascript, to call server-side functionality. Taglibs call Action classes and DWR calls Agent classes. The storage needs are taken care of through PostgreSQL, with the PostGIS extention libraries. Translation between the Java code and the database is performed through Hibernate (an Object-Relational Mapping library) automatically. Apache Lucene is the library used for full-text searching functionality.

There is also use of Jython (a Java implementation of Python) for some of the logic processing of user input. The remaining of the middle-ware is itself defined through the use of JSP and Java.

The front-end is served by Apache Tomcat, which communicates with the application through JSP and the above libraries. Enhancements of the presentation are achieved using common Javascript libraries, including jQuery and Scriptacilous, as well as some custom Javascript functionalities that enable customized sorting and tag cloud implementations.

Most functionality provided by JSP (the majority of the code) is also callable through the web, using a service-like infrastructure enabled by Taglibs. The API documentation is generated using Javadoc with any new functionality added. The source code for both LIT and VCC is maintained through a Concurrent Version System (CVS), restricted to the project members.

Architecture

The following diagrams illustrate the client-server architecture described in the previous section:

Overview of PGIST Architecture

ArchDiagram.png


Client-Server Architecture with Taglibs (Actions) & DWR (Agents)

BS-Ajax.png


Model-View-Controller Detail

MVC-structure.png

Web Service(s)

While there is no current web service made available to the public, LIT and VCC use a service-like architecture to enable the client (browser) to call functionalities on the server through the use of JavaScript (DWR) and taglibs. PGIST requires a rich graphical user interface for participatory, structured deliberation, which is better developed through web application technologies such as DWR, taglibs and Struts. Each process is dependent on the input of the previous process and is encapsulated in a workflow engine, which is supported by the Struts MVC architecture. Web services could be developed to query the results of the deliberation (keywords and categories generated), but are not suited for the GUI-dependent, user driven, dynamic deliberation process.

Open Source Aspects

LIT and VCC are built on top of open source applications, including Apache Tomcat, PostgreSQL, lucene, hibernate, taglib and more. Currently, neither application has their source code released under any Open Source Initiative (OSI) Approved license. However, this is a desired outcome for both LIT and VCC.

Integration Activities

The following content documents early integration efforts, that focused on a loose coupling of PGIST and the CyberGIS Gateway.
We have now moved to an integrated approach, which is described here: PGIST SPT Implementation

DWR

Q&A from 6/9/11 telco

Q1. In assessment step, the javascript library use DWR, what is DWR and how to use it?

A1. DWR stands for direct web remoting/EasyAJAX. See architecture diagrams above for high-level overview of DWR in PGIST. DWR allows Javascript and server-side Java code to interact. The DWR Agents perform much of the logic processing work in PGIST (versus the Action classes, which appear to handle view-oriented tasks).

PGIST uses DWR version 2.0.1, the current stable version is 2.0.6. Download DWR 2.0.1.

See the following for more information on using DWR: Quick Start and Scripting with DWR

Q2. Any tool library documentation?

A2. See PGIST JavaDoc. Via the 'All Classes' view the various DWR Agent implementations are easy to discern (the XXXAgent classes).

Q3. Does DWR uses http post or get for requesting server-side functions? How to set properly the default path to DWR engine (Gateway might need a proxy php to redirect dwr ajax calls to PGIST server to conform same-origin constraint of ajax)?

A3. DWR uses http POST when on same machine and GET for cross domain communication.

See this developer thread, starting with the entry from Feb 18, 2011; 5:17am for information about setting the default path and communicating with DWR remotely: Cross Domain DWR

See here for information about the crossDomainSessionSecurity init parameter DWR Servlet Config.

PGIST ToDo: Add this parameter, set up an example page for remote client access and share with UIUC.

Q4. What is the format for returned data on each library call?

A4. See scripting link in A1. Data is returned in a call options object in a callback function. Will provide more documentation with example client page.


Security

Tue, Jul 19, 2011 at 8:27 PM, Yan Liu <yanliu@illinois.edu> wrote:

   My picture of how Gateway-PGIST interactions work:
   1. Establishing trust between Gateway and PGIST. Gateway is a consumer of PGIST services. To build trust, there is a secure conversation that has to happen regularly between Gateway (G) and PGIST (P):
   G->P: hello, this is Gateway. I need a 7-day token from you for accessing the CyberGIS instance on PGIST. Here is my SSL certificate.
   P->G: (checking IP address of request from G). Yes, you come from the correct IP. I also verify that your cert says you are who you are. Here is the token: AABBCCDD, valid for 7 days
   G->P: Got it. I'll come back later if I need to access the instance in the future

See here for Documentation of Tomcat server/client SSL config File:TomcatAuthenticationSetUp.docx

   2. IFrame-based user (U) interactions with CyberGIS instance at PGIST through Gateway
   U->G: login and navigate, click "I have comments" button
   G: pops up a dialog in which the html code looks like attached html. key is the URL
   P: read token and cybergis_userid from the request URL; check whether this token is valid
   (optional: G can also send a callback url to P along with token and userid parameters; P, upon receiving the req, use the URL to verify the user is logged in to Gateway)
   P: if it is valid, check if PGIST has this cybergis user. If not, create it and give it default permission set for cybergis
   P: if it is valid, and user is in PGIST user db, send back a redirect HTTP response pointing to the CyberGIS instance with session id. If not valid, returns nothing

Feedback Component

Requirement

Based on several discussions, including the 12/8/2011 Integration Telco, it appears that only the BCT (Brainstorm Concerns Tool) is currently desired for the PGIST-CyberGIS Gateway integration. Two technical options have been identified for achieving this, which are described below. OPTION 2 has been selected as the implementation approach.

The BCT will be used for feedback on the Gateway and elements (applications and eventually data) integrated in the Gateway. Feedback will be captured and maintained separately for each element and the Gateway.

The Filterfunctionality in BCT has been adjusted to only show the feedback related to the selected element, unless the user chooses to 'See only my feedback' or searches for a specific keyword. This allows for a multi-dimensional sorting - either by element or by user-defined criteria. When the user removes their filters, then the feedback is again filtered by element.

Implementation Options

These options have been developed with the following considerations:
1. PGIST is a user-oriented web application. The user interface is implemented in DWR/easy AJAX to allow for seamless updating of content. It also utilizes JSP tag libraries to access server-side functionality.
2. Each component, BCT/CCT/CST, was developed as part of an entire structured discussion (i.e. Experiment) workflow, and rely on workflow, context (workflow step) and activity (workflow substep) initialization.

OPTION 1: Integration of PGIST user interface directly into Gateway
For each application in MyApps(?) feedback button or link would open a feedback/comment interface(where? separate page? seems like it should not replace the current My App or My Analysis and Modeling frames). If just allowing user to enter information, could possibly be a small frame under the application description on the left side of Gateway frame; however if also displaying other users comments and keywords then there does not seem to be enough space. See image below for minimum space requirement example, only showing one other feedback and no further comments: BCTPage.gif

Implementation Issues:
1. Cross-domain DWR (see above DWR section, especially A3) seems to be limited, problematic and not recommended.
2. The BCT JSP also uses taglibs to invoke server-side functionality. It does not appear that these can be deployed on a different server than the implementation code.
3. PGIST requires feedback to be associated with a specific user. Clicking the 'Feedback' button would need to call the LoginAction on the PGIST server, which puts User info in the session to be used by the JSP page. However, if the JSP is being loaded in the Gateway frame, unclear how to modify the response session to include this information. Note: for the BCT page the bct.id, associated with a user, may be sufficient, but this information still needs to be available in the response.
4. The Gateway appears to be implemented in PHP, and I am unfamiliar with the PHP server environment. Can it also serve a Java web application? Or will the page be re-developed in PHP?

Generally, it is unorthodox (and perhaps only possible with difficult workarounds, if at all possible) to separate the GUI (deploy it to a different server) from the server-side implementation code in a Java web application. It seems necessary to re-engineer the GUI to work in a different technology anyway if integrating directly into the Gateway, which will also require reimplementing server-side logic used by the page via taglibs as well.

OPTION 2: Gateway invokes CyberGIS-custom PGIST application (Current Implementation Approach)
Based on previous discussions about DWR (see above), remote(from a different server) access to server-side logic, and the difficulty of re-implementing the PGIST GUI in a different technology, we had decided to pursue integration via I-Frame, which would open PGIST in a new browser page. Perhaps, based on the desire to only use the BCT functionality, it makes sense to try for a more direct integration described in Option 1. However, if the feedback interface will open in a separate page anyway (so that the user does not lose their current analytical environment and so that other users' feedback can be viewed and voted upon), it seems we can continue with this option.

The automatic login and system initialization has already been implemented and deployed. I also have the direct access to BCT (skipping over the Experiment and Agenda page) working in my development environment, but need to make the solution more systematic based on the Implementation Specifics described below. I also need to customize the GUI so that it makes sense within the CyberGIS context, but this requires minimum effort.

The other benefit to this approach, aside from being nearly ready, is that we can expose other aspects of PGIST (CCT/CST) if we decide to support full, structured deliberation about either CyberGIS functionality or research topics.

Implementation Specifics:
1. Feedback for the Gateway and each software element (Apps & eventually Data) will be associated with a unique Experiment. The Experiment is the logical information container in PGIST. In the normal workflow, categories and category hierarchies are created after the brainstorming/keyword step.
For example: All feedback for the Spatial Point Generator will be associated with an Experiment. After a period of time for feedback, the step could be closed and the generated keywords could be grouped into categories such as 'Documentation', 'User Interface', 'Performance', 'Functionality'.
Implication: It will not be possible to view all feedback for the Gateway and elements at one time. New functionality will be needed to combine these into one overview.

2. I will create a separate experiment (which is identified by a workflow_id) for the Gateway and each software element as well as anything else that should receive unique feedback. I will provide a list of workflow_ids (and associated context_id & activity_id) to the developers.

3. The Gateway developers will add the specific feedback links to the Gateway interface that will open the feedback page in a new window.
Link example: https://pgistdev.geog.washington.edu:8443/login.do?domain=cybergis&token=%TOKEN%&loginname=%USERNAME%&workflow_id=%WORKFLOWID%&context_id=%CONTEXTID%&activity_id=%ACTIVITYID%
Note: To simplify the link I could look up the workflow/context/activity based on the element name. However, since both the login needs to be processed (including checking if already logged in), and the feedback page initialized and loaded, an extra look-up seems like an unnecessary processing step.

Feedback Separation Considerations

While implementing the above solution described in OPTION 2, a few issues arose that require some thought. Initially we thought we make the separation of feedback happen behind the scenes. When the user clicks on the feedback link, that feedback would be associated with the specific element that 'owns' the link in the Gateway.

ISSUE 1: The user may want to provide feedback on multiple elements (e.g. the Gateway and the spatial point generator) at one time once they are in the 'Feedback' mode. Perhaps they clicked on the link in the Spatial Point Generator App, but don't realize that we only expect feedback related to that App. Requiring the user to return to CyberGIS and find the appropriate link to click, in order to provide associated feedback seems to be an undue burden. Likely, they will enter whatever feedback comes to mind, regardless of the context from which they accessed the page.

ISSUE 2: We may want to group feedback by users rather than elements, in the case of classroom use?

One solution may be to have one standard feedback link in the Gateway (rather than multiple unique links), and send users to the Main overview page (see below) so they can select what context/which element they want to give feedback on. Then it is clear that we want to maintain feedback for different elements/and possibly groups separately. When the user has finished one feedback session, they can click on Home to return to the main page to give feedback on another element. We can still SKIP the agenda page, with all of the steps, and go directly to the feedback page. This would address initial concerns that there were too many steps to get to the feedback.

ExperimentOverview.gif

When they click on Participate, users would go straight to the Feedback page. I can also change the button text as well as the 'Running experiments' heading.

FeedbackPage.gif

BENEFIT 1: One link in the Gateway, not multiple, that have to be added as new elements are added.

BENEFIT 2: What they are commenting on will be transparent, rather than hidden, from the users, so perhaps more likely to cooperate in using different feedback 'experiments'.

BENEFIT 3: Core team members can receive admin accounts and define their own feedback 'experiments'.

Implementation

Status 4/5/2012

We are currently implementing a modified version of Option 2 described above:
1. One Feedback experiment (defined by workflowid) for all components. This allows users to provide feedback about different aspects of Gateway (overall, elements, data) without having to go back to the 'Home' page and select a different experiment.
2. Gateway users are sent directly to the Feedback page from Gateway.
3. Feedback is pre-categorized by using a select box above feedback input field.

Immediate TODOs
1. Fix the Login page redirect that it occurring because of POST request.
2. Use a variable in the HTTP request to preselect an element from the 'pre-categories'. This can be set programmatically from the Gateway, but could initially be hardcoded for testing.
3. Automatically filter comments by 'pre-category'. User can also select a different element to give feedback on, and the comment list will be updated accordingly. Also need to clarify how the current filter functionality works. I thought it was by keyword.
4. Take out 'Home' link to prevent user from navigating to other parts of application.
5. Change look-n-feel to match CyberGIS Gateway stylesheet.
TODOs 1-4 are highest priority. Will work on these today and tomorrow.

Synthesis TODOs
In addition to capturing feedback, we want to provide team members access to the synthesis tool, with which they can further structure the feedback they are receiving about their element. They could add categories such as 'Documentation', 'Usability', 'Performance', etc... to group keywords/feedback. These categories can then be developed into metrics.
The following changes need to be made:
1. Make the workflow non-linear. Currently the feedback step has to be closed before synthesis can begin.
2. I have changed the synthesis tool to load feedback by the 'pre-categories'. However, each user only sees their own feedback and has to select a different user to see that of others. Need to change to show all feedback, for all users, by pre-category. These changes need more thought and will be addressed after the immediate todos have been resolved.

Status 4/19/2012

List corresponds to above.
1. Redirect fixed, must set Cookie in response.
2. Open
3. Filter functionality changed to filter by selected element unless the user selects 'See only my feedback' or searches by keyword. When user removes these filters, then view returns to default filtering by selected element.
4. Open
5. Open

Also working on obtaining a real security certificate for PGIST server through UW.