Architecture Goals

The nGen platform requires integration of distributed heterogeneous applications into a common UI "portal". 

Evaluation Criteria

• Performance/Efficiency
  • Chosen solution must be able to handle up to 10 "widgets" per page with a minimum of server hits.
• Portability/Reusability/Security
  • Can "widgets" be reused for different functional needs.
  • Are "widgets" deployable in different environments.  A truly reusable widget can be deployed in any hub as well as external environments such as iGoogle or Sales Force.
    
  • If a "widget" is deployed outside on another hosting application, does the architecture support locking down data access with user approval.
• Developer Productivity
  • Widgets architecture and best practices should be quickly learned
  • Widget libraries and toolkits should either be available or easily produced by the Platform Team.
• Interoperability
  • Selected architecture needs to work seamlessly across a technology agnostic deployment.
  • At the very minimum, it should be supported by the currently supported environments of Ruby, Java, and .NET.
  • Different widgets on the page should be protected from "overwriting" each other.
• Scalability
  • Selected architectures should support scaling on multiple dimensions of the AKF scale cube

Candidate Solutions

JSR-168 portlets

JSR-168 is a java specification for providing interoperable "portlets" to any portlet container.  A portlet is a pluggable user interface software components that produce html fragments that are aggregated via a portal page hosted by the portlet container.   The specification defines a set of APIs that must be implemented by the portlet container that addresses areas of personalization, presentation, and security.

Similar to Java Servlets implementing the class javax.servlet.Servlet, portlets are built by defining Java classes that implement the interface javax.portlet.Portlet.  Portlets are deployed to portlet containers as war files with a portlet.xml defining portlet classes.  Therefore, the solution is inherently java specific, and requires all portlets be deployed ultimately on the same server as the portal container.

Evaluation

• Performance/Efficiency
  • Calls to the server are minimized because all content is typically aggregated server side.
    
  • Responsiveness is often suboptimal because the page requires all portlets on a page to finish building.  If even one portlet is slow to load, it delays the entire page.
  • Some of the open source portlets serially aggregate a page because of thread security issues.  This causes total rendering time to be based on "n" widget load times.
• Developer Productivity
  • JSR-168 API should be very familiar to any Java servlet developer, as the two object models are very closely related.  Also, the portlet model emphasizes clean separation between "GET" rendering and "POST" processing actions.
    
    • javax.portlet.Portlet interface must be implemented just like javax.servlet.Servlet.
    • RenderRequest and ActionRequest replace ServletRequest with almost identical method sets.  However, RenderRequest methods are optimized for "GET" viewing, whereas ActionRequest are optimized for "POST" processing.
  • Most portlet configuration is handled through a portlet.xml configuration file.  This file is somewhat similar in both use and structure as a web.xml is for servlets.
  • Portlets are typically delivered as war files similarly to servlets.
  • JSR-168 is widely supported throughout the Java industry and has several open source solutions available, including:
    
    • JBoss Enterprise Portal
    • Liferay, Apache Pluto
    • and its more feature rich extension Apache Jetspeed.
• Portability/Reusability/Security
  • The portlet API is a Java standard designed with portability/reusability in mind.  Any portlet conforming to the spec should be portable to any other portlet container.
    • The only caveat to this is the war file requirements are not well defined in the spec.  Therefore, the structure of the web.xml file typically differs between portlet containers.
  • Portlet security using the same role based Java EE security as servlets.
• Interoperability
  • This is a purely Java solution, no platform or language not supported by the Java VM will work.  Therefore, ruby and .NET are not supported.
• Scalability
  • Portlet containers can have strong X-axis scalability via standard Java EE clustering.  However, its requirement to run all portlets from the same container strongly hinders both Y and Z axis scaling.

There are further limitations with portlets:

• No support for inter-portlet communication.
• No support for Ajax.
• Limited control of portlet caching.  (Portlets may only specify an expiration cache for an entire page in its configuration).
  

WSRP Portlet

Web Services for Remote Portlets is an OASIS approved specification for communicating with remote portlets via Web Service calls.  Portlets following this spec are similar to JSR-168 portlets in functionaility.  WSRP portlets rely on a server-side aggregator to build out a page from several disparate portlets just like JSR-168 portlets.  However, instead of relying on Java Servlet technology to communicate with the portlets, WSRP portlets are instead accessed remotely via SOAP calls.  Therefore, the spec supports both the container and portlets to be language/platform agnostic.  Also, because WSRP spec matches the JSR-168 spec, it is trivial to get portlets to interop, and most portal containers support both specs.

The 2.0 version of the spec was approved on April 1st, 2008.  However, portlets must still implement SOAP web services, and Ajax support is not built in to the spec.  In order to provide Ajax support, a WSRP server would need to provide a proxy server for the portlets to communicate through. Furthermore, the specification does not specify inter-portlet communication.

Evaluation

• Performance/Efficiency
  • Calls to the server are minimized, because all content is typically aggregated server side.
  • Every page refresh requires the server to communicate with all distributed portlets.  Plus it runs into the same trouble as JSR-168 portlets will all content being aggregated before the page can be displayed.
  • Since AJAX is currently not supported in the spec, all form posts of any portlet require a full page refresh, thus hitting all portlet containers again.
• Developer Productivity
  •  Each portlet must provide the follow sets of SOAP based Web Services in order for the container to communicate with the portlet:
    • Service Description - Includes 1 operation to provide a static xml meta-data about the portlet<br /> o Markup - Includes 3 operations including getMarkup which contains the rendered html fragment for the portlet.
    • Registration - Includes 3 operations for a server to register, update its registration, and deregister from a portlet.
    • Portlet Management - Includes 6 operations for getting/setting porlet properties and descriptions, as well as cloning and destroying a portlet.
  • The Java platform provides 2 frameworks for writing/hosting WSRP portlets.
    • WSRP4J
    • OpenPortal WSRP
  • NetUnit provides a WSRP Framework for .NET
  • All other languages will need to implement the above mentioned SOAP web service operations.
    
• Portability/Reusability/Security
  • Any WSRP portlets should work with any WSRP aggregate.
  • Most JSR-168 portals will support WSRP portlets.
  • Security is not defined in the spec.
• Interoperability
  • Both Java and .NET have implementations of the spec and have been demonstrated to work together.
  • Currently no other languages are supported, but building a custom portlet lib or aggregator could be done to support other languages such as ruby
• Scalability
  • The spec supports Y axis scaling, and depending on implementation may support Z axis.

nGenera Widget Framework

A framework developed in house by the platform team using AJAX to dynamically load HTML fragments.  Structured around making the resulting widget display inline, without the requirement of an iFrame.  This allows the resulting widgets to be first class citizens on the resulting page.  Makes use of AJAX calls to a proxy servlet to load remote data restfully, however, requires a developer add proxy methods per request.  Does not require any independent server, the fragment is parsed and loaded all through client side javascript.

Evaluation

• Performance/Efficiency
  • Every widget on the page will require a separate server hit, no caching is enabled.
  • Since calls are done through AJAX, pages can load immediately and render asynchronously.
  • Because of its heavy reliance on AJAX, page refreshes should be minimized limiting hits on server.
• Developer Productivity
  • The framework provides a very light learning curve as a developer writes their widget as a standard HTML page. The only extra integration with the framework occurs when the "onload" method of the body is invoked
• Portability/Reusability/Security
  • The framework is propriety, and must be maintained by the platform team.
  • All proxy calls must be supported by every server that is displaying the widget.
  • No Security is defined.
• Interoperability
  • The framework is purely client side, therefore any language/platform can serve up widgets.
• Scalability
  • Although widgets can load from anywhere, and data can be retrieved from any other server, it does not support unlimited Y axis scaling because all calls must go through a proxy server that exists on the server displaying the widget.

OpenSocial Gadgets

OpenSocial gadgets is a specification for mini-applications built using HTML, JavaScript, Flash, etc.  Designed to run on multiple sites and platforms.  They are supported by a number of existing platforms including: iGoogle, Yahoo!, MySpace, Friendster, LinkedIn, Sales Force, Oracle, etc. OpenSocial gadgets contains a suite of JavaScript libraries to assist a widget developer.  Each library is provided through an extensible feature framework.

OpenSocial gadgets are typically rendered through an iframe pointing at a gadget server.  The gadget server loads the gadget's module specification from its server and renders the page in the iFrame.  The loaded gadget on the client side can then communicate with the platform or to remote resources via a builtin proxy server.

OpenSocial gadgets contain many features out of the box, including implicit data conversions and caching on AJAX requests, back-end security through OAuth. OAuth is an open protocol to provide secure authorization of user data.  For example, if the user service implemented an OAuth service provider, then any gadget that needed access to user account profile information would first need to get user authorization to access their user information.  This becomes useful if gadgets are hosted externally to nGenera and therefore cannot be implicitly trusted.

Currently there is only one available full implementation of the OpenSocial spec, Apache Shindig.  Shindig is the reference implementation of Open Social written in both Java and PHP.  It was started in December 2007, and had full support for OpenSocial 0.7 by 2/1/2008.  Its currently still an Apache Incubator project and therefore has no official release dates yet.  Friendster is the current largest user since 8/20/2008 pushing the number of users to over 90 million.

Also, both JBoss Portal and Liferay provide implementations for deploying gadgets on their portals.  However, only Liferay supports rendering specific portlets outside of a portal page.  Therefore, Liferay could be use as an alternative to shindig for rendering gadgets.  However, this solution would require the overhead of deploying a full blown portal of which we would only be using a fraction of its total functionality.

For ruby on rails, there are currently two project devoted to open social. 

• OpenSocial plugin - A rails plugin created by ELC Technologies that provides generators to add open social persistence to any rails project.  It creates the RESTful backend for open social entities as well as javascript assets to access them.  However, it does not provide appear to provide support for gadgets to call these APIs.  Also, the opensocial javascript files are all pulled from the shindig project.  According to their latest progress report they are currently at support for v0.5 of the opensocial spec.
• rSocial - A rails project to provide an OpenSocial host according to its announcement.  However, after a year of development with no updates, the project appears to be dead.

The OpenSocial gadgets specification currently contains 67 methods across 13 features.  The methods range from as little as setting preference values to a full JSON object parser.  In addition, the specification requires a gadget container capable of parsing a gadget's module xml configuration file and providing the rendered gadget on request.  Also, a cachable proxy server must be provided for ajax support.

Evaluation

• Performance/Efficiency
  • Although every gadget on a page will require a separate server hit, shindig supports caching of the module XML as well as browser caching of the rendered gadget.
  • All calls through AJAX likewise support caching dramatically reducing the number of server hits.
• Developer Productivity
  • Gadgets consist of a light weight XML around HTML fragments.  Developers mainly write gadgets as HTML fragments using the JavaScript libraries for getting data or page navigation.
  • All javascript libraries are added by shindig automatically, and data caching is likewise automatically handled by shindig.
• Portability/Reusability/Security
  • OpenSocial gadgets that conform to only the core feature set, are fully portable across any other server supporting the spec.  For example, OpenSocial gadgets could be deployed as is into Sales Force or iGoogle.
  • OpenSocial supports both request signing and the OAuth spec for user enabled data security.  This type of security allows users to approve that applications can see their data.
• Interoperability
  • Although shindig is only supported in Java and PHP, the gadgets themselves are written in HTML/JavaScript and can thus be hosted in any language/platform.
•  Scalability
  • Gadgets can be loaded from any server.  Data loaded through gadgets is proxied through shindig.  This data can also be cached for good X axis scalability

Recommendations

Based on the Widget Architectures Comparison Matrix the platform teams recommendation is to proceed with OpenSocial gadgets.

JSR-168 portlets are a Java only solution, and the lack of AJAX support as well as poor server scaling make either solution unattractive.  Apache shindig provides a full implementation of a well thought out spec that can be deployed in environments outside of nGenera.  This provides more flexibility in both what nGenera can host as what can host for nGenera.  Also, it limits development needs as the spec is already written and proven, as opposed to the nGenera Widget platform.

The following image displays the proposed architecture and flow during a gadget request: