Apache Wave Proposal (Apache Incubator)

This is a cross-post from my blog on the Military Social Networking system milBook. I have also started a new blog on the new online space for FedWave here, which is restricted access for the time being.  

As a reminder, there is an online community for Federal Wave (FedWave) located here. You must apply for membership first at APAN.org. I will approve all reasonable requests for group access.

The Wave in a Box project has submitted a proposal to the Apache Foundation to be admitted into the Apache Incubator. The proposal itself is located here, and an extract follows below.

Abstract

Apache Wave is the project where wave technology is developed at Apache. Wave in a Box (WIAB) is the name of the main product at the moment, which is a server that hosts and federates waves, supports extensive APIs, and provides a rich web client. This project also includes an implementation of the Wave Federation protocol, to enable federated collaboration systems (such as multiple interoperable Wave In a Box instances).

Proposal

A wave is a hosted, live, concurrent data structure for rich communication. It can be used like email, chat, or a document.

WIAB is a server that hosts waves. The best analogy for this is a mail server with a web client. WIAB is comprised of a few high-level components: the client and the server. They have the following major functionality (though this is not an exhaustive list):

• Client
  • A dynamic web client for users to create, edit, and search waves. Users can access this client by directly visiting the server in a browser.
  • Gadgets provide the ability to insert, view, and modify the UI -- exposing the Wave Gadgets API (http://code.google.com/apis/wave/extensions/gadgets/guide.html)
  • A console client that can create and edit waves via a command-line-like interface.
• Server
  • Hosts and stores waves. WIAB comes with a default storage mechanism. The administrators of the server may configure it to use alternative storage mechanisms.
  • Indexing, allowing for searching the waves a user has access to.
  • Basic authentication, configurable to delegate to other systems.
  • Federation, allowing separate Wave in a Box servers to communicate with each other using the Wave Federation Protocol (http://www.waveprotocol.org/federation).
  • Robots, using the Wave Robots API, (http://code.google.com/apis/wave/extensions/robots/) may interact with waves on a WIAB instance.

Background

Wave expresses a new metaphor for communication: hosted conversations. This was created by Lars and Jens Rasmussen after observation of people's use of many separate forms of communication to get something done, e.g, email, chat, docs, blogs, twitter, etc.

The vision has always been to better the way people communicate and collaborate. Building open protocols and sharing code available in an open and free way is a critical part of that vision. Anyone should be able to bring up their own wave server and communicate with others (much like SMTP).

We hope this project will allow everyone to easily gain the benefits of Wave with a standard implementation of Wave – in a box.

Rationale

Wave has shown it excels at small group collaboration when hosted by Google. Although Wave will not continue as a standalone Google product, there is a lot of interest from many organizations in both running Wave and building upon the technology for new products.

We are confident that with the community-centric development environment fostered by the Apache Software Foundation, WIAB will thrive.

Initial Goals

The initial goals of the project are:

1. To migrate the codebase from code.google.com and integrate the project with the ASF infrastructure (issue management, build, project site, etc).
2. To quickly reach a state where it is possible to continue the development of the Wave In a Box implementation under the ASF project.
3. To add new committers to the project and grow the community in "The Apache Way".

Current Status

The open source Wave in a Box project has existed in various forms for approximately 16 months (starting out life as the FedOne open source project).

FedOne began in July 2009 in order to accelerate adoption of the wave federation protocol, and serve as a proof of concept that a non-Google implementation of the wave federation protocol could interoperate with the Google production instance. It worked. FedOne's existence lead to a prototype by Novell that demonstrated federation between Google Wave and Novell Pulse (now known as Vibe). In addition, in May of 2010, SAP unveiled a prototype version of SAP StreamWork that federated with both Novell Pulse and Google Wave. All three systems interoperated, sharing real-time state, and gadget updates. In May 2010 Google released significantly more code (including the cross-browser rich text editor) to connect with other components that were built from scratch, resulting in a simple web client.

The project has grown over the last year to include many Google and non-Google contributions. The project has picked up steam in recent months as the direction of the standalone Google Wave product has shifted. At this time the Wave in a Box project enjoys very active development, with new features and functionality being added almost daily. The first Wave Protocol Summit was recently held and included developers from a variety of countries, companies, and organizations.

The code base is a mixture of mature core code from Google Wave, and somewhat immature integration code forming WIAB. WIAB is quickly becoming highly functional and is already in a very "demoable" state. The development mailing lists are very active indicating wide community support. We recognize that now is a good time to migrate to the Apache Foundation while the codebase and community is a manageable size. Assuming the current momentum continues, we expect strong growth in the code and community in the near future.

Meritocracy

The initial set of committers includes many Google employees, and there is an active and growing community outside Google contributing to WIAB already today. Google culture itself encourages meritocracy, and the community has always grown – and will continue to grow – in this fashion.

As shown by the initial committers list below, several members from outside of Google have already demonstrated interest, skill, and commitment to contributing to the project. These individuals have been recognized on those merits by the initial committers. Their selection as the first wave of new committers is a sign of the burgeoning meritocracy.

Community

Wave currently has a healthy community around waveprotocol.org, with conversations hosted at http://groups.google.com/group/wave-protocol. We plan to move this community to the Apache Software Foundation incubator.

Core Developers

The initial committers comes from a variety of backgrounds and includes many from Google. There are a few existing Apache committers amongst this initial group. We anticipate early future committers coming from places like Novell, SAP, companies related to the US Navy's usage of wave, startups in the wave ecosystem, and many independent individuals.

Alignment

The developers of WIAB want to work with the Apache Software Foundation because Apache has proven to provide a strong foundation with good infrastructure and support for developing projects in an open community. As WIAB continues to grow, the community will look to both reuse available Apache projects as well as look for opportunities to contribute back to the larger Apache community.

Known Risks

Orphaned products

Wave is a new means for communication, and thus it is still maturing. While the initial implementation (Google Wave) did not gain sufficient traction for it to continue as a standalone Google product, there are other related projects (e.g. Novell Vibe, SAP StreamWork), and several startups in the space that are continuing to build on the technology. In addition, the US Navy has contracted with four companies as part of evaluating using wave technology on every ship. The community itself is still growing, with several new contributors recently added.

Inexperience with Open Source

The initial committers have varying degrees of experience with open source projects. Many from the community are familiar with open source.

Homogeneous Developers

The initial set of developers does include many from Google. However, the project has accepted many patches from independent individuals, and some have already gained committership. Several companies have expressed interest and forty individuals participated in the Wave Summit.

Reliance on Salaried Developers

Following Google's change of focus for Wave in August, some of Wave's Google developers have chosen to continue working on Wave, but it is imperative that we continue to grow the community larger in the coming months.

Relationships with Other Apache Products

We currently use the following libraries from Apache

• Commons CLI
• Commons Codec
• Commons HttpClient
• Commons Logging
• Velocity
• Ant

We've also contributed the Wave Gadget implementation into the Apache Shindig project.

Documentation

Entry point for documentation of all the specs and designs. http://waveprotocol.org/

Wave Robots API http://code.google.com/apis/wave/extensions/robots/

Wave Gadgets API http://code.google.com/apis/wave/extensions/gadgets/guide.html

Initial Source

The initial source will come from http://code.google.com/p/wave-protocol/source/browse/. This consists of the Java code necessary for the client and server. These are already open source repositories licensed under the Apache Public License.

Source and Intellectual Property Submission Plan

Beginning with the initial unveiling, Google published a liberal patent license:

Subject to the terms and conditions of this License, Google and its affiliates hereby grant to you a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this License) patent license for patents necessarily infringed by implementation of this specification. If you institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the implementation of the specification constitutes direct or contributory patent infringement, then any patent licenses for the specification granted to you under this License shall terminate as of the date such litigation is filed.

http://www.waveprotocol.org/patent-license

Trademarks

Google retains all rights to the trademarks "GOOGLE WAVE" and the wave design logo, neither of which will be used in the Apache Wave project.

External Dependencies

In addition to the previously mentioned Apache dependencies, the initial code relies on the following libraries that have Apache compatible licenses:

antlr, aopalliance, asm, bouncycastle, cglib, dom4j, emma, gson, guava, guice, gwt, gxp, hamcrest, jackson, jdom, jetty, jline, jmock, joda_time, jsr305, junit, libidn, mockito, mongo-driver, oauth, protobuf, protobuf-format-java, protostuff, stringtemplate, websocket, whack, xpp3

Cryptography

We use standard crypto library methods available in java.security.*. Wave federation plans to uses encryption for sending deltas to remote Wave servers.

Required Resources

Mailing lists

• wave-dev
• wave-commits
• wave-private

It is possible that if the project does grown to include many sub project that we would split the mailing list up by sub project. Again we have flexibility.

Subversion Directory

https://svn.apache.org/repos/asf/incubator/wave

Issue Tracking

Please help us setup a JIRA instance for both issue tracking and code review.

Other Resources

• a wiki (for the sites pages) (http://incubator.apache.org/guides/sites.html or a wiki http://wiki.apache.org/incubator/)
• code review on reviews.apache.org
• a server to run a dogfood instance
• continuous build bot

Initial Committers

• Alex North (Google)
• Anthony Watkins (SESI)
• Christian Ohler (Google)
• Dan Danilatos (Google)
• Dan Peterson (Google) 
• David Hearnden (Google)
• David Wang (Google)
• Ian Roughley (Novell) 
• James Purser
• Joseph Gentle
• Lennard de Rijk
• Michael MacFadden (Solute)
• Soren Lassen (Google)
• Tad Glines
• Torben Weis (University Duisburg-Essen)

Sponsors

Champion

• Paul Lindner

Nominated Mentors

• Santiago Gala
• Ben Laurie
• Upayavira
• Brian W. Fitzpatrick (emeritus on the Incubator PMC)

Sponsoring Entity

The Apache Incubator.

Comments are being sought on the proposal at this time. Please post questions or comments regarding the blog entry below.

UPDATE: Wave was accepted into the Apache Incubator on December 3, 2010. This will be the subject of a subsequent post.

HyLighter- Advanced Document Collaboration

This is a cross-post from my blog on the Military Social Networking system milBook. 

HyLighter (www.hylighter.com) is a new technology that has some unique and very useful functionality.This blog post will give an overview of the system and some of potential use cases. This is not a commercial endorsement for Hylighter, but I will use their promotional material in different places throughout this post. 

Today, it is common to have documents that require input from 10, 20, 30 or more contributors. The most common practice for managing document reviews for large groups is for the owner of the document to send out emails with attachments to contributors. Each contributor makes changes and adds comments, usually with track changes, and then emails the file back to the owner. The owner has the time-consuming and frustrating task of maintaining version control and reconciling all the changes manually. Once complete, the owner recycles the draft for another round of reviews.

HyLighter is an easy to learn web annotation and hypermedia system for collaborative analysis and review of documents. Supported file types include HTML, Word and, soon, images, PowerPoint, PDF, and Excel. In general, Web annotation systems enable users to add, modify or remove information from a Web resource without modifying the resource itself. The information can be thought of as a layer of annotation displayed on top of the existing resource. Depending on how the owner of the resource configures permissions, users who share the same annotation system can see all or parts of the annotation layer.

The name, HyLighter, is a combination of the terms hypermedia + highlighter (i.e., a felt-tip pen which is used to draw attention to sections of documents by marking them with a vivid, translucent color). Hypermedia refers to graphics, audio, video, and text that are intertwined through hyperlinks. A user following hyperlinks is said to navigate or browse the hypermedia. The most familiar example of a hypermedia system today is the World Wide Web which is a system of interlinked hypermedia documents contained on the Internet.

What is unique about HyLighter as a Web annotation and hypermedia system is that it enables almost any number of reviewers to engage in threaded discussions tied to selected locations or fragments within a document in an organized and efficient manner. It also enables users to copy a Uniform Resource Locator (URL) for any annotation (i.e., a highlighted section of a document and associated commentary and other metadata) to a clipboard in order to (a) send or post a link to a selected annotation to one or more individuals through various applications for exchanging information over the Internet (e.g., chat, instant messaging, microblogging, and email) and (b) create a link from a selected annotation to one or more annotations within the same document or a different document either on the same machine or different machine(s).

In addition to creating a bridge between social networking applications and specific locations within various file types, HyLighter has public APIs available for integrating with content management systems and other types of applications. For example, HyLighter offers a plugin for SharePoint that enables a user to push a document from a SharePoint library to a browser with HyLighter installed. Once the review activity is completed, HyLighter provides various mechanisms for analysis of input and editing in Word or other native application and checking the document back into the SharePoint library for maintaining version history.

HyLighter is straightforward to use. Simply import the document into the system (or use something like the SharePoint plugin shown above), access the document using a browser with the proper plugin, and begin editing or making comments. For example:

The key to the technology is the use of XML and XHTML that allows for the formation of the annotation layer mentioned above. There are also many other ways that the XML in HyLighter can be leveraged, such as metadata, and other integrations with XML-based technologies. While the use of HyLighter and Google Wave had been discussed (and Wave is mentioned in the HyLighter literature) it has not been actively pursued. 

HyLighter solves three major challenges that are relevant to expertise location and knowledge mining that are not adequately addressed by any other available technology. The challenges:

- How can large groups of reviewers (that is, 5-50 or more) engage in collaborative discussions tied to specific sections of a document or source in an organized and efficient manner?

- How can the potential of collective intelligence or crowd sourcing be harnessed to increase the value of existing content?

- How can knowledge workers collaboratively link related ideas and objects together or “connect the dots” across increasingly large repositories of unstructured data in order to make associations, discover patterns, and create new knowledge and innovations?

Here are some potential use cases for HyLighter:

- Policy Creation: Most organizations have a need to create policy documents, especially in Government. There is usually a review and approval process when writing policy, in addition to multiple authors of the content. HyLighter would allow for many authors and reviewers, and it automatically creates an audit trail for changes on the document.

- Technical Writing: This seems a logical use for the system. Often technical writing requires peer review or input from multiple experts, and HyLighter would allow for these individuals to easily review and/or provide input on specific parts of a document.

- Proposal Management: Writing proposals is another natural use case for HyLighter. Collective authoring and review is generally a requirement when writing proposals. Having different authors and reviewers for each section of the proposal is much easier to manage with HyLighter. 

- Training: HyLighter can be used for both training development (collaborative creation of content and programs of instruction) and also as part of the learning experience where learners can interact with instructors on collaborative online documents.

- Enhance capabilities of wikis and blogs: Sometimes on wikis and blogs there is content that lends itself to a collaborative effort. Placing links on the original site to a HyLighter document allows for the development of the content in a more organized fashion than most Social Media systems.

- Support online communities of learners and practitioners: Online social networks around specific areas of interest can benefit from HyLighter in that they can use the system to collaboratively create content to enhance the building of knowledge on the specific topics involved.

There are trial instances of HyLighter accessible from their website, and there is also a military-focused test instance as well. The technology is sophisticated and the best way to understand it is to give it a try.

Again, this does not constitute a commercial endorsement of this product, and is strictly the opinion of the author. Please post your questions or comments below, and thanks for reading.

FedWave Update

This is a cross-post from my blog on the Military Social Networking system milBook. I have also started a new blog on the new online space for FedWave here, which is restricted access for the time being.  

Three months have passed since Google announced that they would discontinue development on Wave. From my post in August:

On August 4, 2010, Google announced that they were going to discontinue development on Google Wave. This came as a surprise to much of the community of users and supporters of Wave, and sparked an outpouring of commentary on the topic in the Blogosphere. These postings included how Google doesn't do Social Media, how Google botched the rollout of the system, and how Google did the right thing by cutting their losses. All of these missed one point: the value of the technology itself.

In these three months, the Wave in a Box project has made impressive strides, with the current version looking much like the main Google product (see also below). There is still a lot to do, but much has been accomplished. In addition, all the information has been consolidated in one place. This project is moving fast, as the plan is to have a fully functional open source WIAB server ready for general release by the end of 2010. I will continue to monitor (and contribute) and will post updates like this one on a regular basis. 

The FedWave effort has made progress as well, with a Working Model of FedWave now operational in a limited mode. More information on this working model is in the FedWave group located here. You must apply for membership first at APAN.org. I will approve most requests for group access.

This virtual space is designed to facilitate online collaboration between different government agencies and with non-government entities as well. It has many different capabilities including blogs, wikis, document upload, and others. I selected this platform to enable collaboration with a wide group and to consolidate information in one place online. The site is a little sparse now, and I would welcome contributions and content.

Here is a current screenshot of the Working Model: 

FedWave Working Model as of 4 Nov 2010

I have let a long time pass between posts, but I will work harder to post more frequently. As always, please enter your questions and comments below.