6.005 Elements of Software Construction | Spring 2010
Project 3: Realtime collaborative editing

Due: April 22 @ 5pm;
April 29 @ 5pm; and
May 9 @ 11:59:59pm

Problem

A collaborative editor allows multiple users to work on a single document simultaneously, across a network. Examples include Writely and Etherpad (both acquired by Google), SynchroEdit, ZohoWriter and many others. In this project, you will build your own collaborative editor. This is a challenging application, since it involves handling concurrency (so that simultaneous actions interleave in a reasonable way), dealing with network connections, and building a graphical user interface. It should also be fun, because you'll be building a tool you can use yourself, you'll become familiar with a variety of technologies, and you'll have an opportunity to practice applying the fundamental ideas you've learned in the course, and working on a team.

Purpose

The purpose of this project is threefold. First, you will have an opportunity to practice using the fundamental design and implementation techniques you've learned in the course -- most notably, object models and mutable datatypes, but also ideas from the earlier paradigms (state machines, functions over immutable types, data abstraction, decoupling, etc).

Second, the project will give you practice using a variety of technologies -- networking, sockets and I/O, and Swing (Java's GUI toolkit) -- and will help you deepen your understanding of common design patterns, such as view hierarchy and model-view-controller for user interfaces, publish/subscribe patterns (such as Observer), and queue-based treatments of concurrency.

Third, you'll have another chance to work on a team, this time for a slightly more ambituous project, and with some more (self-determined) structure than before.

Specification

You have considerable freedom to design your editor as you choose: not only in the implementation, but also in designing the behaviour. We recommend, however, that you design and implement for the minimal required functionality first, and add extensions only when you are confident that the core is stable and solid.

Some of the decisions you will have to make are:

Nevertheless, your tool must meet some minimal requirements:

Tasks

  1. Individual preparation. Complete the lab on networking.

  2. Team preparation. Meet with your team and complete the team building exercise.

  3. Familiarize yourself with Swing. The Java tutorial on Swing provides a good introduction.

  4. Abstract design. Using object models, state machines and datatypes, explore the behavioural design of your tool. In particular, define carefully and precisely what an edit is. Make sure to record not only the decisions you made, but the decisions you rejected, along with their rationale.

  5. Architecture and protocol. Design an architecture and a protocol for the system's components. Express your protocol as a state machine or grammar, and give the structure of the protocol messages using a grammar or datatype productions.

  6. Usability design. Sketch your user interface and its various screens and dialogs. Use these sketches to explore alternatives and to plan the structure and flow of your interface. Sketching on paper is recommended.

  7. Code design. Design the code structure of your program using object models and module dependency diagrams. Record important design decisions -- in particular all the patterns you chose (see the lecture notes for reminders of all the patterns covered in class) -- and their rationale. Pay particular attention to concurrency design, and to making it easy to run automated tests.

  8. Testing strategy. Devise a strategy for testing your system (determining what order you will test components in, how you will derive test cases, what level of coverage you plan to achieve, how you will test the user interface, etc). Justify your strategy with an argument that the level of effort is appropriate for the likely prevalence and location of bugs, and the degree of reliability required.

  9. Implementation. Write code based on your design. Make sure to update the design documents if the design changes; at the very least, you are likely to find additional dependences that you had not anticipated. Make sure to make extensive use of runtime annotations, data abstraction and representation invariants.

  10. Testing. Execute your testing strategy, and document the results.

  11. Reflection. Write a brief commentary (eg, half a page to one page in length) describing what you learned from this experience:

Deliverables and Grading

There are three deadlines for this project listed at the top of this document, each with its own deliverables.

For the first deadline:

For the second deadline:

The purpose of the demo is to demonstrate progress, and to help you focus on understanding a critical or high-risk area of the design. It might include, for example, a basic server that sends and receives messages but without a GUI client (see the the hints about telnet). Or you might have a working basic GUI with no server backend but a simple API for connecting to one. Talk to your TA beforehand if you are unsure about what is sufficient.

For the third and final deadline:

The grading breakdown is as follows:

Each deliverable should be submitted as one PDF in the deliverables folder of your project repository. After each of the first two deliverables, you will meet with your TA the next day during the lecture period to review them and plan your next steps. Be prepared to do your demo for your TA at your second meeting. You should bring a printout of your deliverable to these meetings. For your user interface sketches, either scan the files and place them in your PDF document (see resources for a list of scanners on campus), or use a drawing tool.

Hints

Design models. In the grading of the design documents, we will be looking to see whether you really addressed fundamental issues. A small and focused design model is always better than a large, diffuse one; your aim is to shed light on real problems, not to do mindless documenting of obvious things. A reader can tell when a design model was developed incrementally as the focal point of the design (because it's clear, easy to follow, and addresses interesting questions) and a design model that was hurriedly generated after the fact (because it's usually obscure, misses important issues, and includes boring details). Brief comments on your design models will help the reader understand the role they play and the key decisions you made. Also, the relationship between your design models and code should be very clear, and should be documented using the language of design patterns you have learned.

Defining documents and edits. Perhaps the hardest and most interesting part of this project is the abstract design, in which you will determine what constitutes a document, and what an edit is. Is a document like a chat room or an email message? How are documents named, and do the names have a context? Does an edit occur at a point in the document, or are global changes allowed? If it occurs at a point, what happens when two users edit simultaneously? Will both their edits be reflected or might one override the other? Are some edits independent and some interfering, and if so how is interference resolved? The most important advice we can give you here is to keep it simple. You can always extend your program, but once you have a mess, it's hard to go back.

Designing architecture and protocol. You must also devise a network architecture and a protocol for this project. A client/server architecture is potentially the easiest choice here, but it isn't required. You should strongly consider using a text-based protocol, which may be easier for testing and debugging.

Services that use plain text protocols — e.g. HTTP or SMTP — can talk to a human just as well as another machine by using a client program that sends and receives characters. Such a client program already exists on almost all operating systems, called telnet. You can run telnet by opening a command prompt and typing telnet hostname port. The lab gives you some experience with using telnet.

Handling multiple users. Since realtime collaborative editing is not realtime without at least two people, your system must be able to handle multiple users connected at the same time. The Friendly server you'll develop in the lab gives you some starting code, but note that Friendly doesn't need its clients to interact or share any state. Your system will certainly need to do that. One reasonable design approach follows the Friendly model (using one thread for reading input from each client) but adds a central state machine representing the state of the server (using one more thread, to which each of the client threads pass messages through a shared queue).

Read the socket documentation referenced in the lab to understand how network sockets operate in Java. Consider how, for example, the server will write to clients while at the same time awaiting messages from them.

Design for safe concurrency. In general, making an argument that an implementation is free of concurrency bugs (like race conditions and deadlocks) is very difficult and error-prone. The best strategy therefore is to design your program to allow a very simple argument, by limiting your use of concurrency and especially avoiding shared state wherever possible. For example, one approach is to use concurrency only for reading sockets, and to make the rest of the design single-threaded.

And note that, even though user interfaces are concurrent by nature, Swing is not thread-safe. Recommended reading: Threads and Swing.

Design for testability. To make it possible to write unit tests without having to open socket connections and parse streams of responses, you should design your components so that they can be driven directly by a test driver -- eg, by calling methods, or by putting messages into a queue.

Testing GUIs is particularly challenging. Follow good design practice and separate as much functionality as possible into modules you can test using automated mechanisms. You should maximize the amount of your system you can test with complete independence from any GUI.

Resources

Available scanners: