20 February 2010

Developing with Microsoft Surface – a few lessons learnt

I’m coming to the end of the largest commercial development that I’ve done for Microsoft Surface so far. Although I’m pretty proud of the results, there are some inevitable learning points that will help us to use our time and effort in a more focused way the next time around.

Designing the experience

On a practical level, Surface is pretty straightforward to develop with if you’re comfortable with WPF. However, the unique nature of the Surface experience means that it’s not quite like any other rich application development.

A big obstacle to designing an experience for Surface is getting everyone involved to really understand how the Surface interface differs from a traditional  user interface. For starters, you’re catering for multiple users rather than a single person. The user experience tends to be relatively unstructured as you  to have to cater for multiple simultaneous inputs where there’s no way of telling one user from another or predicting how their actions may interfere with each other.  The spatial, 360 degree nature of the table top means there’s no “up” or “down”, “left” or “right”. The direct and tactile interface allows users to directly manipulate the interface rather than doing so through the abstraction of a keyboard and mouse.

How do you develop and communicate a Surface experience to a everybody involved in the project without relying on a leap of faith?

Traditional wire-frames just don’t work here, as they cannot convey the full range of possible  interactions. It’s very easy to fall into the mindset of developing for a traditional single-user interface when you’re working with a static two-dimensional design. Validating a user experience is particularly difficult. Eye-tracking software can be a great way of determining the effectiveness of a design for single-user applications, but how do you test the effectiveness of a multi-user experience?

Prototyping can be a valuable approach, but rigging up prototypes for Surface can be a laborious process and  impractical for a commercial development that is subject to the usual limitations of time and money. An agile development process can help to mitigate some of these difficulties, but no agile process will help you if it takes a few development iterations to identify the weaknesses in your  interface design.

Ultimately, you can only really foster a shared understanding of a Surface interface design by encouraging your client, designers and developers to sit around a Surface unit and play with it. If you see a design in the context of the Surface unit then it can all fall into place in a way that is not really possible through static designs.

Organising the code

Good code design is always important, but it’s critical for a Surface application.

Most Surface applications tend to be based around a single window containing a myriad of different controls that interact with each other in unpredictable ways.  It is impossible to script a user experience – your code has to be robust enough to handle any type of interaction without falling over.

If you have a team of developers working on an application then file contention can become a problem if you don’t organise your code properly. The main Surface window can become the focus of a lot of critical code and no source code control can prevent developers tripping up over each other.

A design pattern based around well-designed data models and that bind to interface elements is crucial. Time spent in carefully designing models for your components is time saved in debugging and adding new functionality later on. It also helps to reduce file contention among developers by properly separating functionality into different files. Although this applies to pretty much any WPF-based application, it’s particularly important for Surface.

Implementing a visual design

We are lucky enough to work with  some very talented designers, which is just as well given the practical difficulties associated with graphic design for a Surface application.

Designers are encouraged to use Microsoft’s Expression suite when developing vector graphics for WPF. This is impractical, as graphic designers generally prefer to work with Adobe tools. Rather than forcing designers to work with unfamiliar tools that they regard as inferior, we established an efficient design workflow based around exporting XAML from Adobe Illustrator.  The output isn’t necessarily the most optimal XAML, but it yielded some excellent-looking results.

The Surface unit itself can be a challenge to design for as it has a 1024×768 resolution that is stretched across a very large screen area. This causes a significant loss of fidelity between the designer’s screen and the Surface display: detail gets obscured, smaller fonts look unreadable and colour can lose its sharpness. Designing with vector-based tools can encourage designers to pack in intricate detail that can render pretty badly on the Surface unit itself. This loss of quality is something that designers have to be prepared for and continuous testing of designs is the only way to ensure that designs keep their value.

Designers and developers working together

WPF does appear to offer clean separation between the application logic and visual presentation, but the reality is a little more muddy.

We found it helpful to start by creating a skeletal set of controls with no logic or styling attached to them. This allowed us to split development so that the developers could implement the application logic while the user interface developer could concentrate on skinning. Despite this, there are occasions where the boundaries can blur between the two activities.

If designers and developers are going to work together effectively then it’s important to get the right blend of skills in place and ensure that everybody understands exactly which elements of source code they are responsible for. A user interface developer with a thorough understanding of XAML is a key player in the project as they act as a bridge between graphic designers and developers, but they will also need some understanding of C# and the rudiments of control binding if they are going to be effective.

Filed under C#, UI Development.