The challenge
When Apple introduced the iPhone, Microsoft's natural user interface (NUI) technology was still in its early research stage. With Apple first-to-market with multi-touch user interfaces, all eyes were on Microsoft's response.
Microsoft relied on my organization to produce libraries, proofs-of-concept, keynote showcases, and technical demos to fast-track their NUI story. Ultimately, the patterns and conventions we established and even some of the code we wrote formed the basis for Microsoft's Surface hardware platform today.
A device built for research
Five years before the tablets and laptops that carry the Surface brand we know today, Microsoft produced an interactive coffee to facilitate research into natural user interface behavior, developing gesture libraries and forming the basis for a future mass-market consumer story. Microsoft retired the table form factor once they released the Surface Pro in 2013, shortly after rebranding it "PixelSense" to disambiguate it from the consumer desktops, tablets and laptops that would carry the brand.
The Surface table retailed for about $10,000 so Microsoft primarily sold the device to commercial customers rather than home users. Consequently, the nature of my early work in developing applications for Surface focused on installations in public settings like retail stores, restaurants and hotel lobbies.
Gridless, omnidirectional interfaces
The original Surface is unlike any other device I've ever designed for in a number of ways. Every other screen-based interface we're familiar with has a distinct top and bottom, but a coffee table is omnidirectinal – usable from any side with no sense of top and bottom in the interface. Plus, because of the size of the display, it invited more than one person to use it at a time, often strangers. This created unique design challenges that lead to deeply insightful user research sessions.
Microsoft even used an application I helped design and develop in their own Enterprise Engagement Center, a research facility for large-scale cloud services.
A screen that's looking back at you
Another aspect that made Surface so unique was that it's touchscreen used a camera array rather than a capacitor – it was actually "looking" at the screen from the inside. This made for some fascinating interaction opportunities through object recognition.
I helped design the mechanism that blurred the distinction between the physical and digital realms, creating interactions that seemed like magic at the time.
Cutting edge invention
In addition to the interactions, I also helped invent the means by which Surface connected to other devices via optical means, long before NFC was mature enough to initiate a Bluetooth connection. Finally, I helped develop the C# and Java libraries that powered these interactions.
Lasting impact
I consider myself privileged to have had the opportunity to help develop a technology platform as unique as Surface. While very few people had the opportunity to experience the original Microsoft Surface themselves, I find deep satisfaction in the knowledge that my research, designs, inventions and libraries live on today in Microsoft's ongoing natural user interface story.
