Any Surface UI

It Starts With a Point

A point is zero dimensional and belongs to no device. From it, lines emerge to define the display area. Information surfaces inside that area. Then it returns to silence. The abstraction is the easy part to describe and the hard part to build, because every state in between had to be drawn.

The Device Is Another Surface

The premise of the programme was a class of device that cannot assume a large display, a keyboard, or that the person is even looking. The Xperia Agent carries a projector in its head and a small screen in the centre of its body. Any Surface renders on both, which is the part the name is actually claiming.

Four Goals, Set Before Anything Was Drawn

The programme committed to four goals at the start, and every design decision had to answer to all of them at once. They pull against each other on purpose, which is what stopped the system drifting into something that only works in a photograph.

One System, Any Surface

The same interaction language, rendered onto whatever was in front of the person. A kitchen counter mid-recipe. A wall above a speaker. A table between two people. A rain covered window. Each surface changes the contrast, the viewing angle and the amount of attention available, and none of them changes the grammar.

Modeless, Which Is Harder Than It Sounds

Voice and touch are interchangeable at any point. Not voice first with touch as a fallback, and not a touch interface you can also talk to. Either one can pick up the thread the other started, at any moment, without the person announcing a switch.

That constraint removes the easiest way to build a conversational interface, which is to hold the person in a mode and make them leave it. Every state had to accept both kinds of input and stay coherent afterwards, which is why the state work took longer than the visual work.

What It Does When It Does Not Understand

Most agent interfaces of the period answered a failure to understand with a mechanical apology and a repeat request. That is the moment the illusion dies. The design answer was to change medium rather than repeat, the way a person reaches for a pen when the words are not landing.

The dialogue system was a Sony original rather than an adaptation of anything available at the time, and it remains the most conceptually difficult piece of design work in this portfolio. The visual language can be shown in a photograph. The dialogue model can only be judged by using it.

Finding the Language

Before the system existed there was a wall of surfaces and a stack of sketches. Glass, timber, a car windscreen, a rain covered window, and the same small geometry drawn over and over until it held on all of them.

What It Was Actually About

The brief reads as an interface project and it is not quite one. Any Surface is an argument that the right amount of interface is usually none, and that the system’s real job is deciding when there is a clear reason to appear. Non-intrusive, focused, and built around the person rather than the device.

Everything that looks like styling in the final work is doing that job. The point has a personality because it is the only thing visible most of the time, and it was drawn to feel organic and alive rather than switched on. The boundary is drawn before the content because the interface should commit to how much of your room it is taking. The retreat into silence is designed because leaving well is the feature.

The hardest part was never the visual language. It was the dialogue: a modeless model where voice and touch are interchangeable at any point, and where a failure to understand is answered by changing medium instead of repeating the question. That system was a Sony original, and it remains the most technically and conceptually complex design problem in this portfolio.