9.2011 – 10.2015
Microsoft Windows Precision Touchpads
MY ROLE
I was responsible for the first three generations of Surface Type Cover. My focus was on the keyset, touchpad, backlighting and cover interaction design. This work formed the Windows Precision Touchpad experience, aligning the broader ecosystem with the launch of Windows 8.
In the previous post, I shared my experiences and learnings while helping to create Microsoft’s Surface Touch Cover. Interestingly, Type Cover was not part of Surface’s original vision. Instead as Touch Cover transitioned from concept to product (traversing the risks common with new, never manufactured technology and its unproven typing experience), only then was Type Cover born. In other words, Type Cover was the “insurance plan” while Surface bet on Touch Cover.
Anyone desiring a tablet yet reluctant to lose the productivity of their laptop because they only want one device. Ultra-books just weren’t the solution and tablets just weren’t something you used to get real work done. Even the Bluetooth accessories compromised ergonomics, battery life and required yet another thing to manage.
There was clearly a need and opportunity to design something better. The solution mattered and critical customer feedback on each generation of Surface Type Covers was testimony.
For the first generation Type Cover were to…
- Design the most productive tablet experience, differentiating Microsoft’s Surface from Apple’s iPad, Google’s Nexus and Amazon’s Kindle,
- Showcase a Windows 8 experience with Microsoft designed hardware, and
- Establish Surface as a premium brand, growing the PC tablet category and raising the bar for other PC manufacturers.
With the next two generations of Type Cover, these goals shifted towards refining the experience, adding new functionality and showcasing other Windows features. Surface was driven by the drum beat of “faster, thinner, lighter, stronger, better”, never sacrificing craftsmanship and experience all while trying to drive costs down.
Boom, boom, boom…
Simple – make a cover that has a keyboard & touchpad, which snaps in effortlessly, turning a Surface into something you could do work with. Ensure it feels like a book when covering the screen, remains reliably connected through typical use and abuse, yet easy enough for a child to tear off.
I was responsible for defining the cover’s backlighting experience, keyset geometry, functionality, and legend typography (localized for 25 regions around the world). After nearly 8 years developing award winning keyboards, we were about to ignore the ergonomic standards we helped form, favoring form over function, targeting an overall thickness of 5mm in pursuit of a modern looking, super thin keyboard. Much which once was essential to keyboard design was now challenged, forcing difficult tradeoffs and making the tiniest details (i.e. key spacing, dish, chamfer, radius, travel resistance, texture) more important.
For some touch typists, we knew we might be going too far.
Mitigating inadvertent key presses and touchpad brushes as well as achieving an elegant interplay with the onscreen keyboard was critical to get right. I was responsible for the state and transition design which determined how the physical and onscreen keyboard, touchpad and backlighting would behave as the cover rotated around the device.
Was most constrained by Surface’s 16×9 aspect ratio, overall size and desirable fabric wrapped design. Full screen movies, normal keyset geometry and a delightfully touchable palm rest were more important (at the time) than a decent trackpad because form was again prioritized over function. A touchpad which both PC and Apple fans could intuitively use, with a set of gestures that beautifully lit up Windows 8 were critical for Microsoft.
Let’s face it, touchpads are polarizing. Some people appreciate their convenience, while others abhor them because they lack the precision of a mouse or tend to disrupt our ‘flow’ when typing. In all honesty, putting anything where your hands are when typing is just asking for a source of annoyance. Some devices ignore this problem entirely while others address it through software settings or a hardware affordance which toggles the touchpad on / off. While these solutions work for some people, more than half of Windows users rely on the tap to select interaction model. A better solution was needed which would not disable this expected functionality.
When a problem is technical, one of the best things I’ve learned to do is familiarize myself with design verification test tools. As I sought to more robustly address unintended touches, seeing things from the sensors perspective revealed the strengths and limitations of the design and helped me understand what stresses capacitive systems. Many of the low hanging mitigations meant tradeoffs in performance, often introducing some form of latency. Touch system vendors were providing low level mechanisms to help, but none were fully effective. This technical knowledge helped me define the touchpad interaction model and align a cross functional team on the feature scope and priority.
I evangelized the importance of mitigating unintended behavior over enabling multi-finger gestures. Together with the firmware engineers we designed proprietary algorithms which contextually filtered touches based on multiple indications of user activity. To understand sources of frustration and user behavior, I drove to evolve these test tools to visualize specific events. With this new tool, I characterized common changes in touch characteristics, influencing the algorithms to contextually dampen movement. The first generation algorithms reduced inadvertent tap actuations by more than 50%, but it was not enough.
A way to turn on / off touchpad functionality was still needed and discreet Windows touchpad settings did not exist. Third party drivers were also not a viable solution any longer because with Windows 8, Microsoft was tightening the reins on what was allowed. The only solution feasible for launch was a modern store app but that meant those frustrated needed to realize there was an app, find it in the store, install it and then figure out what setting would help them. Clearly far from an ideal experience. I proposed the design with the business justification and convinced the leadership team to support the development just six months before launch. Without much evangelism, the app reached ~80K downloads and a 4.0 average rating before Windows 8.1.
The first generation Surface touchpads were good but not great. What was learned, I shared with Windows in the compilation of the Design recommendation for Windows 8 touchpads.
Was developed closely with Windows. This time around I drove aspects of Surface’s touchpad design into Windows, influencing the Windows Precision Touchpad Implementation Guide and the Windows Precision Touchpad Device Validation Guide. Changing the architecture simplified the cover’s firmware design, delivered settings in Windows where everyone expected them, and enabled the Surface organization to focus on other aspects of the product while Windows engineered the gestures.
Convergence on a single, low cost architecture and consolidating the firmware development was a large effort with the second generation products. With the Surface Touch Cover being the thinner of the two (at less than 3mm), its thickness constrained the experience. I pushed for button feedback within those constraints leading efforts to prototype innovative feedback solutions feasible in super thin form factors. These prototypes demonstrated promise but were not ready for prime time. I had to find an alternative solution and settled on extending the Touch Cover’s keyset audio feedback to the touchpad. Audio feedback was an improvement for the second generation Touch Cover, but a regression for Type Cover.
The touchpad gestures were appealing, but performing them comfortably and consistently on the small touchpad was still awkward. Establishing touchpad settings in Windows posed additional challenges. For new Surface customers, settings were where they were expected …in Windows. For customers upgrading existing Surfaces to Windows 8.1, the transition was less ideal (if you used them before and now needed to look in a new location for them). These compromises were noted in almost every product review which was most frustrating because I anticipated the reaction but could do nothing in a program where schedule was more important than experience.
Was when the organization finally aligned with passion to nail the touchpad experience. With Surface Pro 3’s new aspect ratio, I was able to define a larger touchpad size. I also advocated for a much more touchable, smoother material and refined the button force to fire. These changes were significant improvements. The larger touchpad size and new tilted keyboard position though increased the frequency of unintended touches.
To address this side effect, I led efforts between Surface, Windows and the touch sensor vendor to identify opportunities. Filtering unintended touches with additional latency was not an option. The touchpad latency in specific cases was perceivable to some already. Instead of adding more latency, I drove efforts to characterize the context of unintentional touches, learning what they look like, how they move, where they appear and when. Realizing the existing test tools were not enough to see the big picture, I again defined requirements for new tools which would visualize large sets of recorded touchpad and keyboard input. With these new tools, I was able to champion solutions with Windows, delivering fixes through Windows Update in time for the product launch and later through the Windows 10 upgrade.
Even with these fixes in place, the touchpad was still susceptible to inadvertent actuation. While clearly better than previous generations, I knew the hardware architecture limited how the problem could be solved. I later advocated for hybrid sensing architectures which provide more input to disambiguate intent and helped an incubation team investigate this promising opportunity. After I left Microsoft these efforts were realized in fourth generation products.