Dial it back to the cusp of a new millennium. Each tick of the clock punctuated by the rapid pulse of progress, the world ablaze with silicon sparks. A digital big bang unfurling across the landscape. Amid this roaring whirlwind, nestled in Delaware’s quiet corners, a fledgling entity emerged. Its name: FingerWorks. Small in stature, but gargantuan in ambition.
The concept of multitouch technology was independently developed on multiple occasions. One instance took place during the 1960s at England’s Royal Radar Establishment, while another occurred in the 1970s at CERN. However, the specific multitouch technology integrated into the iPhone was spearheaded around the time of the new millennium by an unfamiliar name to most: Wayne Westerman.
This ingenious individual, holding a Ph.D. in engineering from the University of Delaware, dedicated himself to advancing multitouch technology due, in part, to his personal struggle with severe repetitive strain injury. This condition rendered conventional keyboard interfaces an agonizing experience for him.
In 1999, Westerman and John Elias officially co-founded FingerWorks, a company dedicated to the development of multi-touch technology. Westerman served as the Vice President of Software Engineering, while Elias took on the role of Chief Executive Officer.
Their collective genius bore fruit in the form of revolutionary multi-touch devices, including the iGesture Pad and TouchStream keyboard. These devices transformed the way users interacted with their computers, allowing for a range of hand and finger gestures to replace the conventional mouse and keyboard inputs. The technology behind these devices is fascinating.
The iGesture Pad was a large touchpad that allowed users to control their computers using hand and finger gestures, replacing the traditional mouse. Imagine you are reading a long document and want to scroll down. Instead of scrolling the mouse wheel or dragging the scrollbar, with the iGesture Pad, you simply place two fingers on the pad and slide them down, as if you’re physically pushing the document down. The same gesture can be performed in reverse to scroll up. You can also zoom in or out of a document by placing your thumb and index finger on the pad and sliding them apart or together, similar to how you would on a smartphone.
The TouchStream keyboard, on the other hand, was a touch-sensitive keyboard that also supported various gestures. The TouchStream keyboard allowed for typing, pointing, scrolling, and many other functions, all without lifting your hands off the keyboard. Imagine typing a paragraph and you want to copy a sentence. Instead of reaching for the mouse to select the sentence, you can simply slide your left hand from right to left on the keyboard while holding a designated key, just like highlighting text with a marker. Then, a simple tap with three fingers can copy the highlighted text, and another three-finger tap can paste it elsewhere.
The beauty of these devices was not only in their ability to streamline computer interaction but also in the level of customization they offered. Users could define their own set of gestures, allowing for a uniquely personalized interaction with their computer.
Multi-touch technology works by recognizing multiple simultaneous touch points. The devices FingerWorks developed used a matrix of touch-sensitive electrodes to detect the position and motion of fingers over the device’s surface. The system then interpreted these inputs based on predefined gesture ‘vocabularies’, allowing for a wide range of actions from simple point-and-click operations to complex multi-finger chord gestures. And just like that, the finger became mightier than the mouse.
Fast forward to today, and these once groundbreaking concepts may seem somewhat ordinary. As we navigate our digital lives, we now take for granted the ability to pinch, swipe, and scroll with our fingers.
In a practical demonstration of FingerWorks’ technology, let’s hearken back to the early 90’s. Chuck Butler, a music producer from Monster Tracks Original Music, shared his experience with the iGesture Number Pad in a video interview. Butler uses the iGesture Number Pad to control his Macintosh during his audio production work. He uses software called Digital Performer, which allows him to record and edit audio and MIDI information, control synthesizers and create music productions within his computer.
Butler praises the iGesture Number Pad for its ease of use and the relief it provides from the physical strain associated with traditional mouse usage. He explains how the pad allows him to make fine adjustments to MIDI events with a single finger, a task that would typically require a strenuous grip and repetitive side-to-side movements with a mouse. This capability not only saves him from physical discomfort but also enhances his productivity by allowing continuous, uninterrupted dragging of events.
Furthermore, Butler appreciates the iGesture Number Pad’s ability to mimic the tactile experience of a physical mixing board. He can adjust audio levels by moving his finger up and down on the pad, just as he would with a fader on a hardware mixing console. This intuitive interaction is difficult to achieve with a traditional mouse, which primarily supports left-to-right movements.
Most importantly, Butler credits the iGesture Number Pad with saving his piano-playing career. Before using the pad, he experienced excruciating pain in his right wrist due to the strain of mouse usage, threatening his ability to play the piano. After switching to the iGesture Number Pad, he noticed a significant reduction in his wrist pain within two weeks, and complete relief within a month. Since then, he has been using the pad for two years without any recurrence of the problem.
The success and profound impact that the iGesture Number Pad had on Butler’s career is just one example of the transformative power of FingerWorks’ innovative technology. The tech industry, always on the lookout for novel and disruptive inventions, quickly took note. It wasn’t long before this small but significant device, which had already begun changing the way people interacted with their computers and potentially saved careers, caught the attention of a prominent player in the field.
In 2005, all the assets of FingerWorks were acquired under a veil of secrecy. The buyer remained unnamed at first, adding an element of intrigue to the transaction. However, it wasn’t surprising to find out later that it was Apple, a company that has consistently been at the cutting edge of technological innovation, which made the strategic acquisition. This purchase of FingerWorks and its impressive portfolio of multi-touch patents marked yet another pioneering move by Apple, always keen to reshape and redefine the user experience.
In a span of just a year and a half, the diligent research efforts within the company underwent a transformative evolution. They took the groundbreaking multitouch ideas pioneered by FingerWorks and applied them to create a transparent layer of a capacitive touchscreen. The same technology that saved Butler’s career would now be sold as an integral part of Apple’s newest flagship.
This innovation paved the way for the remarkable multitouch manipulation showcased by the iconic iPhone in January 2007. The once opaque trackpad surface was now replaced by cutting-edge touchscreen technology, ushering in a new era of intuitive and immersive user interaction.
During the introduction of the iPhone, Steve Jobs fearlessly declared that the era of stylus-driven interfaces, once championed by Microsoft’s Gates, was no longer worth exploring. With a touch of boldness, Jobs posed the question, “Now, how are we going to communicate this?” Rejecting the idea of carrying around a mouse or a stylus, he expressed the sentiment that nobody wants to deal with the hassle of a stylus — having to retrieve it, put it away, and the ever-present risk of losing it. Steve even hated the idea of an Apple Pencil.
When Jobs dismissed the stylus, he was thinking of devices like PDAs and early smartphones, which required a stylus for basic navigation. In Jobs’ mind, the stylus was a clumsy tool for these tasks, hence the pivot towards a multi-touch interface.
The Apple Pencil we have today was designed for a completely different purpose: to enhance the capabilities of the iPad, particularly in creative fields like art and design. Unlike the styluses of old, the Apple Pencil was never meant to replace your finger for basic navigation.
Instead, it was developed to unlock a new level of precision and control that fingers simply couldn’t achieve.
Jobs proposed an ingenious solution on stage. He proclaimed, “We’re going to use the best-pointing device in the world. We’re going to use a pointing device that we’re all born with — born with ten of them. We’re going to use our fingers. We’re going to touch this with our fingers.”
Thus, the revolutionary concept of multi-touch was born. Jobs marveled at this groundbreaking technology, describing it as phenomenal and magical. It surpassed the accuracy of any touch display ever shipped, effortlessly ignoring unintended touches and showcasing unparalleled intelligence. Multi-touch allowed for the seamless execution of multi-finger gestures, a testament to its brilliance. And, of course, Apple had already secured a treasure trove of patents to protect this innovation.
Reflecting on Apple’s remarkable track record, Jobs acknowledged their knack for bringing revolutionary user interfaces to the market. The first was the mouse, followed by the click wheel. And now, they were poised to unleash the power of multi-touch. Each of these breakthrough interfaces paved the way for transformative products: the Mac, the iPod, and now, the iPhone. With Jobs at the helm, Apple was ready to revolutionize the way we interacted with technology once again.
This move revolutionized the smartphone industry and set the standard for nearly all touch-based mobile devices that followed. By incorporating multi-touch technology, Apple made it possible for users to pinch to zoom, swipe to scroll, and tap to select, making the user experience more interactive and intuitive than ever before.
In the wake of the iPhone’s groundbreaking arrival, Microsoft stepped forward with its own multitouch system, a captivating piece of technology called the Surface. This camera-driven, tabletop appliance aimed to create a grand information kiosk that could effortlessly respond to multiple touch points and even specially barcoded objects placed on its surface. The Surface represented Microsoft’s vision for the future of interactive computing.
However, while Microsoft showcased its ambitious Surface, the company continued to cling to its stylus-oriented interface for its struggling Tablet PC devices and Windows Mobile smartphones. The promise of a touch-based upgrade, akin to the iPhone’s seamless interaction, was repeatedly delayed due to technical difficulties. Windows Mobile 7, with iPhone-like touch features, remained a distant hope, pushed back until early 2011 — a full four years after the iPhone’s groundbreaking debut.
Meanwhile, Palm, a company facing its own challenges and with far more limited resources, managed to deliver its own multitouch device, the Palm Pre, a mere three years after the iPhone hit the market. This remarkable achievement was made possible by Palm’s infusion of Apple talent. Executive Jon Rubinstein, alongside a team of skilled engineers who had previously worked on the iPhone, joined forces to bring the Palm Pre to life. Their collaboration resulted in a device that showcased the power and potential of multitouch technology, even in the face of stiff competition.
Palm and Microsoft weren’t the only companies attempting to follow in Apple’s footsteps, but many found themselves treading on treacherous ground. The path to innovation was fraught with the challenge of navigating Apple’s extensive portfolio of patented technology. Research In Motion (RIM), known for its BlackBerry devices, encountered significant hurdles in developing the iPhone-like BlackBerry Storm. The journey proved to be both arduous and problematic for RIM.
Similarly, Google embarked on a cautious journey to transform its Android platform from a “Windows Mobile killer” into a touch-centric system that could rival the iPhone. However, the task of achieving this without stepping on Apple’s multitouch patents presented a formidable challenge. The early Android prototypes were reminiscent of Windows Mobile devices, featuring numerous physical buttons. But over time, they evolved to embrace the touch interface features popularized by the iPhone, striking a balance between innovation and avoiding potential legal disputes.
As the mobile industry raced to catch up with Apple’s groundbreaking iPhone, companies faced the daunting task of emulating its success while avoiding patent infringement. Palm emerged as a remarkable contender, driven by a fusion of Apple talent, while Microsoft grappled with delayed touch-based upgrades. RIM and Google navigated the delicate balance between innovation and legal complexities, striving to offer iPhone-like experiences without falling afoul of Apple’s patented technology. It was a tumultuous journey, one that showcased both the impact of the iPhone and the challenges faced by those seeking to replicate its triumphs.
Despite the considerable passage of time since FingerWorks’ inception, the impact of their technology is still evident today. Multi-touch technology is now a staple in a wide range of devices, from tablets and smartphones to laptops and interactive kiosks. It’s even found its way into the realm of wearable technology and gaming.
Smartphones and tablets, for instance, have seen continuous improvements in multi-touch technology. This has led to more responsive touchscreens that can distinguish between different levels of pressure, allowing for a wider range of user inputs.
The development of pressure-sensitive or “3D Touch” technology was a significant step in this direction. Furthermore, advancements in gesture recognition have allowed users to control their devices using multi-finger gestures, further enhancing the user experience.
In the realm of laptops, multi-touch technology has revolutionized the functionality of trackpads. Modern trackpads can recognize multiple points of contact and a variety of gestures, enabling users to navigate their devices with increased precision and efficiency. This technology has also paved the way for the advent of touchscreen laptops, which offer an even more interactive user experience.
As for the pioneers behind FingerWorks, both Westerman and Elias have continued to contribute to Apple’s legacy of innovation. As of 2023, Dr. Wayne Westerman serves as a multi-touch architect at Apple, developing new ways to integrate and improve touch technology within Apple’s product line. John Elias, on the other hand, has been instrumental in Apple’s patent portfolio, being listed as an inventor on a number of patents related to touch technology and magnetic tracking systems.
Among the patents associated with Elias are systems for compact multi-user magnetic tracking, which explore innovative methods to track multiple users’ interactions in a compact and efficient manner. This paves the way for enhanced multi-user experiences on touch-enabled devices.
Additionally, Elias has contributed to the development of touchscreen liquid crystal displays that seamlessly integrate touch-sensing elements with display circuitry.
This integration allows for more streamlined and efficient touch experiences on Apple devices. Another notable patent involving Elias is the dynamic current control of a transmitter for magnetic proximity sensing. This innovation optimizes the performance of magnetic proximity sensors, enhancing their accuracy and responsiveness.
The story of FingerWorks is a testament to the power of innovation and the impact that a small team of dedicated individuals can have on the world. From their humble beginnings to their acquisition by one of the world’s leading tech companies, Westerman and Elias have truly left their mark on the world of technology. Their work continues to influence the way we interact with our devices today, making our digital experiences more intuitive and engaging.
Their legacy is embodied in every swipe, pinch, and tap that we make on our devices. It’s in the way we casually zoom in on a map or effortlessly switch between apps. They redefined our interaction with technology, making it more intuitive and engaging.
Next time you use your multi-touch device, take a moment to appreciate the genius that went into its creation. Consider the depth of innovation that allows such complex technology to feel so simple and natural. It’s the culmination of years of research, countless hours of development, and an unwavering belief in an idea.
It’s not just about creating something new, but also about significantly enhancing the way we live our lives. By pushing the boundaries of UX and UI design, we have a technology that has become a natural extension of ourselves, making our interaction with digital devices as intuitive as turning the pages of a book.
This is the legacy of FingerWorks — a testament to the power of innovation and the enduring impact of two visionaries. Let it inspire us to continue pushing the boundaries of what’s possible because in the realm of technology, every touch has the potential to transform the world. And who knows, maybe one day we’ll be scrolling through documents with just a flick of the wrist, or pausing our favorite show with a double snap of our fingers. Until then, let’s embrace the touch revolution.
