oktober 20, 2020
In the 2002 film Minority Report, Tom Cruise’s character stands in front of a wraparound screen, puts on some cool gloves, and proceeds to interact with the digital recording of a murder that hasn’t yet happened. The ability to accurately predict and then stop crimes before they happen might forever be the stuff of science fiction, but the gesture control technology on display then was only 18 years too early.
With the world still reeling from the effects of a global pandemic and questions about what a return to normal may look like, the safest ways to get people back into stores, museums and other locations is top of mind. Conveniently, two possible answers to these questions are staring us in the face: voice control and gesture control.
Anyone looking at the most common technology would be forgiven for thinking the computer system in Minority Report is still years away from existence, but the truth is we can create these systems today. Instead of telling the world and hoping they’d listen, we decided to prove it.
More Than Just a Lot of Hand Waving
Touchscreens are on the way out. For all of the traction they gained through the last few years, all signs indicate that public trust in high-traffic touch points will be low for the foreseeable future. Conversely, the need for dynamic, interactive experiences to keep consumers interested isn’t decreasing. Keeping the touch out of touchpoints is a challenge facing all brands.
Creating a program that can run without needing a person to touch a screen is one thing, but what about creating one that doesn’t require touching anything at all? With some basic hardware to which most companies will already have access, or that they can acquire with little needed investment, gesture control is a readily available option.
The first POC we want to showcase is one in which simple hand gestures can be used to control information displayed on a screen.
With this POC, you can see how a camera can recognise common gestures like a thumbs up or down or even the peace sign and can translate those inputs into actual commands like scroll forward or backward. This is a simple setup needing a camera and some code on the back end and because of that, it could quickly be implemented by most companies right now.
The far-reaching implications of this technology could lead to interfaces that effortlessly translate sign language, giving hearing impaired individuals the ability to communicate with associates without the need of an interpreter. More immediately, this could be used to create fun games and interactions at theme parks, or even for programs like streaming services to allow users to navigate their music or film libraries at home.
However, for retailers and other service providers, the most immediate use can be seen in the ability to allow consumers to leave quick, unambiguous feedback about an experience. A thumbs up or down to a screen upon exiting a store could allow you to take the temperature of customers and get real time feedback. If coupled with a voice interface, deeper responses could be recorded by giving respondents the ability to expound on their rating vocally with their feedback being recorded for internal use. All without needing anyone to touch a public screen.
The most holistic system wouldn’t simply rely on computer vision to recognise standard gestures. Incorporating swipe control gestures to this kind of interface is the stronger solution. With something like this, customers could swipe backward and forward and up and down to control the information visible on a screen. And then through a gesture such as the thumbs up, they could make selections.
Imagine someone in an airport looking to get food but not wanting to disrupt the social distancing protocols necessary to keep everyone safe. At one nearby kiosk, the traveller could gesture control a screen featuring the menus of each nearby restaurant—up and down swipes could scroll through a single menu while right and left swipes move through the different restaurants available. When they come to an item they want, a thumbs up gesture could add that menu item to their cart. Then through QR codes or even a connected app for the airport on their phones, they could complete the purchase, pay and have their order sent directly to the kitchen of their choice. Once their meal is ready, an associate could bring it to wherever they’re sitting, or they can be alerted to pick it up from the nearby contactless pickup window.
Putting Power in Consumers’ Pockets
A common push back against gesture control tech in public spaces is that people might not be as interested in making use of it for fear of looking stupid. One easy way around this is to put these updates in games first as people are more likely to do things that might seem a little ridiculous in public in the name of fun and being crowned a winner. Another option, however, is to move away from gesture control and move more into the direction of giving people the ability to control public screens through their smartphones.
Turning a person’s phone into a universal remote still eliminates the need for a high touchpoint area. To avoid the problem of needing someone to download yet another app or to opt in to sharing their data, QR codes are used. Making your touchless experience web-based immediately increases the audience’s likelihood to make use of it.
This POC takes a simple concept that could be used by museums and natural science centres to deliver in-depth information without requiring a new touchpoint. Users in front of an exhibit simply scan a QR code and can change and interact with the display on the screen. Anyone wanting to interact with the display can use their own phone to do so, removing concerns about sanitisation.
Ultimately, the best experience will be created by incorporating both gesture and smartphone remote manipulation into the same interface. This would appeal to the largest segment of end users while accomplishing the same goals around sanitisation and social distancing.
Each of the POCs on display will have their own barriers to usability and accessibility, but the truth is that the future is contactless. It can be hard to tell just how ready a userbase is for such a major change until the tech is in people’s hands, figuratively speaking. The good news is that each of these POCs is relatively easy and inexpensive to implement. Innovating a better, safer tomorrow that still gives users the kind of experience they deserve is more possible now than it has ever been before, so long as you’re ready to take action.
