Meta, formerly known as Facebook, recently unveiled a new haptic glove prototype designed to simulate touch and feel in augmented and virtual reality environments. The experimental device features a complex array of sensors and actuators that recreate tactile sensations on the wearer’s hands when interacting with objects in AR/VR.
Meta’s research lab Reality Labs demonstrated the glove at the company’s annual Connect conference. While still an early research initiative, the prototype highlights Meta’s ambitions to achieve true sensory realism in next-generation immersive computing platforms spearheaded by the Metaverse.
How the Meta haptic glove works
The Meta haptic glove consists of a lightweight fabric glove covered with a network of miniaturized actuators to create tactile feedback. The actuators apply pressure and vibration to the wearer’s hands to simulate the feel of shapes, textures, and forces when grabbing digital objects.
The glove incorporates a suite of sensors to detect the precise pose and motion of the wearer’s fingers. The sensor data gets fed into the actuators in real-time to recreate appropriate haptic sensations tied to the wearer’s interactions. For example, the actuators change the sensations when the fingers contact a round object vs a sharp object in VR.
The glove also features tracking markers on the back that allow cameras on a VR headset to capture hand position and orientation. This provides six degrees of freedom motion tracking. The glove connects to a computer via a tether cable to process all the electronics and haptics.
Current capabilities and limitations
In its current research phase, Meta’s haptic glove enables the wearer to experience basic sensations like shapes, textures, and pressures when interacting with VR environments. However, the realism remains limited compared to human touch sensitivity.
Each glove finger contains just a single row of actuators. More actuators will likely be required to emulate fingertips capable of fine discrimination between objects. The glove’s electronics and bulky form factor also limit the hand’s freedom of movement right now.
Despite its limitations, the current prototype demonstrates promising progress toward lifelike tactile feedback for VR/AR. The miniaturized actuators represent a breakthrough in recreating sensations of touch in compact, wearable devices. Meta expects advancements in materials and haptics will enable more sophisticated versions over time.
Potential consumer and enterprise applications
More immersive touch experiences in virtual worlds could benefit both consumer and enterprise use cases such as:
– Social VR – Shaking virtual hands or giving high fives could add warmth to social connections in VR spaces like Meta Horizon Worlds.
– Gaming – Feeling textures, shapes and vibrations enhances realism in VR games, making the experience more engaging.
– Training simulations – Trainee surgeons could practice procedures and get simulated tactile feedback on tissue and instruments.
– Manufacturing – Engineers could prototype and interact with digital models, feeling tactile feedback to evaluate designs pre-manufacturing.
– Retail – Online shoppers could feel materials and fabrics from products to better assess quality before purchasing.
– Remote collaboration – Having tactile feedback during virtual meetings could facilitate more intuitive collaboration and presentations.
– Healthcare – Doctors could palpate patients in virtual exams to feel for symptoms remotely. The glove may also have therapeutic applications.
– Navigation aids – Integrated with other sensors, haptic gloves could guide people through physical spaces in an intuitive tactile manner.
Widespread adoption of VR/AR technologies relies heavily on achieving immersive realism. While visuals and audio are improving rapidly, adding compelling touch experiences remains one of the biggest challenges and opportunities. Meta’s haptic glove points the way toward more natural, intuitive digital interactions.
Challenges in developing viable consumer haptic gloves
While promising, Meta’s prototype glove highlights challenges that must be addressed to make hand haptics into a viable consumer technology.
– Miniaturization – Current electronics and actuators are bulky. Components must shrink substantially while increasing haptic fidelity in gloves people actually want to wear.
– Finger tracking – Tracking the precise pose of each finger remains complex, especially accounting for varying hand shapes and sizes.
– Power consumption – Mobile power sources are needed so gloves aren’t tethered by cables. This requires optimizing power-hungry haptic components.
– Thermal management – Processing and actuators generate heat that could become uncomfortable for wearers over time, requiring innovations in cooling.
– Material science – Researchers are experimenting with smart fabrics and materials that enable haptic feedback while retaining wearability, breathability and comfort.
– Wireless connectivity – Fast and reliable wireless connectivity is needed to deliver real-time haptic feedback without lag disrupting the experience.
– Calibration – Algorithms must be developed to properly map actuators and sensors to users’ hands to recreate authentic feel.
– Software integration – Seamless integration with graphics engines and physics simulation is critical for unified visual, audio and haptic experiences.
– Price point – Enterprise-grade haptic gloves currently cost over $5,000 per hand. For wide adoption, consumer versions must hit affordable price points closer to gaming controllers.
Achieving the glove’s full potential will require interdisciplinary breakthroughs bridging electronics, material science, haptics, software and human interface design. But early research initiatives like Meta’s prototype build confidence these challenges can be overcome in 5-10 year timeframes.
Technology powering Meta’s haptic glove
Meta’s haptic glove builds on years of internal research and acquisitions in hand tracking technology:
– Finger pose tracking – Internal computer vision research enables high resolution tracking of users’ hand motions and finger poses. External tracking markers on the glove increase tracking fidelity.
– Tactical AI research – Meta AI researchers have published papers on using artificial intelligence to model the properties of materials and surfaces for haptic replication.
– Actuators – Meta acquired haptics startup Tanvas in 2021, gaining their expertise in thin film actuators that recreate textures via patterns of micro-vibrations.
– Soft robotics – Meta acquired Ctrl-Labs in 2019, incorporating their neural interface technology to decode motor neuron signals from the arm for intuitive control.
– Wireless drivers – Meta acquired custom chip designer Ineda in 2020 to develop high speed wireless drivers optimized to sync haptic feedback with low latency.
– Wrist haptics – Meta acquired wristband haptics company Dipod Technologies in 2021, bringing in their technology for delivering touch feedback on wrists and forearms.
The glove combines these various internal research efforts and acquisitions into an integrated full hand haptic solution. The fusion of tracking, AI, soft robotics and haptics illustrates Meta’s multi-disciplinary approach toward developing next-gen immersive interfaces.
Broader ecosystem for haptic hardware innovation
Meta isn’t alone in this space. A range of startups and research initiatives are pushing haptic hardware innovation for seamless experiential computing:
– HaptX – This startup produces realistic haptic gloves for applications like remotely operating robots or drones. Their gloves recreate forces, textures and motions with microfluidics, motion tracking and exoskeleton-like frameworks.
– Ultrahaptics – This company uses focused ultrasound waves to project sensations of touch directly onto a user’s hands. The technology works in mid-air with no need to wear gloves.
– Facebook wristband – Researchers at Meta developed wristbands using pneumatic air pockets to create haptic feedback on wrists for AR/VR.
– Stanford Haptic Glove – Researchers created a glove using soft flexible skin with embedded actuators to recreate sensations of grasping and touching different materials.
– Carnegie Mellon – Scientists here developed electroadhesive pads that recreate friction and texture by controlling electrostatic forces between hands and touched objects.
R&D initiatives like these are racing toward the vision of comfortable, highly realistic haptic gloves. They point to a future where touch and feel become as integral to metaverse experiences as sight and sound.
Recent advancements around haptic vests and other haptic wearables
In addition to haptic gloves, researchers are also developing haptic-enabled clothing and accessories to expand tactile feedback across the body:
– Haptic vests – These wearables contain a matrix of actuators to deliver vibrations, presses, and spatialized effects simulating physical interactions on a person’s torso. Some models also recreate temperature sensations.
– Haptic belts – Belts with embedded eccentric rotating mass motors vibrate to provide haptic directional guidance by signaling left, right, front or back to the wearer.
– Haptic arm bands – Wrist and forearm bands use compressed air, eccentric motors or electroadhesion to simulate touch, taps or squeezes on lower arms.
– Haptic headbands – These create tactile vibrations on heads to signal direction. They are being developed to enhance accessibility for navigation and spatial awareness.
– Haptic shoes – Speciality shoes and insoles with embedded actuators send vibrations through feet to simulate textures and terrain like sand, mud, pavement, etc.
– Haptic exosuits – Emerging powered exoskeletons are gaining haptic feedback capabilities to recreate forces on the body from interactions in the environment.
These examples demonstrate the scope of haptic wearables R&D targeting wider touch realism. As options expand, developers gain more possibilities to make immersive experiences truly surround and engage users both physically and emotionally.
Incorporating temperature, texture and friction
To create convincing tactile illusions, haptics researchers are also working to incorporate sensations like temperature, friction and diverse textures. Approaches include:
– Thermal modulation – Materials like peltier junctions, wax actuators and resistive coatings alter heat flow to warm or cool users’ skin.
– Electrostatic friction – Varying electrostatic forces between skin and touchpads modulate perceived texture and friction of surfaces.
– Ultrasonic vibration – High frequency vibration patterns can emulate rough, bumpy or slick textures against users’ fingertips.
– Actuator arrays – Small, high density arrays of actuators allow more control over distribution of pressures to recreate intricate textures.
– Variable adhesion – Smart materials like gecko-inspired adhesives or electroadhesion can dynamically vary stickiness and traction on users’ skin.
– Shear modulation – Lateral stretches and compressions applied in ulnar/radial patterns simulate different viscous and textural properties.
– Mid-air effects – Focused ultrasound fields project tactile pressure without requiring contact with physical actuators.
Advancing these areas will remove remaining sensory gaps compared to interactions in the physical world. This helps pave the path for touch illusions indistinguishable from reality.
Evolving VR controllers with haptics
VR controller designs are also evolving to incorporate more advanced haptic feedback:
– Oculus Touch – Vibrations recreate hand impacts and gestures like giving a thumbs-up. Future versions may add tactile feedback.
– Valve Knuckles – These controllers use electromagnetic actuators to stop hand motion when grasping virtual objects so they feel solid.
– HTC Vive Trackers – Attachable to wrists or objects, these modules provide vibration haptics to expand possibilities for VR experiences.
– Pico Interactive Kit –Contains haptic ring and wearable bands that deliver touch sensations matched to VR environments.
– Sense Glove – An add-on glove module for VR controllers with an actuator matrix providing shear and pressure effects.
– CaptoGlove – This force feedback glove for enterprise VR training enables programmable resistance when interacting with virtual objects.
More advanced haptics in standard VR controllers reduce the need for separate gloves or addons for basic tactile effects. This helps expand adoption while seeding expectations around touch interactivity in VR.
The role of AI in simulating touch experiences
Artificial intelligence has an important role in filling the gap between available haptic hardware capabilities and the brain’s expectations for realistic touch. AI techniques help generate plausible tactile simulations including:
– Procedural textures – Algorithms synthesize variations of textures and sensations rather than pre-programming each surface.
– Physics inference – Neural networks estimate appropriate tactile feedback by analyzing visuals of deformable objects and hands.
– Audio-haptic transfer – Machine learning transfers audio properties like loudness and pitch into haptic effects users feel.
– Predictive modeling – AI models predict expected haptic, visual and audio outcomes from a given interaction to guide simulations.
– Adaptive actuators – Using sensor input, AI dynamically controls actuator intensity to optimize power efficiency and comfort.
– Noise cancellation – Machine learning filters out distracting artifacts from haptics hardware to sharpen tactile resolution.
– Psychophysical models – Computational models of neuroanatomy predict the vibrational frequencies, forces and patterns that best mimic real touch for humans.
AI fills in gaps in current haptic fidelity and bandwidth limitations by crafting plausible tactile illusions. It also compensates for delays or sensor errors to sustain immersion. Over time, AI techniques will enable more dynamic, responsive and realistic tactile experiences from limited hardware.
Outlook for mass consumer adoption of haptic tech
While still in early R&D phases, rapid progress on devices like Meta’s haptic glove increases confidence consumer-ready haptic interfaces could arrive within 5-10 years. But what will mainstream adoption look like?
– Phased approach – Initially expect accessories like haptic gloves and wearables working in tandem with VR/AR headsets and controllers. Fully integrated solutions will follow.
– Purchase bundles – Haptics may be first sold as add-on bundles with headsets rather than standalone offerings. Subscription bundles are also possible.
– Iterative enhacements – Hardware capabilities will improve gradually over generations rather than immediately achieving full realism.
– Developer support – Universal integration APIs will make adding haptics easy across engines like Unity, Unreal, etc. to drive adoption.
– Retroactive enhancements – Older VR/AR content will gain new tactile depth by applying real-time haptic effects using sensory algorithms.
– Competitive pressure – Once pioneered, other AR/VR players will race to add advanced haptics to avoid sensory gaps between platforms.
While technological limitations remain, rapid progress in advanced haptics brings the vision of fully immersive digital experiences tantalizingly closer. In the coming years, the sensory barriers between virtual and physical worlds will erode further, unlocking new horizons for interactive applications that engage our senses both digitally and physically.
