I’ve been running a small experiment over the last six weeks. Every VR session I do — whether it’s casual gaming, work in immersive collaboration tools, or testing for review purposes — I default to hand tracking instead of reaching for controllers. The goal was simple: figure out, honestly, whether hand tracking in mid-2026 is actually good enough that controllers can become the exception rather than the rule.

The short answer is “almost, for some things, never for others.” The longer answer is more interesting.

What’s actually improved

The technical progress over the past eighteen months is real. The current generation of headsets — Quest 3S, Vision Pro, Pico 5, and the various business-focused devices — are all running hand tracking systems that would have been considered remarkable in 2024.

Specifically, three things got better.

Latency dropped meaningfully. The end-to-end delay from a hand movement to its representation in the headset display has come down to the 25-40ms range on the better devices, which is below the threshold where most users consciously notice lag. In 2023 the same number was typically 60-80ms.

Tracking through occlusion improved. Older systems lost the plot the moment one hand crossed in front of the other or when fingers curled out of camera view. The current generation handles these scenarios competently, partly because predictive models trained on hand pose data are filling in gaps that the cameras can’t directly see.

Pinch detection got reliable. The pinch gesture has effectively become the universal “click” for hand tracking interfaces, and the false positive and false negative rates on it have dropped to a level where it doesn’t actively annoy users. This sounds minor, but it was the single biggest source of frustration with previous generations.

Where it works well right now

Browsing menus, navigating spatial interfaces, casual interaction with virtual objects, conversational interaction in social VR, and most enterprise collaboration scenarios are all handled well by current hand tracking. If you’re sitting in a VR meeting, taking notes on a virtual whiteboard, or moving around a 3D model with your colleagues, you don’t need controllers. The interaction model has matured to the point where it feels reasonably natural.

Apple’s Vision Pro implementation deserves specific mention here because it pushed the industry on the idea that hand tracking could be the primary input rather than a fallback. The combination of eye tracking for targeting and hand pinching for selection has become a template that other vendors are now imitating with varying degrees of success.

For productivity-style work, I’ve genuinely stopped reaching for controllers. The friction of putting them down, picking them up, and managing batteries has become more annoying than the occasional missed gesture.

Where it still falls apart

Three categories of use case still need controllers, and they’re not edge cases.

Anything requiring tactile feedback. Hand tracking provides no physical sensation when you grip a virtual object, swing a virtual tool, or push a virtual button. For training applications where the muscle memory of physical interaction matters — and there are a lot of these — controllers with haptics remain meaningfully better.

Precision tasks under load. Drawing in 3D, fine sculpting work, surgical training simulations, anything where small movements need to translate to small results. Hand tracking introduces tiny tremors and jitter that controllers, with their physical handles you can grip steadily, don’t have.

Sustained gaming. This is the one that surprised me. Hand tracking for a 20-minute casual session is fine. Hand tracking for a two-hour gaming session produces fatigue in shoulders and forearms that I genuinely don’t experience with controllers. It’s a posture issue: holding hands in the camera’s field of view requires more sustained muscle engagement than resting hands on controllers.

The developer story

Building applications that work well with hand tracking is harder than building for controllers, and developers are still working it out. The control schemes that work well for controllers — analog sticks, multiple buttons, triggers, grip squeeze — don’t translate directly to hand gestures, and trying to map them often produces awkward interactions.

The applications I see succeeding are the ones that redesigned their interaction model from scratch around hand tracking, rather than retrofitting controller-based designs. That’s a significant engineering investment, and most studios haven’t done it. The result is a software ecosystem that’s still controller-first with hand tracking as a sometimes-supported alternative, rather than the other way around.

The tooling is getting better. Meta’s hand tracking SDK added meaningful upgrades through 2025, and Apple’s frameworks for Vision Pro have set a high bar for what’s possible. But there’s a gap between what the platforms can do and what most apps actually implement.

What I’d actually recommend

If you’re buying a headset in mid-2026 and you care about hand tracking, the practical advice is this: it’s now good enough that you should expect it to handle most casual and productivity use cases without controllers. If your primary use case is gaming, fitness, or any application where you’ll be using the headset for hours at a stretch with sustained input, controllers are still the better tool.

For enterprise deployments — training, design review, remote collaboration — hand tracking is now mature enough to be the default, with controllers available for the specific workflows that need them. Custom enterprise rollouts, the kind that tend to involve custom AI builds and bespoke interaction design, are increasingly the testbed where hand-tracking-first interfaces get refined before flowing back to consumer software.

We’re not at the point where controllers are obsolete. We’re at the point where the conversation has flipped from “is hand tracking usable yet” to “what specific use cases still need controllers.” That’s a real shift, and it happened faster than I expected.