User Guided Localization

A guided flow within Meta's mixed-reality platform that lets people place, anchor, and fine-tune virtual content against their physical space, so experiences stay correctly positioned across sessions and rooms.

Mixed reality experiences rely on precise alignment between virtual content and physical space, but automated localization isn't perfect — content can drift, misalign in unfamiliar rooms, or fail to persist correctly between sessions. When alignment breaks, the experience breaks with it, and users had no way to fix it themselves without exiting the experience entirely.

I designed the user-guided localization flow — the interaction layer that lets people step in when automated placement isn't quite right. My approach focused on giving users just enough control to correct alignment confidently, without requiring any understanding of the spatial tracking happening underneath.

I explored how much manual control users needed versus how much should stay automated, testing interaction patterns for placing, nudging, and locking virtual content against real-world surfaces. I iterated on feedback cues — visual anchors, snapping behavior, confirmation states — so users could tell at a glance whether their space was correctly localized before committing.

When a user enters a mixed-reality experience in a new or changed room, the system attempts automatic localization. If alignment is off, the user is guided through a simple flow to anchor and fine-tune virtual content against their physical space — adjusting placement directly until it locks in. Once confirmed, that alignment persists, so the experience stays correctly placed across future sessions in the same room.

Spatial tracking accuracy varies by device sensors, lighting, and room geometry, so the guided flow had to account for a wide range of real-world conditions rather than a single controlled environment. The interaction also had to stay simple enough for users unfamiliar with spatial computing concepts, while still giving enough precision for content to persist reliably across sessions.

Before: users had no way to correct misaligned virtual content — experiences that drifted or failed to localize correctly simply broke, with no in-experience recovery path. After: a guided flow lets users anchor and fine-tune content themselves, keeping experiences aligned across sessions and rooms without needing to exit or restart.

At Meta, I designed a user-guided localization flow for mixed reality — helping people place, anchor, and fine-tune virtual content against their physical space, giving users a reliable way to recover alignment when automated tracking fell short and keeping experiences consistent across sessions and rooms.

As spatial tracking technology matured, the localization flow evolved toward requiring less manual correction — with future iterations aimed at reducing how often users need to intervene at all, while keeping the guided flow available as a reliable fallback.