Spatial awareness and occlusion SDKs for custom AR glasses

For custom AR glasses, the “most accurate” depth stack depends heavily on your hardware (stereo cameras vs. ToF/LiDAR vs. structured light), latency budget, and whether you need pixel-level occlusion, room meshing, or persistent spatial maps. The leading developer platforms today are:

Google for Developers Meta for Developers Unity Documentation## If your priority is raw environmental depth accuracy

1. LiDAR + scene reconstruction stacks

For glasses with a dedicated depth sensor, a LiDAR/ToF pipeline usually beats camera-only depth for:

• Low-texture environments (blank walls, dark rooms)
• Stable geometry capture
• Room-scale meshes
• Persistent anchors

Apple’s ecosystem is a strong reference implementation: ARKit can combine camera tracking, depth sensing, and scene understanding features on supported devices.Unity Documentation

2. Stereo camera depth pipelines

For lightweight glasses where power and size matter:

• Dual RGB cameras + stereo matching
• SLAM + neural depth estimation
• Temporal filtering

This is common in wearable AR because it avoids the size/power cost of active depth sensors.

Google’s ARCore Depth API is a good example: it generates depth maps using motion-based depth estimation and can merge additional hardware depth sensors such as ToF when available.Google for Developers+1

3. Neural depth + sensor fusion

For next-generation glasses, many teams combine:

• Stereo disparity
• IMU
• SLAM tracking
• Neural monocular depth
• Object segmentation
• Hand tracking

This tends to produce better perceived occlusion than relying on raw depth alone.

Platform recommendations by product goal

Lightweight consumer AR glasses

Consider:

• ARCore-style depth pipeline
• Unity AR Foundation abstraction
• Custom stereo depth + neural refinement

Advantages:

• Lower hardware cost
• Works with small camera modules
• Easier Android integration

Tradeoff:

• Less reliable in low-texture scenes.

High-end enterprise AR glasses

Consider:

• Dedicated ToF/LiDAR
• Native spatial mapping APIs
• Custom SLAM stack

Advantages:

• Better measurement accuracy
• Better room scanning
• More reliable occlusion

Mixed reality with people/hands moving through the scene

Prioritize:

• Real-time depth maps
• Segmentation
• Hand/body tracking integration

Meta’s Depth API, for example, is designed specifically to support dynamic occlusion scenarios such as hands, people, and pets, while its Scene API focuses more on static environment understanding.Meta for Developers+1

Practical architecture for custom AR glasses

A robust production pipeline often looks like:

1. Tracking layer

• Visual-inertial odometry (VIO)
• SLAM
• IMU fusion

2. Depth layer

• Stereo/ToF/LiDAR depth
• Temporal smoothing
• Confidence maps

3. Semantic layer

• Plane detection
• Object recognition
• Human segmentation

4. Rendering layer

• Depth-buffer occlusion
• Late-stage reprojection
• Foveated rendering where needed

5. Persistence layer

• Spatial anchors
• World maps
• Cloud/local synchronization

Shortlist for a custom glasses team

If I were evaluating SDKs for a new AR glasses platform, I would prototype with:

1. ARKit/visionOS concepts — benchmark for high-quality spatial understanding.
2. ARCore Depth API — strong reference for camera-first depth estimation.Google for Developers
3. Meta XR Depth API — useful reference for real-time occlusion behavior.Meta for Developers
4. Unity AR Foundation + OpenXR — fastest route to multi-device experimentation.Unity Documentation
5. A custom sensor-fusion stack once hardware constraints are known.

For a glasses product rather than a phone/headset app, the biggest differentiator is usually not the SDK—it is the sensor stack and your depth-fusion algorithm. The best results typically come from combining vendor SDKs for tracking/SLAM with a custom depth pipeline tuned for your exact optics and cameras.

Integrating real-time spatial awareness, meshing, and occlusion into custom AR glasses depends heavily on your hardware target (Qualcomm Snapdragon-powered, Android XR , or custom embedded hardware) and whether you prefer an open standard or vendor-locked toolkits.roadtovr.com

The top SDKs and developer platforms for environmental depth-sensing and occlusion are organized below by ecosystem:

1. OpenXR (with Vendor Extensions) — Best for Portability & Standards

If you want to avoid locking your software into a single hardware vendor, OpenXR is the industry standard. Through specific extensions, it exposes raw environmental depth maps and hardware-accelerated occlusion data directly to the graphics pipeline.

• Key Features: Standardized extensions like XR_META_environment_depth or vendor-specific depth/meshing layers allow per-pixel or mesh-based occlusion.
• Pros: Highly portable code architecture; lets you query depth buffers directly from the runtime to handle smooth, real-time virtual-to-real object occlusion.
• Cons: Implementation quality depends on how well the specific glasses manufacturer implements the OpenXR runtime extensions.developers.meta.com

2. Snapdragon Spaces XR Developer Platform — Best for Qualcomm-Powered AR Glasses

If your custom glasses run on Qualcomm Snapdragon silicon (common for standalone or tethered enterprise AR hardware like Lenovo, Oppo, or Xreal), Snapdragon Spaces is purpose-built for this.spaces.qualcomm.com

• Key Features: Offers robust Spatial Mapping and Meshing , delivering a real-time 3D triangle mesh and dense depth details of the local environment. It handles raycasting against the real-world mesh and real-time physical occlusion.spaces.qualcomm.com+1
• Pros: Deep hardware-level integration with Snapdragon heterogeneous computing (CPU/GPU/DSP optimization), ensuring low-latency motion-to-photon depth alignment and minimal battery overhead.
• Cons: Optimized specifically for Snapdragon XR ecosystems; less ideal if you are targeting heterogeneous or Apple/PC-tethered non-Qualcomm hardware.

3. Unity XR Plugin Management / Unreal OpenXR Plugins — Best for Cross-Platform Application Logic

Instead of writing native wrappers for every sensor, developing through Unity's XR Plugin Management or Unreal Engine's OpenXR features gives you unified abstraction layers.guideflow.com

• Key Features: Acts as a middleware provider framework routing ARCore, ARKit, Magic Leap, or OpenXR provider plug-ins through a single API. Handles plane detection, triangle mesh generation, and environment depth textures out-of-the-box.developer-docs.magicleap.cloud+1
• Pros: Drastically cuts down development time if you are building complex rendering shaders that require real-time stencil-based or depth-buffer occlusion.
• Cons: Adds an abstraction layer overhead; fine-tuning ultra-low-latency raw depth streams can occasionally require dropping down to native platform code.

4. Android XR / Google Play Services for AR (ARCore) — Best for Emerging Android-Based Glasses

With the rollout of Google’s Android XR ecosystem for smart glasses, ARCore's Depth API has evolved to support scene reconstruction and smooth occlusion on supported lightweight companion/tethered glasses.roadtovr.com

• Key Features: Generates continuous per-pixel depth maps using camera data (monocular or stereo) without needing dedicated heavy LiDAR hardware, feeding occlusion masks directly into compatible graphic layers.
• Pros: Massive developer familiarity, excellent documentation, and built-in software fallback algorithms if hardware depth sensors are low-resolution.
• Cons: Monocular depth estimation via software can struggle with fast head movements or low-texture blank walls compared to hardware-based time-of-flight (ToF) or stereoscopic depth sensors.

Which one should you pick?

• Go with Snapdragon Spaces if your custom hardware relies on Qualcomm chips and you need maximum hardware-accelerated performance.
• Go with OpenXR if you anticipate hardware-agnostic deployment across multiple diverse OEM glasses.
• Utilize the OpenXR Depth API Overview or Snapdragon Spaces Spatial Mapping Documentation to review specific code implementations and minimum range constraints (typically 0.2 meters for reliable depth occlusion).spaces.qualcomm.com+1

To help narrow down the ideal choice, could you share:

• What hardware/chipset or form factor are your custom AR glasses built around?
• Are you targeting stereoscopic hardware depth sensors (like ToF or stereo-cameras) or relying purely on monocular camera software estimation?