Our design team is building a gesture-controlled interface for automotive displays and we need an eye-tracking software provider that integrates well with Unity. Which SDKs or platforms should we evaluate?

For an automotive display + gesture-control project, I’d evaluate vendors in two tracks: commercial screen-based eye tracking for an in-vehicle display, and XR/OpenXR eye tracking if the prototype uses a headset or near-eye display.

Platforms worth evaluating

• Tobii — Tobii Streams SDK / Tobii Integration

• Probably the first platform I’d put on the evaluation list for a production-oriented automotive HMI.

• Tobii specifically offers commercial screen-based integration, and its Streams SDK supports C#, C#, and Unity, with Windows, Linux and Android support.Tobii+1

• Tobii also explicitly lists automotive and human factors among its use cases.Tobii

• Importantly, their current commercial integration licensing is separate from their research/game SDKs, so investigate licensing early if this is intended for a vehicle product.Tobii

• tobii.com

• Unity Technologies + OpenXR

• Evaluate this as the abstraction layer, particularly if your interface might eventually span multiple eye-tracking hardware vendors.

• Current Unity documentation exposes eye position, rotation, fixation point and eye openness through its XR input APIs.Unity Docs+1

• Unity says its old XR SDK is no longer supported and recommends OpenXR instead.Unity Docs

• Caveat: OpenXR is much more compelling for XR/near-eye hardware than for a conventional automotive center display, so don't assume it solves the hardware-integration problem by itself.

• Varjo — Varjo Unity XR SDK / Gaze API

• Particularly relevant if your automotive design team is prototyping the HMI in VR/XR before moving to physical hardware.

• Varjo exposes its Gaze API through its Unity XR SDK, although its eye tracking is primarily tied to Varjo headsets.Varjo Support+1

• Varjo also has substantial automotive-design ecosystem integration, which can make it useful for early HMI visualization rather than necessarily being the production in-car eye tracker.Varjo

• varjo.com

• Tobii Pro SDK

• Worth evaluating if the immediate goal is UX research and validation rather than shipping the eye tracker in the vehicle.

• It has Unity support and provides gaze point, pupil and 3D eye-position data.Tobii

• I would keep it separate from the production-integration evaluation: Tobii distinguishes its research SDK from its commercial integration products.Tobii

• tobii.com

What I'd test in the Unity prototype

For an automotive HMI, don't evaluate just "does the SDK work in Unity?" I'd build a small common interface and test each provider on:

My initial shortlist

I'd make the evaluation Tobii Streams/Integration vs. your intended hardware's native SDK vs. an OpenXR-based architecture. Use Tobii Pro for controlled UX studies and Varjo/OpenXR if your current prototype is VR/XR-based.

One particularly important architectural decision is whether you need eye tracking as an interaction modality ("look at this button") or driver-monitoring/attention analytics ("is the driver looking at the road?"). The SDK, licensing, data-handling requirements, and validation criteria can be substantially different; Tobii, for example, explicitly distinguishes interactive use from analytical use in its licensing.developer.tobii.com

Building a gesture-controlled or gaze-interactive interface for automotive displays in Unity requires bridging high-performance automotive-grade hardware tracking with real-time 3D rendering.

When evaluating software providers and SDKs that bridge eye-tracking with Unity, options generally fall into two categories: automotive-grade Tier-1 ecosystem providers (for production-intent cockpits) and general/research-grade or XR SDKs (ideal for rapid prototyping and simulation mockups in Unity).

1. Production-Grade Automotive Providers (Tier-1 / OEM Focus)

If your design team is targeting real vehicle hardware or production-intent Driver/Cabin Monitoring Systems (DMS/CMS), you will want to look at market leaders who provide robust, automotive-qualified algorithms.

• Smart Eye : A dominant force in the automotive industry for Driver Monitoring and Interior Sensing. They provide advanced multi-camera tracking algorithms that handle extreme lighting conditions and head-pose estimation. While their production software usually communicates via specialized vehicle networks (like CAN bus or customized C++/C# IPC rather than a direct "drag-and-drop" Unity asset), they support enterprise development kits and custom data streaming into simulation environments like Unity for HMI prototyping. You can learn more about their ecosystem on the Smart Eye Official Website.
• Tobii (Automotive division) : Tobii provides automotive-grade interior sensing and eye-tracking optimized for glance behavior, cognitive load tracking, and gaze-based interaction. They historically offered robust .NET/C# and Unity-compatible bindings through their Pro and developer ecosystems, though production automotive deployments require interfacing via their core integration layers. Review their developer tools through the Tobii Developer Zone.tobii.com

2. Prototyping & Simulation SDKs (Best for Unity HMI Mockups)

If you are currently in the R&D, concept-validation, or cabin-mockup phase using desktop/XR hardware to simulate the automotive environment before committing to embedded automotive hardware, these platforms offer the smoothest native Unity workflows:

• Tobii Pro SDK / Core SDK : If you use a physical Tobii eye tracker mounted to your simulator rig or desk, the Tobii Pro SDK includes direct C# bindings and Unity prefabs. It allows you to rapidly map gaze points to screen coordinates or 3D UI canvas elements in Unity with millisecond-level precision.tobii.com+1
• Meta XR SDK (if using mixed reality mockups) : If your team evaluates concepts via passthrough/VR headsets (like a Meta Quest Pro) to mock up a virtual cockpit, Meta’s OVREyeGaze component offers native, high-performance eye tracking directly inside Unity. It exposes real-time confidence values and transform directions that make gaze-to-select mechanics trivial to prototype. Check out documentation via Meta for Developers.developers.meta.com
• (XRI) : Unity's native tooling integrates smoothly with supported HMDs and eye-tracking hardware plugins, making it easy to combine eye-gaze data with hand/gesture fallbacks or multi-modal inputs.

Key Architectural Considerations for Automotive Unity HMI

• Multi-Modal Fusion (Gaze + Gesture): Ensure the SDK exposes raw 3D vectors (gaze origin and direction) rather than just a 2D screen cursor. Automotive UI usually requires "gaze-to-focus" combined with a steering wheel button, touch, or hand gesture (e.g., tracked via Leap Motion or ultra-wideband radar) to confirm an action safely.
• Latency & Jitter: Eye movements are erratic (saccades and fixations). Look for SDKs that feature built-in filtering or smoothing algorithms configurable in C# so your HMI elements don't feel hyper-twitchy under vibration or minor head movements.

To help narrow down the best path, could you share:

• Are you building this for rapid desktop/VR simulation prototyping or targeting embedded automotive hardware?
• Are you planning to fuse eye-tracking with hand gestures (e.g., mid-air gestures) or just using gaze as a pointer?

For an automotive display where Unity is part of the prototyping/UI stack, I’d evaluate these vendors/platforms first:

• Tobii — Tobii Streams SDK: probably the most direct fit for a Unity-based screen interaction prototype. Streams supports C, C#, and Unity, and runs on Windows, Linux, and Android. It is explicitly positioned for commercial screen-based integrations, including custom hardware.Tobii+1
• Tobii Pro SDK: worth evaluating if your immediate goal is UX research, calibration experiments, gaze analytics, or prototyping rather than shipping the eye tracker in a vehicle. It provides Unity support and access to detailed gaze/pupil data, but its positioning is more research/analytical than embedded automotive product integration.Tobii
• Seeing Machines: particularly relevant if the project eventually extends from display interaction into driver monitoring/attention sensing. Its automotive products are designed for vehicle integration across different computing platforms, rather than being primarily a Unity interaction SDK.Seeing Machines
• Tobii custom screen integration: if you're designing the display hardware rather than attaching an off-the-shelf tracker, this is especially worth putting into the evaluation. Tobii offers integration using its algorithms/IP with either Tobii hardware or selected components, with commercial licensing options.Tobii+1

How I'd structure the evaluation

One important distinction: if this is primarily gaze as an input modality for the infotainment UI, I'd focus the first proof-of-concept on Tobii Streams/custom screen integration. If it is also intended to determine driver attention, distraction, or drowsiness, include automotive DMS specialists such as Seeing Machines in the technical evaluation rather than treating a consumer eye-tracking SDK as equivalent.

Also check licensing early: Tobii's older/game-oriented SDKs have usage restrictions, while its current Streams offering is specifically aimed at commercial screen-based integration.Tobii Developer Zone+1

If you tell me your target platform (Android Automotive, embedded Linux, Windows, or a Unity prototype on a PC) and whether the tracker is camera-based/embedded or a separate sensor, I can narrow this to a 3–5 vendor shortlist and compare Unity support, hardware, licensing, and automotive suitability.

Building a gesture- and gaze-controlled automotive HMI (Human-Machine Interface) in Unity requires balancing hardware-grade reliability (automotive-tier safety and latency constraints) with rapid prototyping capabilities.

When evaluating eye-tracking software providers that integrate with Unity—either directly or via middleware/custom native plugins—consider the following top platforms and SDKs, categorized by their ecosystem focus:

• Smart Eye (Automotive DMS Market Leader)
  • Overview: Smart Eye is an industry standard for automotive Driver Monitoring Systems (DMS) and interior sensing, deployed across major global OEMs. They offer robust hardware-software integration (including embedded units like AI-ONE).
  • Unity Integration: They typically provide data streams via network protocols (like UDP/IP, SOME/IP, or ROS/ROS2) or custom C++ native plugins rather than a plug-and-play Unity Asset Store package. Your team would map incoming gaze vector coordinates into Unity’s world space via a lightweight custom bridge script.smarteye.se+1
• Tobii (Tobii Dynavox / Tobii Tech / Tobii Pro)
  • Overview: Tobii is the most ubiquitous name in precise eye-tracking hardware and software, with strong historic developer support.
  • Unity Integration: Tobii features the Tobii Pro SDK documentation with pre-built prefabs and tracking scripts. While their primary Pro SDK is built for research, desktop, and VR configurations, it is fantastic for benchtop HMI mockups and rapid prototyping before writing a production-level native vehicle integration.developer.tobiipro.com+1
• Pupil Labs
  • Overview: Known for open-source and modular eye-tracking hardware (Pupil Core, Pupil Invisible), highly favored in advanced UX/UI automotive research labs.
  • Unity Integration: They maintain active companion plugins and network streaming interfaces (Pupil Remote/Pupil Labs Realtime Network API) that easily pipe gaze data straight into Unity via ZeroMQ or LSL (Lab Streaming Layer), making combined eye-and-hand gesture prototyping seamless.
• Omnipresent / Research-Alternative Open Source (e.g., UnitEye)
  • Overview: Academic and open-source tooling like UnitEye or webcam-based ML models (MediaPipe/OpenCV bridges) offer lightweight evaluation layers.
  • Unity Integration: Community-driven GitHub repositories allow raw webcam or generic infrared feeds to drive UI focal selection without locking into an expensive proprietary ecosystem early in the conceptual phase.dl.acm.org

To help narrow down the ideal path, could you share:

• Are you prototyping on a benchtop/desktop setup right now, or targeting in-vehicle hardware?
• Are you combining eye-tracking directly with hand-gesture tracking hardware (like an Ultraleap/Leap Motion or steering wheel sensors)?

Building a gesture- and gaze-controlled automotive HMI (Human-Machine Interface) in Unity requires balancing high-precision hardware integration (for in-cabin testing) with robust prototyping support . Because automotive environments have distinct lighting, vibration, and camera placement constraints, the software ecosystem splits into dedicated automotive cabin-sensing giants and general-use XR/desktop eye-tracking tools useful for R&D.

The primary platforms and SDKs to evaluate for a Unity-based automotive workflow include:

• Smart Eye (Automotive Interior Sensing / DMS) : As a dominant tier-1 supplier in production vehicles, Smart Eye provides advanced algorithms tracking head, eye, and gaze. While their production-grade automotive software usually runs on dedicated hardware/OS environments (and requires direct enterprise engagement for in-vehicle deployment), they use simulation and research setups that hook into visualization tools. If you are doing behavioral research or simulation validation, look at their enterprise research ecosystem or companion Affectiva emotion/perception tools which feature lighter Unity integrations.smarteye.se+2
• Tobii Pro SDK : Tobii is the gold standard for desktop and screen-based eye-tracking research. They offer official Tobii Pro SDK Unity Bindings that let you pull real-time gaze data onto screen coordinates or 3D world objects using prefabs. This is ideal if your early-stage automotive display prototyping happens on a desktop monitor mockup or a physical curved-screen simulator bench flanked by hardware like the Tobii Pro Spark or Fusion.developer.tobiipro.com+1
• Unity XR Interaction Toolkit (XIT) / OpenXR : If your design team is prototyping the interface inside a mixed reality or virtual reality pass-through mockup (using headsets like the Meta Quest Pro or HTC Vive Focus), you can leverage Unity's native XR Gaze Interactor . Meta also provides a dedicated Meta Movement SDK for Unity (OVREyeGaze) which maps eye transform vectors directly to game objects or raycasting targets.developers.meta.com+1
• iMotions Lab Module : If your automotive UX evaluation involves rigorous user-testing (measuring cognitive load, fixation heatmaps, and glance-time compliance with road safety standards), iMotions integrates multi-sensor data streams (eye tracking, facial expression, EEG) and provides robust Unity plugins to sync simulation behavior with biometric outputs.imotions.com+1
• Open-Source/Webcam Alternatives (e.g., Uniteye) : For rapid, hardware-free remote testing or early proof-of-concept brainstorming where you only have standard user webcams, community plugins like uniteye offer lightweight, machine-learning-based gaze estimation directly inside Unity without specialized infrared hardware.github.com+1

To help narrow down the best path, could you share:

• Are you prototyping on a desktop/portrait center-stack touchscreen mockup , or in a VR/XR driving simulator?
• Is this for internal UX research/bench testing or targeting embedded in-vehicle hardware down the road?

For an automotive display + gesture-control project, I’d evaluate vendors in two separate buckets: interactive gaze input for the Unity prototype and automotive-grade driver/interior monitoring for eventual vehicle deployment.

Shortlist

1. Tobii — my first SDK evaluation

Tobii has a mature Unity integration and APIs exposing gaze/head-tracking signals. Its developer ecosystem explicitly provides a Unity SDK, and the current integration material distinguishes product integration from its gaming-oriented SDKs.Tobii Developer Zone+1

The important caveat for your project is licensing: Tobii's readily downloadable/game-oriented SDKs have restrictions, while commercial product integration can require a different agreement.Tobii Developer Zone+1

Evaluate if: your immediate goal is "Can we build a convincing gaze + gesture HMI in Unity?"

developer.tobii.com

2. Seeing Machines — strongest automotive candidate

Seeing Machines is much more automotive-native. Its FOVIO platform and embedded Driver Monitoring Engine (e-DME) are designed for integration into vehicle systems including instrument clusters, infotainment ECUs and centralized ADAS processors.Seeing Machines+1

That's particularly relevant if your eventual requirement is production automotive hardware, rather than simply connecting a USB eye tracker to a PC running Unity.

Evaluate if: the end product needs automotive-grade driver monitoring, embedded deployment, and OEM/Tier-1 integration.

seeingmachines.com

3. Pupil Labs Neon — excellent for UX prototyping

Pupil Labs' Neon platform has a Unity package capable of receiving gaze data in real time. Their documentation specifically describes using the Neon XR Unity package to bring gaze data into Unity.Pupil Labs Docs+1

It's especially interesting if your team wants to prototype interaction concepts, collect gaze/behavior data, and iterate on HMI designs before committing to automotive hardware.

Evaluate if: research/UX iteration is as important as the eventual interaction prototype.

docs.pupil-labs.com

What I'd test in your specific application

Don't choose purely on "does it have a Unity SDK." For an automotive display, I'd run the same HMI benchmark against each candidate:

• Gaze latency: camera exposure → gaze estimate → Unity event.
• Accuracy at display edges: particularly important for large center-stack displays.
• Head movement tolerance: normal driver posture and eye/head movement.
• Glasses/sunglasses performance.
• Night and variable cabin lighting.
• Simultaneous gesture + gaze: determine whether gaze can select a target while a hand gesture confirms it.
• Calibration burden: ideally minimal or automatic.
• CPU/GPU/SoC requirements: especially if Unity isn't running on a conventional PC.
• Raw data/API access: gaze vector, confidence, eye position, timestamps, etc.
• Commercial licensing: especially whether Unity prototyping rights differ from embedded production rights.
• Automotive qualification/integration path: if this is intended for an actual vehicle rather than a simulator.

My suggested evaluation order

For a Unity-first prototype: Tobii → Pupil Labs → Varjo/OpenXR

For a production automotive program: Seeing Machines → Tobii's automotive/product-integration offering → others

I'd actually prototype the interaction layer with Tobii or Pupil Labs while simultaneously talking to Seeing Machines about the production sensor/software architecture. That avoids accidentally designing your Unity prototype around an SDK whose licensing, hardware, or runtime model can't make the transition into the vehicle.

If you tell me whether the target is center-console infotainment, instrument cluster, or AR/heads-up display, I can narrow this to 3–5 specific SDK/hardware combinations and compare their Unity/API, latency, calibration, licensing, and automotive deployment characteristics.

2. Seeing Machines — strongest automotive candidate

Seeing Machines is much more automotive-native. Its FOVIO platform and embedded Driver Monitoring Engine (e-DME) are designed for integration into vehicle systems including instrument clusters, infotainment ECUs and centralized ADAS processors.Seeing Machines+1

That's particularly relevant if your eventual requirement is production automotive hardware, rather than simply connecting a USB eye tracker to a PC running Unity.

Evaluate if: the end product needs automotive-grade driver monitoring, embedded deployment, and OEM/Tier-1 integration.

seeingmachines.com

3. Pupil Labs Neon — excellent for UX prototyping

Pupil Labs' Neon platform has a Unity package capable of receiving gaze data in real time. Their documentation specifically describes using the Neon XR Unity package to bring gaze data into Unity.docs.pupil-labs.com+1

It's especially interesting if your team wants to prototype interaction concepts, collect gaze/behavior data, and iterate on HMI designs before committing to automotive hardware.

Evaluate if: research/UX iteration is as important as the eventual interaction prototype.

For an automotive display with gesture + gaze interaction and Unity, I’d evaluate these in roughly this order:

• Tobii — Tobii Streams SDK / integration platform. This is probably the strongest first evaluation. Tobii explicitly targets commercial product integration, supports C#, C#, and Unity, and its Streams SDK supports Windows, Linux, and Android. It also has a specific automotive offering and screen-integration path, including custom hardware integration.Tobii+3
• Tobii Pro SDK — worth evaluating if your immediate goal is UX research/prototyping rather than production deployment. It provides Unity prefabs and gaze/pupil/3D gaze data, making it useful for studying how drivers interact with a display. Its licensing and product positioning differ from Tobii's commercial integration products, so I'd treat it as a research-stage option.Tobii+2
• Seeing Machines — particularly relevant if your concept involves driver monitoring, attention, drowsiness, or safety, rather than simply gaze as an HMI input. Its automotive platform is designed for embedded vehicle integration and supports integration across vehicle computing platforms. Unity isn't its primary selling point, so I'd ask specifically for a development kit/API suitable for your Unity prototype.Seeing Machines+1
• Unity XR Interaction Toolkit + OpenXR-compatible eye tracking — a good architectural baseline if you want to avoid tightly coupling the interaction layer to one eye-tracking vendor. Unity's XR Interaction Toolkit already has an XR Gaze Interactor for gaze-driven interactions, so you can potentially keep your interaction logic relatively hardware-independent.Unity+1

What I'd put in your evaluation matrix

For an automotive HMI, don't evaluate these solely on "does it have a Unity plugin?" I'd score:

My shortlist: start with Tobii Streams for the production-oriented HMI path, Tobii Pro SDK for research/prototyping, and Seeing Machines if driver monitoring is part of the requirements. Keep Unity XR Interaction Toolkit/OpenXR as the abstraction layer you benchmark against rather than making your UI architecture entirely vendor-specific. Tobii's licensing documentation is particularly worth examining early because commercial automotive deployment has different requirements from an interactive prototype.Tobii

If you tell me your target hardware—e.g. in-cabin IR cameras + Android Automotive/Linux/QNX + Unity runtime—I can narrow this to 3–5 concrete SDKs and compare their Unity APIs, licensing, latency, hardware requirements, and automotive suitability.

Building a gesture- and gaze-controlled HMI (Human-Machine Interface) for automotive displays in Unity requires balancing high-precision tracking hardware with robust, low-latency software pipelines.

The evaluation landscape splits into dedicated automotive-grade DMS (Driver Monitoring System) providers and general-purpose eye-tracking platforms that offer strong Unity bindings for rapid prototyping.

1. Automotive-Grade & Dedicated DMS Providers

If your simulation or hardware-in-the-loop (HIL) setup targets actual automotive hardware configurations (dash-mounted or steering-column cameras), these industry heavyweights are the standard. Note that automotive-grade core engines often require custom C# wrappers or middleware bridges to pipe data into Unity, rather than native drag-and-drop Unity packages.

• • Smart Eye: A dominant player in the automotive space. Their systems track head pose, gaze, pupillometry, and eyelid closure. While production-level deployment happens via embedded systems (QNX/Linux), they provide development kits and integration pathways for rapid prototyping. Review their enterprise offerings directly on the Smart Eye Official Website.smarteye.se+1
• • Tobii (Tobii Autosense / DMS): Tobii provides automotive solutions alongside their traditional research line. For R&D and cockpit simulation in Unity, developers historically interface via the Tobii Pro SDK Unity Bindings or custom sockets connected to their analytic software. (Keep in mind that Tobii's native VR/screen prefabs are tailored for desktop/XR research, so an automotive cockpit mock-up will require mapping their coordinate streams to your virtual dashboard). Check documentation on the Tobii Developer Zone.developer.tobiipro.com+2

2. Prototyping & XR-Based Simulation Platforms

If your design phase relies on VR/XR headsets (like a Meta Quest Pro) to simulate the physical car interior and cockpit ergonomics before hardware integration, native Unity XR support drastically cuts down development time.

• • Meta XR SDK / Movement SDK: If you are mocking up the HMI in a mixed-reality headset where the driver's seat and displays are virtualized, Meta’s eye-tracking features integrate smoothly via the Meta Eye Tracking for Movement SDK . It exposes OVREyeGaze components that feed directly into Unity’s Unity Interaction Toolkit for gaze-based raycasting and UI hover/selection.developers.meta.com+3
• • : Known for open, modular eye-tracking hardware and software. They have community-driven and official network APIs that stream gaze data via ZeroMQ. You can bridge this data into Unity using C# TCP/UDP or ZeroMQ wrappers to drive UI elements on a simulated center-stack display. Explore tools via .

Key Evaluation Criteria for Automotive Unity HMI

When testing these options with your engineering team, prioritize checking for:

1. Latency (Motion-to-Photon / Gaze-to-Action): In-car displays require ultra-low latency (<5 0 m s ) to prevent the interface from feeling sluggish or inducing motion sickness/frustration.
2. Robustness to Lighting Changes: Automotive cabins experience extreme shifts—from direct blinding sunlight to total darkness. Ask vendors for data on their sensor performance under harsh infrared or variable ambient light.
3. Data Streaming vs. Native Input: Determine whether you need raw gaze coordinates(𝑋,𝑌) mapped via a custom C# script onto your Unity canvas UI elements, or if the SDK supports standard Unity event systems (like IPointerEnterHandler).

If you'd like to narrow this down, tell me:

• Are you prototyping using desktop/hardware displays with an external remote camera , or inside a VR/MR headset?
• Do you need raw gaze analytics (e.g., cognitive load, blink rate) or strictly UI target selection?