HMD-Eyes v1.4

We are pleased to announce the release of Hmd-Eyes v1.4!

Download the latest bundle and refer to the readme and documentation for how to get started.

Feel free to join our #vr-ar channel on discord and let us know what you think.

Motivation

This release introduces support for our upcoming Pupil 3.0 release.

Improvements

pye3d pupil data support - be2830e

Pupil 3.0 uses a new 3d pupil detector - pye3d - which uses a slightly different pupil format. Because pye3d continously updates a single model instead of fitting multiple parallel ones the new 3d pupil datum does no longer contain the following keys: model_id and model_birth_timestamp. Hmd-eyes will fallback to "0" and 0.0 respectively.

Support for binocular 3d gaze data with string-only keys - 63d3bf4

In order to be compatible with the msgpack-python v1.0.0 (specifially the new strict_map_key=True default), Pupil 3.0 uses only strings objects as dictionary keys for all msgpack-encoded data that is published via the Network API.

This change affects primarily binocular 3d gaze data which used to contain interger dictionary keys. Hmd-Eyes v1.4 is compatible with both types of dictionary keys.

Dependencies

• Pupil Capture/Service v2.0+
• Unity 2018.4+
• ProjectSettings/Player/Configuration/Scripting Runtime Verion set to .NET 4.x Equivalent.
• Due to an issue with MessagePack, the default project setting for ProjectSettings/Player/Configuration/API Compatibility Level is not supported and needs to be set to .NET 4.x

HMD-Eyes v1.3

We are pleased to announce the release of Hmd-Eyes v1.3!

Download the latest bundle and refer to the readme and documentation for how to get started.

Feel free to join our #vr-ar channel on discord and let us know what you think.

Motivation

This release contains improvements for a few non-critical issues that have piled up over the past.

Improvements

Avoid unnecessary restart of NetworkAPI - #98

Starting with Pupil v2.0, the frame publisher functionality was integrated into the NetworkAPI plugin which runs by default in Pupil Capture and Pupil Service. The published frame format can be controlled via notifications. The calibration script will use the new notification instead of restarting the plugin. This avoids restarting Pupil Remote functionality of the NetworkAPI unintendedly.

Avoid sending more reference data as necessary - #99

Due to how the calibration worked previously, HMD-Eyes was sending twice as many reference points as necessary for the newer version. Specifically, the reference data was sent separately for each eye, although the data was identical. This would not cause any issue, as Pupil would correctly interpret the duplicated data, but we have still corrected the behavior.

Add better log messages for unconnected components - #101

Most of the Unity components of HMD-Eyes require being hooked up to scene objects, e.g. the camera GameObject. If not hooked up correctly, they would log a non-descriptive warning in the Unity console. Since this also applies to our prefabs, some users were confused when the prefabs would not work out-of-the-box when being dragged into the scene. We have adjusted the error messages to make it easier to identify missing scene references for our components.

Dependencies

• Pupil Capture/Service v2.0+
• Unity 2018.4+
• ProjectSettings/Player/Configuration/Scripting Runtime Verion set to .NET 4.x Equivalent.
• Due to an issue with MessagePack, the default project setting for ProjectSettings/Player/Configuration/API Compatibility Level is not supported and needs to be set to .NET 4.x

Hmd-eyes v1.2

Hmd-eyes v1.2

We are pleased to announce the release of hmd-eyes v1.2!

Download the latest bundle and refer to the readme and documentation for how to get started.

Feel free to join our #vr-ar channel on discord and let us know what you think.

Features and Changes

This is a compatibility release for Pupil Core v2.0. All internals have been adjusted to adapt to the changes in Pupil's public API. No public behavior of hmd-eyes should have changed, except:

• A bug in the GazeRaysDemoScene has been fixed which prevented the scene from working correctly.
• The project to create the unitypackage releases has been bumped up from 2018.3 to the following LTS 2018.4 to be long term supported.

Dependencies

• Pupil Capture/Service v2.0+
• Unity 2018.4+
• ProjectSettings/Player/Configuration/Scripting Runtime Verion set to .NET 4.x Equivalent.
• Due to an issue with MessagePack, the default project setting for ProjectSettings/Player/Configuration/API Compatibility Level is not supported and needs to be set to .NET 4.x

Remote annotation support

We are pleased to announce the release of hmd-eyes v1.1!

Download the latest bundle and refer to the readme and documentation for how to get started.

Feel free to join our #vr-ar channel on discord and let us know what you think.

Dependencies

• Pupil Capture/Service v1.15+
• Unity 2018.3+
• ProjectSettings/Player/Configuration/Scripting Runtime Verion set to .NET 4.x Equivalent.
• Due to an issue with MessagePack, the default project setting for ProjectSettings/Player/Configuration/API Compatibility Level is not supported and needs to be set to .NET 4.x

Features and Changes

Remote Annotations

• Support for remote annotations added
• AnnotationPublisher offers high level API for sending annotations with optional custom data
• Thanks to @eioe for the contribution

Publisher abstraction

• Calibration class now uses the publisher protocol for sending calibration data instead of request/response
• C# Publisher class added as abstraction layer for PublisherSockets
• abstraction also used by new AnnotationPublisher and Screencast component

Others

• improved UX/UI via custom inspector for RequestController
• bug fix for using multiple RequestController

---

Current Limitations

• no hololens support

Hmd-eyes integration remake v1.0

We are pleased to announce the release of hmd-eyes v1.0!!

hmd-eyes v1.0 is a complete overhaul of the old hmd-eyes plugin, with a cleaner and more flexible architecture.

We are dropping support for the 2d approach and focus entirely on 3d gaze tracking. Starting with a new gaze demo scene and more robust gaze visualization.

Hmd-eyes now offers screen casting to Pupil Capture (v1.15+), which makes it fully compatible with the record and playback functionality of Pupil Capture and Pupil Player.

All new features and changes are listed below.

Download the latest bundle and refer to the readme and documentation for how to get started.

Feel free to join our #hmd-eyes channel on discord and let us know what you think.

Dependencies

• Pupil Capture/Service v1.15+
• Unity 2018.3+
• ProjectSettings/Player/Configuration/Scripting Runtime Verion set to .NET 4.x Equivalent.
• Due to an issue with MessagePack, the default project setting for ProjectSettings/Player/Configuration/API Compatibility Level is not supported and needs to be set to .NET 4.x

Features and Changes

Architecture overhaul

Overview

• replacing pure static classes by single responsibility MonoBehaviors
• event based communication and wiring components in the Unity Inspector
• plugin split into three layers (network, communication, flow control) building on one another
• simplified settings and flexible calibration targets utilizing ScriptableObjects
• instead of hidden objects due to on start initialization, everything can be pre-setup in the scene

Gaze representation optimized for VR

• Gaze representation
• Gaze visualization via raycasting

Screen casting

We dropped support for Unity screen recordings (which weren't compatible with Pupil Player). Instead Hmd-Eyes now offers streaming VR content directly to Pupil Capture. This way it is fully compatible with the record and playback functionality of Pupil Capture and Pupil Player.

• Screen cast utility

New time synchronization approach

From v1.0 on, Hmd-Eyes and Pupil Capture/Service are not running on the same clock anymore (previously set by Hmd-Eyes). Instead the TimeSync component now supports converting between pupil time and unity time.

• Time sync utility

Extendable calibration routine

• Calibration Controller
• Calibration Targets

Pre-parsed Pupil data

The new PupilListener offers parsed PupilData, no need to parse the nested "pupil" dictionary by hand anymore.

• Working with Pupil Data

Changelog

v1.0 since v1.0-beta

• Screen Casting functionality added
• PupilListener and PupilData introduced to simplify working with the "pupil" topic
• TimeSync: changed into its own component (MonoBehaviour) to improve accessibility and reduce dependencies
• Calibration Controller:
  • new event for 'calibration routine done' (user perspective)
  • preview for calibration targets added
• Gaze:
  • GazeController introduced to decouple controlling the GazeListener from GazeVisualizer
  • GazeVisualizer now displays the raw gaze direction additionally to the hit marker
  • removed unity time from GazeData, use TimeSync component instead
• Connection:
  • solve freeze on connect issue by using async receive and request abstraction
  • cancel running routines (like calibration) on disconnect
  • simplified inner network resource management
• recorder: fix recording path issue and allow setting a custom path

Current Limitations

• no heatmap or spherical video demo
• no hololens support

Beta release of the hmd-eyes integration remake v1.0

We are pleased to announce the beta release of hmd-eyes v1.0!!

hmd-eyes v1.0 is a going to be a complete overhaul of the current hmd-eyes plugin, with a cleaner and more flexible architecture. We are dropping support for the 2d approach and focus entirely on 3d gaze tracking. Starting with a new gaze demo scene and more robust gaze visualization.

Please refer to the release/beta branch for checking the readme and documentation.

We also created a dedicated channel #hmd-eyes-alpha on our discord server to gather your feedback and post about updates.

Dependencies

• Unity 2018.3+
• ProjectSettings/Player/Configuration/Scripting Runtime Verion set to .NET 4.x Equivalent.
• Due to an issue with MessagePack, the default project setting for ProjectSettings/Player/Configuration/API Compatibility Level is not supported and needs to be set to .NET 4.x

Changelog

v1.0-beta

• time sync: timestamps are now based on the Pupil Capture/Service clock and synchronized on connection
  • RequestController offers conversion methods between Unity and Pupil timestamps
  • GazeListener/GazeData also support both timestamps (as will other Listeners in the future)
  • switching all timestamps from float to double
• sub and pub port now accessibly via the RequestController
• based events on System.Action instead of custom delegates
• new frame visualizer prefab using a dedicated material to avoid missing shaders
• documentation updated including references to pupil-docs

v1.0-alpha-2

• improved and refactored GazeData
  • renamed GazeData.Mode to GazeData.MappingContext
  • added GazeData.GazeDirection and GazeData.GazeDistance
  • deprecated the data field GazeData.GazePoint3d in favor of the equivalent representation of the 3d gaze point as GazeData.GazeDirection and GazeData.GazeDistance (more details)
  • resulting in a simplified GazeVisualizer
  • added detailed inline comments for GazeData
• added RecordingController to support controlling Pupil Capture recordings
• ability to change IP&Port by code
• new demos showcasing:
  • scene management
  • controlling recordings
  • visualizing individual gaze vectors as rays
• important bugfix for time synchronization with Pupil Software
• fixed application freeze on quit while trying to connect
• updated documentation and clearer dependencies

v1.0-alpha-1

• complete overhaul of the hmd-eyes plugin
  • clean and flexible architecture with three layers (connection, communication, utility components)
  • simplified settings and flexible calibration targets
  • instead of hidden objects due to on start initialization, everything can be pre-setup in the scene
• focused on 3d gaze tracking (no further support for the 2d approach)
  • much more stable gaze visualization
• updated dependencies
• supporting latest Unity 2018.3 and .NET 4.x

Current Limitations

• no support for screen capture recording
• no heatmap or spherical video demo
• no hololens support

Alpha-2 release of the hmd-eyes integration remake v1.0

We are pleased to announce the alpha-2 release of hmd-eyes v1.0-alpha!!

hmd-eyes v1.0 is a going to be a complete overhaul of the current hmd-eyes plugin, with a much cleaner and way more flexible architecture. We are dropping support for the 2d approach and focus entirely on 3d gaze tracking. Starting with a new gaze demo scene and much more stable gaze visualization.

Please refer to the release/alpha branch for checking the readme and documentation.

We also created a dedicated channel #hmd-eyes-alpha on our discord server to gather your feedback and post about updates.

Dependencies

• Unity 2018.3 latest
• ProjectSettings/Player/Configuration/Scripting Runtime Verion set to .NET 4.x Equivalent.
• Due to an issue with MessagePack, the default project setting for ProjectSettings/Player/Configuration/API Compatibility Level is not supported and needs to be set to .NET 4.x

Changelog

v1.0-alpha-2

• improved and refactored GazeData
  • renamed GazeData.Mode to GazeData.MappingContext
  • added GazeData.GazeDirection and GazeData.GazeDistance
  • deprecated the data field GazeData.GazePoint3d in favor of the equivalent representation of the 3d gaze point as GazeData.GazeDirection and GazeData.GazeDistance (more details)
  • resulting in a simplified GazeVisualizer
  • added detailed inline comments for GazeData
• added RecordingController to support controlling Pupil Capture recordings
• ability to change IP&Port by code
• new demos showcasing:
  • scene management
  • controlling recordings
  • visualizing individual gaze vectors as rays
• important bugfix for time synchronization with Pupil Software
• fixed application freeze on quit while trying to connect
• updated documentation and clearer dependencies

v1.0-alpha-1

• complete overhaul of the hmd-eyes plugin
  • clean and flexible architecture with three layers (connection, communication, utility components)
  • simplified settings and flexible calibration targets
  • instead of hidden objects due to on start initialization, everything can be pre-setup in the scene
• focused on 3d gaze tracking (no further support for the 2d approach)
  • much more stable gaze visualization
• updated dependencies
• supporting latest Unity 2018.3 and .NET 4.x

Current Limitations

• no support for screen capture recording
• no heatmap or spherical video demo
• no hololens support

Incremental Bugfix Release - Time Sync fix

This release contains an important fix for synchronizing timestamps between hmd-eyes and Pupil Capture/Service (v1.11) and stable recordings (while running multiple calibrations).

• using realtime timestamps instead of simulation time (Time.realtimeSinceStart instead of Time.time)
• synchronizing timestamps on connect instead of on calibration start
• fixed toggling gaze visualization in the 2d market demo

Hmd-eyes v0.62 only supports Unity 2017.4 LTS (for Unity 2018.3 support - please checkout our current alpha release).

Please refer to the legacy branch for checking the readme and documentation.

Alpha release of the hmd-eyes integration remake v1.0

We are pleased to announce the alpha release of hmd-eyes v1.0-alpha!!

hmd-eyes v1.0 is a going to be a complete overhaul of the current hmd-eyes plugin, with a much cleaner and way more flexible architecture. We are dropping support for the 2d approach and focus entirely on 3d gaze tracking. Starting with a new gaze demo scene and much more stable gaze visualization.

Please refer to the release/alpha branch for checking the readme and documentation.

We also created a dedicated channel (#hmd-eyes-alpha) on our discord server to gather your feedback and post about updates.

Dependencies

• Unity 2018.3 latest
• ProjectSettings/Player/Configuration/Scripting Runtime Verion set to .NET 4.x Equivalent.
• Due to an issue with MessagePack, the default project setting for ProjectSettings/Player/Configuration/API Compatibility Level is not supported and needs to be set to .NET 4.x

Features

• complete overhaul of the hmd-eyes plugin
  • clean and flexible architecture with three layers (connection, communication, utility components)
  • simplified settings and flexible calibration targets
  • instead of hidden objects due to on start initialization, everything can be pre-setup in the scene
• focused on 3d gaze tracking (no further support for the 2d approach)
  • much more stable gaze visualization
• updated dependencies
• supporting latest Unity 2018.3 and .NET 4.x

Current Limitations

• recordings currently not supported
• no heatmap or spherical video demo
• scene management removed

Incremental Bugfix Release for Pupil Capture version 1.9 and up

This release contains updates to be compatible with Pupil Capture version 1.9 and up.

• new gaze messages gaze.3d.0., gaze.3d.1. and gaze.3d.01. are now handled properly, which fixes the 3D gaze demo with Pupil Capture versions >= 1.9
• fix for initial eyes transition, which should improve the 3d calibration