0 votes
by (370 points)

Hi all,

I am using the Pico Neo 3 Eye to capture some eye-tracking data. I've made a scene that displays on a Canvas the Gaze Origin Vector 3 using:

PXR_EyeTracking.GetCombineEyeGazePoint()

and the Gaze Direction Vector 3 using:

PXR_EyeTracking.GetCombineEyeGazeVector()

There are 2 problems:
1.) The Gaze Origin Vector 3 is always 0 (0,0,0) despite the fact that I move my head, so the origin should change. What I'm missing here?

2.) The Gaze Direction Vector 3 displays values ranging from -1 to 1. I read somewhere that 0,0 corresponds to the top right and 1,1 to the bottom left. How can I convert those values to world coordinates?

Thanks in advance.

1 Answer

0 votes
by (6.6k points)
selected by
 
Best answer

The EyeTracking data is related to the location of the EyeTracking sensors, Matrix transformation is required to map correctly in the scene.
If position data is desired to be transformed in front of camera, a matrix transform is required.
It is recommended to refer to the matrix transformation implemented in demo.
https://github.com/picoxr/Eye-Tracking-UnityXR

For example, you can refer to the EyeRay.cs script in demo:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using DG.Tweening;
using Unity.XR.PXR;

public class EyeRay : MonoBehaviour
{
    public Transform greenpoint;

    private Matrix4x4 matrix;

    // Use this for initialization
    void Start()
    {
        StartCoroutine(EyeRaycast(0.04f));
    }
    IEnumerator EyeRaycast(float steptime)
    {
        while (true)
        {
            if (Camera.main)
            {
                matrix = Matrix4x4.TRS(Camera.main.transform.position, Camera.main.transform.rotation, Vector3.one);
            }
            else
            {
                matrix = Matrix4x4.TRS(Vector3.zero, Quaternion.identity, Vector3.one);
            }
            bool result = (PXR_EyeTracking.GetCombineEyeGazePoint(out Vector3 Origin) && PXR_EyeTracking.GetCombineEyeGazeVector(out Vector3 Direction));
            PXR_EyeTracking.GetCombineEyeGazePoint(out Origin);
            PXR_EyeTracking.GetCombineEyeGazeVector(out Direction);
            var RealOriginOffset = matrix.MultiplyPoint(Origin);
            var DirectionOffset = matrix.MultiplyVector(Direction);
            if (result)
            {
                Ray ray = new Ray(RealOriginOffset, DirectionOffset);
                RaycastHit hit;
                if (Physics.Raycast(ray, out hit, 200))
                {
                    if (hit.collider.transform.name.Equals("zbx"))
                    {
                        Debug.Log("zbx---" + hit.point);
                        greenpoint.gameObject.SetActive(true);
                        greenpoint.DOMove(hit.point, steptime).SetEase(Ease.Linear);
                    }
                }
                else
                {
                    greenpoint.gameObject.SetActive(false);
                }
            }
            yield return new WaitForSeconds(steptime);
        }
    }
}
by (370 points)

With a bit of fine-tuning, it worked. Thanks!

...