Visualize Point of Gaze using Pico XR SDK for eye tracking

Question

Hi there,

I am trying to visualize the point of gaze using the Pico XR SDK (v2.0.4). I am using the code below but it doesn't do anything.

void Update()
{
    PXR_EyeTracking.GetCombineEyeGazeVector(out Vector3 eyeGazeDirection)
    PXR_EyeTracking.GetCombineEyeGazePoint(out Vector3 eyeGazePosition);

    Ray ray = new Ray(eyeGazePosition, eyeGazeDirection);
    Debug.DrawLine(eyeGazePosition, eyeGazeDirection, Color.yellow);
}

What I would like to achieve is have a crosshair or a small circle which corresponds to where the participant is looking in the scene. Is it possible?

Answer 1

hello，
You can try this demo, which shows how to use eye tracking.

https://github.com/picoxr/Eye-Tracking-UnityXR
There is a scene in this demo that is somewhat similar to your requirements, with a small dot follows where the eyes are looking.

Comments

There are 4 demos. Which one is doing that because I can't see any demo with a small dot. Thanks.

---

try this one Eye Tracking, that little green dot in there moves with the eyes.
Other scenes can also be considered, such as several scenes in the Game Demo.
hope thes demo can help.

---

Yep.I found the example. The problem is that I am trying to do it in 3D because the example used a 2d sprite and detects the hit point in this sprite to display the green dot. I want to do the same in 3d. I have done this so far but I am missing something and I can't display the green dot (which I made a green sphere) in 3d correctly:

IEnumerator EyeRaycast(float steptime)
{
    while (true)
    {
        if (Camera.main)
        {
            matrix = Matrix4x4.TRS(Camera.main.transform.position, Camera.main.transform.rotation, Vector3.one);
        }
        else
        {
            matrix = Matrix4x4.TRS(Vector3.zero, Quaternion.identity, Vector3.one);
        }
        bool result = (PXR_EyeTracking.GetCombineEyeGazePoint(out Vector3 Origin) && PXR_EyeTracking.GetCombineEyeGazeVector(out Vector3 Direction));
        PXR_EyeTracking.GetCombineEyeGazePoint(out Origin);
        PXR_EyeTracking.GetCombineEyeGazeVector(out Direction);
        var RealOriginOffset = matrix.MultiplyPoint(Origin);
        var DirectionOffset = matrix.MultiplyVector(Direction);
        if (result)
        {
            Ray ray = new Ray(RealOriginOffset, DirectionOffset);
            //Debug.DrawRay(ray.origin, ray.direction * 10, Color.red);
            //RaycastHit hit;
            if (Physics.Raycast(ray, 200, worldLayer))
            {
                greenpoint.gameObject.SetActive(true);
                greenpoint.DOMove(DirectionOffset, steptime).SetEase(Ease.Linear);
                /*if (hit.collider.CompareTag("target"))
                {
                    Debug.Log("target" + hit.point);
                    greenpoint.gameObject.SetActive(true);
                    greenpoint.DOMove(hit.point, steptime).SetEase(Ease.Linear);
                }*/
            }
            else
            {
                greenpoint.gameObject.SetActive(false);
            }
        }
        yield return new WaitForSeconds(steptime);
    }
}

I think the problem is in the line below (DirectionOffset instead of hit.point) but I don't know how to fix it:

greenpoint.DOMove(DirectionOffset, steptime).SetEase(Ease.Linear);

I've set the layer mask correctly so that it detects only objects in the world layer. Any help will be very much appreciated.

---

Hi, you can try the shooting zombie scene in the game demo.
The scene will generate bullets in the direction of the eye's gaze. You can try to generate 3D objects using this logic.