[Unity] Android media projection causing lag in right eye

Question

Hi,
In our Unity app we have a native android plugin that uses Android media projection to stream the screen capture via OpenTok.
The streaming runs on a separate thread in Android/Java to minimize impact on the Unity application.

Sometimes (about 40% of our tests), the right eye of the headset starts stuttering/lagging when we stream our screen. The left eye is smooth as normal but the right lags for some reason. Other times (about 60% of our tests), we have no problem in the right eye at all. It seems random to us. We think this is weird because even with the lag/stutter in the right eye, the Unity profiler still says we have over 60fps (see img).

I profiled this with the snapdragon profiler too and cpu usage when streaming is always 25% to 36%, with or without lag in the right eye. This leaves me to believe that it is not a performance issue.

I know this is a tough question but could you have any idea of what is making the right eye lag?

Answer 1

We have further investigated the bug and found that the cause of the lag was not actually the streaming, but our usage of the eclipse.january.dataset class to remove image stride from the screen capture. We switched it out with less resource intensive loops and the lag was gone.

Answer 2

Dear developer,

Could you please provide the SDK version,the model type,PUI version and Unity version？

In addition,would you mind sending your demo project with the title of your question to this mail address : pico_devanswers@bytedance.com.We'll test it.

Comments

Hi,

Apologies for the late answer, it took some time to make the sample project.
I have mailed the sample project to the address and sent this info as well:

XR SDK version: 2.0.1
Model type: Pico Neo 3 Pro
PUI Version: v4.3.34
Unity Version: 2019.4.8

Thank you in advance