Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not working in WEBXR #16

Open
Gianlucccc opened this issue Nov 20, 2023 · 4 comments
Open

Not working in WEBXR #16

Gianlucccc opened this issue Nov 20, 2023 · 4 comments

Comments

@Gianlucccc
Copy link

im trying to make the screensharing work on webxr, but while on normal webgl client works like a charm in webxr it doesnt work at all.
im only able to see my screenshare if i launch the screenshare locally ( using oculus link ), the webxr client now renders the video but the information is not passed to the other clients.

@davidzhao
Copy link
Member

@GianlucaC-ED which client SDK are you using?

@Gianlucccc
Copy link
Author

Gianlucccc commented Nov 21, 2023 via email

@Gianluca-C90
Copy link

@davidzhao did you look into it ? am i missing something ? its like the screenshare wont render on the webxr cameras. i could really use your help to understand if its something that livekit cant work with or if its my fault in my configuration

@davidzhao davidzhao transferred this issue from livekit/livekit Dec 21, 2023
CalinR pushed a commit to CalinR/client-sdk-unity that referenced this issue Jan 25, 2024
* remove redundant Task usages on separated threads

* FfiRequestsPool

* IFFIClient interface

* disposable FFIClient

* FFIClient initialize simplify

* PoolableFFIClient

* NativeMethods pass pointer

* response pooling

* tests for reject manual creation of protocol objects

* FfiResponseWrap, remove manual new Request calls

* remove IPoolableFFIClient

* FFIBridge introduction, remove closures, RAII with FfiResponseWrap and FfiRequestWrap

* Clear -> generic ClearMessage (+16 squashed commits)
Squashed commits:
[768fb01] Multithread refactor (livekit#13)

* disconnect support

* Fix name

* To handle better connection and disconnection from room

* change to avoid using of Coroutines, using async methods instead

* async OnAudioRead

* Cleaning logs

* Cancel tokens

* Cancel token

* remove Task.Run, since usually SendRequest is called from other task

* getting rid of use YieldInstruction

* adding audio filter in main thread

* passing CancellationToken instead of CancellationTokenSource

* added check for cancellation at beginning

* Cleaning handles

* fix await

* remove cruft

* Cancellation token cleanup

* Factory async method

* basic handling of remaining events

* more async

* wip

* Compile pass

* formatting

* fix pending check

* revert async changes

* prep audio filter for threads

* loggin

* logs to debug publishing track

* minor cleanup

* added some methods to handle localtrack and SetSubscribed, Clenaing code

* Cleaning up

* fixes for yield instructions

* macro for verbose logs

* Audio data and pointer fix

Still says samplerate and num of channels are incorrect

* remove possibility of null

* todo ref buffering

* minor cleanup

* cruft

* todo object pooling

* remove redundant Task usages on separated threads

---------

Co-authored-by: cdga777 <[email protected]>
Co-authored-by: nickkhalow <[email protected]>
[a400ec9] enable nullables
[a5a63c4] minor cleanup
[892407a] to prevent null reference
[511ef58] Hack for now

Room Ids are different
[6456edc] Test sending data
[eca1cb5] new initialize call
[830ebd1] modification trying to fix compiler errors
[496618c] Added metas.
[2cf59c3] Updated FFIClient with the correct FfiRequest class name based on updated proto file generated classes.
[c8e3906] updated protobuf messages to latest to be able to publish data.
[bb762d2] Updated dll
[80d3dcd] Fixed the last error regarding the leak fix
[7320b31] Fixed a compile error on the distructor
[561282f] Fixed a memory leak
[db871de] Fixed building issue.
@Gianlucccc
Copy link
Author

hello there , do you have any info about this ticket ? im still banging my head to get this working on oculus, the fun fact is that livekit screenshare works until the oculus user dont get inside the webxr instance, what am i missing ?

cloudwebrtc pushed a commit that referenced this issue Mar 30, 2024
* remove redundant Task usages on separated threads

* FfiRequestsPool

* IFFIClient interface

* disposable FFIClient

* FFIClient initialize simplify

* PoolableFFIClient

* NativeMethods pass pointer

* response pooling

* tests for reject manual creation of protocol objects

* FfiResponseWrap, remove manual new Request calls

* remove IPoolableFFIClient

* FFIBridge introduction, remove closures, RAII with FfiResponseWrap and FfiRequestWrap

* Clear -> generic ClearMessage
cloudwebrtc added a commit that referenced this issue Apr 8, 2024
* Fixed a memory leak

* Fixed the last error regarding the leak fix

* fix.

* ring buffer as struct

* to prevent null reference

* Multithread refactor (#13)

* disconnect support

* Fix name

* To handle better connection and disconnection from room

* change to avoid using of Coroutines, using async methods instead

* async OnAudioRead

* Cleaning logs

* Cancel tokens

* Cancel token

* remove Task.Run, since usually SendRequest is called from other task

* getting rid of use YieldInstruction

* adding audio filter in main thread

* passing CancellationToken instead of CancellationTokenSource

* added check for cancellation at beginning

* Cleaning handles

* fix await

* remove cruft

* Cancellation token cleanup

* Factory async method

* basic handling of remaining events

* more async

* wip

* Compile pass

* formatting

* fix pending check

* revert async changes

* prep audio filter for threads

* loggin

* logs to debug publishing track

* minor cleanup

* added some methods to handle localtrack and SetSubscribed, Clenaing code

* Cleaning up

* fixes for yield instructions

* macro for verbose logs

* Audio data and pointer fix

Still says samplerate and num of channels are incorrect

* remove possibility of null

* todo ref buffering

* minor cleanup

* cruft

* todo object pooling

* remove redundant Task usages on separated threads

---------

Co-authored-by: cdga777 <[email protected]>
Co-authored-by: nickkhalow <[email protected]>

* feat: optimisations (#16)

* remove redundant Task usages on separated threads

* FfiRequestsPool

* IFFIClient interface

* disposable FFIClient

* FFIClient initialize simplify

* PoolableFFIClient

* NativeMethods pass pointer

* response pooling

* tests for reject manual creation of protocol objects

* FfiResponseWrap, remove manual new Request calls

* remove IPoolableFFIClient

* FFIBridge introduction, remove closures, RAII with FfiResponseWrap and FfiRequestWrap

* Clear -> generic ClearMessage

* Make the Ffi Handle type consistent.

* room support spans

* cleanup.

* feat: optimisation - memory pooling, FfiHandles (#18)

* fix clear message

* optimise FfiNewRequest - no FfiHandle

* LiveKitInitialize with IntPtr

* const format

* memory pool

* inner message pooling

* ArrayMemoryPool with thread safety

* remove obsolete

* thread safety

* RingBuffer recycling

* dangerous access for memory buffer (#26)

* feat: Audio/Video Support (#29)

* ensure channels and rate match

* subscription change

* fixing early subscription

* track handle fix

temp logs to confirm

* track handle fix

* streaming fixes

* remix fix

* resample log check

* try off frame

* handler fix

* log cleanup

* saving track for local Participant

* remix potential fix

* cleanup checks

* testing without thread

* revert

* destroy cleanup and possible fix to ringbuffer

* log cleanup

* test

* revert

* disable thread for test

* test

* fix

* adding threading back in for writing

* video prep

* handle fix

* fix cast

* removing streams from video for now

* buffer fix

* hack to get it to send a web texture

* more consistent naming

* format fix

* bad merge fix

* first pass on screen share and memory fix

* debug and formatting

* remove logs

* updated protobuff and fixed compiler errors

* video working

* dynamic buffertype and stride

* split video source and partial wip on cameravideoSource

* camera support

* merge cont.

* wip merging

* adding back stomped change

* latest libs for mac

* Fixes for Audio/Video Streaming

* remove cruft

* fix when an AudioTrack is created

* mac fix

* null check which can occur if you are closing app

* format checks

* utils

* fix for metal

* Subscription management

* cleanup

* log cleanup

* cleanup

* log

---------

Co-authored-by: cdga777 <[email protected]>

* update local metadata.

* fix message truncation bug.

* Update livekit_ffi.dll.meta

* Update libwebrtc.jar.meta

* update.

* some minor improvements.

* update.

---------

Co-authored-by: Ashkan Saeidi Mazdeh <[email protected]>
Co-authored-by: nickkhalow <[email protected]>
Co-authored-by: cdga777 <[email protected]>
Co-authored-by: Mark Grossnickle <[email protected]>
Co-authored-by: Nick Khalow <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants