Overview: Real-time 3D avatar streaming and rendering for virtual collaboration refers to a system that enables users to create, customize, and control 3D avatars of themselves or other virtual characters in real-time, while also allowing these avatars to be streamed and rendered dynamically within virtual collaboration environments. The technology aims to enhance virtual collaboration by adding a strong visual and interactive element. Participants can communicate with each other using their avatars, and these avatars can replicate their real-world movements and expressions, making the interactions feel more natural and engaging.
Applications: Applications of this technology are numerous, ranging from remote work and business meetings to online education, virtual events, gaming, and socializing. By combining real-time 3D avatars, streaming, and rendering, this technology bridges the gap between physical presence and virtual interaction, providing a more immersive and realistic experience in digital spaces.
Scope of Work: The project encompasses various tasks, including the utilization of WebRTC APIs for real-time audio, video and 3D data communication. PeerJS or a similar library is used to streamline signaling and connection management. Google MediaPipe is employed for facial landmarks detection and facemesh rendering at both the sender and receiver ends. Additionally, lossless data compression techniques are used to enable higher frame rates for Avatars while streaming facial landmarks data. Furthermore, a 3D face avatar renderer based on ThreeJS is integrated into the system.