Research Team: Mohit Bajaj
Institutional Affiliation: Anand International College of Engineering, Jaipur
Research Context: Computer Vision and Real-time Systems
Domain: Virtual Collaboration and 3D Graphics
Real-time 3D avatar streaming and rendering for virtual collaboration refers to a system that enables users to create, customize, and control 3D avatars of themselves or other virtual characters in real-time, while also allowing these avatars to be streamed and rendered dynamically within virtual collaboration environments.
The technology aims to enhance virtual collaboration by adding a strong visual and interactive element. Participants can communicate with each other using their avatars, and these avatars can replicate their real-world movements and expressions, making the interactions feel more natural and engaging.
Applications of this technology are numerous, ranging from remote work and business meetings to online education, virtual events, gaming, and socializing. By combining real-time 3D avatars, streaming, and rendering, this technology bridges the gap between physical presence and virtual interaction, providing a more immersive and realistic experience in digital spaces.
The project encompasses various tasks, including the utilization of WebRTC APIs for real-time audio, video and 3D data communication. PeerJS or a similar library is used to streamline signaling and connection management. Google MediaPipe is employed for facial landmarks detection and facemesh rendering at both the sender and receiver ends.
Additionally, lossless data compression techniques are used to enable higher frame rates for Avatars while streaming facial landmarks data. Furthermore, a 3D face avatar renderer based on ThreeJS is integrated into the system.
The system implements a peer-to-peer architecture that enables direct communication between participants while maintaining high-quality avatar representation through advanced facial tracking and 3D rendering technologies.
Achieving real-time performance requires careful optimization of facial tracking algorithms and efficient data transmission protocols to minimize latency while maintaining visual quality.
The system is designed to work across different devices and platforms, ensuring broad accessibility for virtual collaboration scenarios.
Advanced compression techniques are employed to reduce bandwidth requirements while preserving the quality of avatar representation and interaction.
Project Resources: