Optimizing Cloud Virtual Reality Networks with Transfer Learning for Frame-size Prediction and Lossy Latent Transmission

Abstract

Despite the growing popularity of Virtual Reality (VR), its adoption remains limited due to bulky hardware and low mobility. Cloud-based VR (cloud VR) offers a promising solution but faces two major challenges: efficient network resource management and high-resolution content compression. Overcoming these challenges is crucial for cloud VR to prevent subpar Quality of Experience (QoE). Predicting network application traffic characteristics in advance offers a potential solution for the first challenge as it enables proactive resource allocation. To this end, we investigated the use of machine learning (ML) models to predict network traffic frame size data, collected from a real-world cloud VR gaming testbed. Furthermore, this thesis explored effectiveness of transfer learning (TL) in predicting frame size traffic patterns across different games and network conditions under online learning settings. The proposed TL approach reduces overall traffic prediction error by up to 54%. For the second challenge, effective compression techniques are crucial for high-resolution VR transmission. This thesis proposed a novel compression framework using Deep Neural Networks (VAE-GAN) for streaming 8K stereoscopic videos which demands significant bandwidth. By mapping latents as 3-channel RGB scenes compatible with standard encoders, the proposed method reduces bandwidth requirements by up to 45.1% across various 8K stereoscopic scenes while maintaining visual quality. Additionally, the impact of varying input frame patch sizes on client-side reconstructions and different transmission configurations for latent frames is evaluated, offering insights into optimizing high-resolution VR streaming systems. Overall, this work tackles network resource management and compression challenges in cloud VR systems, providing valuable insights for next-generation immersive VR experiences.