Hera is a smart, modular system that helps immersive VR/AR apps work flawlessly over 5G. It keeps latency low, video quality high, and multiple users in sync—even when the network gets rough.
Imagine stepping into a virtual world where you and your friends can collaborate, train, or attend concerts—without a single lag or blur 🎧📱. That's the dream of Multi-user Immersive Reality (MIR), and thanks to 5G, it's becoming more real than ever. But… not without some serious technical challenges 🤔.
In today’s post, we dive into a fresh innovation from researchers at NYU and NYU Abu Dhabi. They introduce Hera, a new system that supercharges immersive apps (like Virtual Reality / Augmented Reality experiences) over 5G. We’ll break down what it does, why it matters, and how it opens the door to next-gen digital experiences 🚪✨.
Virtual Reality and Augmented Reality apps are exploding in popularity—think Meta’s Horizon Worlds, Apple’s Vision Pro, or VRChat 🎭. These immersive platforms are going multi-user and high-fidelity, with:
And all that? It takes serious bandwidth (over 100 Mbps per user) and ultra-low latency (under 20 milliseconds) 😮💨. While 5G sounds like the answer, in practice, even 5G struggles with signal drops, latency spikes, and bandwidth fluctuations 📉📈.
That’s where the research comes in.
To fix these problems, the researchers built Hera—a two-layer framework specially made for streaming and synchronizing immersive experiences across unstable 5G networks 📡.
Together, they adapt not just to the network, but with the network—ensuring the best possible Quality of Experience (QoE) for every user. And that’s key.
Quality of Experience (QoE) is how enjoyable and smooth an immersive session feels to the user 🛋️. Hera optimizes for:
⏱️ Startup Delay – How fast the experience loads
📺 Resolution – How clear the visuals look
🌀 Buffering – Whether the scene pauses or freezes
🧤 Interaction Latency – How quickly the system reacts to user movements
👥 Collaborative Fluency – How synchronized users feel while working together
🎯 Hera’s mission: Maximize QoE without overloading the 5G network. And it does it smarter than older protocols like TCP Cubic or even Google’s BBR.
Hera’s magic lies in how it handles congestion control—deciding how fast to send data without overwhelming the network 🚦.
Unlike other systems that react only after packet loss, Hera anticipates problems by reading RTT patterns in advance 👁️. Think of it like defensive driving for the internet 🚗💨.
To see Hera in action, the researchers built a testbed with:
🖥️ A Linux server hosting DASH video content
🥽 Real Meta Quest headsets and synthetic Virtual Reality clients
📶 Network emulators simulating real-world 5G conditions
Then they compared Hera with 4 other popular congestion control protocols:
Each protocol streamed 4K VR videos to multiple users across 6 network types (city driving, beach, rural, walking indoors, etc.).
Hera crushed the competition on several fronts:
| Metric | Hera vs. Others |
|---|---|
| 🎯 Latency | Up to 66% lower than Cubic and 50% lower than BBR |
| 🔊 Throughput | Matches or beats BBR while staying more stable |
| 🖼️ Video Quality | Maintains 4K streaming while others drop to 720p |
| 👬 Fairness | Distributes bandwidth more equally between users |
| ⚡ Responsiveness | Least startup delay and smoothest interactions |
In plain terms: Hera offers high-res video 📽️, low lag 🧊, and consistent performance 🎵—even when the 5G signal gets moody.
Researchers simulated:
🚗 Urban driving – Think self-driving Virtual Reality experiences
🏖️ Beach stationary – For chill Virtual Reality yoga apps
🌾 Rural areas – For farming training modules
🏃♂️ Indoor walking – For gaming or factory inspections
In all of these, Hera adapted faster and more smoothly than other protocols, ensuring that users could keep engaging in shared Virtual Reality sessions without hiccups or lag.
Even with 6 users competing for bandwidth, Hera kept video smooth and fairly allocated across all users—something Allegro and Vivace struggled with, sometimes letting one user dominate the stream 🤷.
The future looks bright for Hera—and for immersive technologies powered by 5G 🌟. Here’s what’s on the horizon:
Hera can potentially support larger crowds—imagine entire virtual conferences or classrooms with dozens of users 👨🏫👩🎓.
By connecting Hera with edge servers closer to users, latencies could drop even further. Imagine AI avatars responding instantly 🤖.
Hera’s open-source design makes it attractive for Virtual Reality / Augmented Reality app developers looking to boost their performance without reinventing the wheel.
Hera can evolve to support upcoming standards like QUIC or 6G to stay ahead of the curve.
As Virtual Reality and Augmented Reality become part of our daily lives—from gaming to remote work to telehealth—the networks powering them must evolve too 🌍. Hera proves that immersive tech doesn’t have to settle for laggy, blurry, or unstable experiences—even on unpredictable 5G connections 🚫📶.
By combining deep network intelligence with real-time application awareness, Hera brings us closer to the dream of fully immersive, collaborative virtual worlds. And this is just the beginning.
📶 5G - The fifth generation of mobile networks—much faster than 4G, with lower delay (latency). It’s what allows you to stream, game, and connect super smoothly on your phone or VR headset. - More about this concept in the article "Healing with Qubits ⚛️".
🥽 Virtual Reality (VR) - A computer-generated 3D environment you can explore and interact with using special headsets. Think of it as stepping inside a video game or simulation. - More about this concept in the article "Revolutionizing Construction in Australia: How Emerging Technologies Are Shaping the Future 🦘 🏗️".
🧑🤝🧑 Multi-user Immersive Reality (MIR) - When several people join the same virtual or augmented space to interact in real-time—like meeting friends in VR for work or fun.
🎮 Augmented Reality (AR) - Unlike VR, AR adds digital stuff on top of your real-world view. Pokémon GO is a popular example. - More about this concept in the article "A New Era of Virtual Interaction: How Augmented Reality and Haptics are Changing the Game 🎮🔬".
⚡ Latency - Latency refers to the duration required for data to traverse from a source device to a destination device. In VR, even tiny delays (over 20 milliseconds) can feel like lag or cause motion sickness. - More about this concept in the article "All Aboard the Future! 🚄 How 6G Will Supercharge Smart Railways with Speed, Safety & Smarts".
📺 Bitrate - The amount of video data sent per second, measured in Mbps. Higher bitrate = better picture quality, but it needs more bandwidth.
🌐 Bandwidth - The maximum amount of data your network can handle at one time—like the size of a digital highway.
🎯 Quality of Experience (QoE) - How good something feels to use—not just speed or resolution, but how smooth, responsive, and enjoyable it is.
⏱️ Congestion Control - A technique used in networking to avoid sending too much data too quickly, which could jam the connection. It’s like driving carefully on a busy road.
🔁 Round-Trip Time (RTT) - Measures the total duration for a data packet to travel from a user's device to a server and then return to the originating device. Lower RTT = faster responses in games and VR.
📊 Throughput - How much data actually gets delivered over a network in a given time. Even with high bandwidth, bad congestion can reduce your throughput.
🧪 Adaptive Bitrate Streaming - A method where your device automatically adjusts video quality depending on your internet speed. Helps avoid buffering!
Source: Rohail Asim, Ankit Bhardwaj, Lakshmi Suramanian, Yasir Zaki. Towards Next Generation Immersive Applications in 5G Environments. https://doi.org/10.48550/arXiv.2507.20050
