Week11-S1: Mixed Reality (MR)

The middle point between real and virtual environments defines Mixed Reality (MR). MR functions as a blend between virtual reality and augmented reality by allowing real and virtual elements to interact with each other in real time according to resmed.com. MR experiences unite physical and digital objects within a shared environment through which virtual objects maintain awareness of the physical space and respond accordingly. The technology surpasses basic AR capabilities because it enables virtual elements to interact with real-world objects. MR enables virtual characters to move through actual rooms while using real sofas for cover and disappearing like a real cat would. The virtual and real elements exist together in the experience while influencing each other.

An easy way to think of it: MR = AR + interactivity between real and virtual.

How It Works

Microsoft describes mixed reality (MR) as the combination of physical and digital worlds which merges virtual objects with actual environments. Users in mixed reality experiences can engage with physical objects at the same time they interact with virtual content (ibm.com). The virtual objects of MR differ from AR displays because users can physically manipulate them and they respond to actual environmental conditions. You can use a HoloLens to hold a floating virtual engine which will remain positioned on a real table while you navigate around it. Advanced MR systems create realistic lighting effects and shadowing which synchronize with the actual environment.

The Microsoft HoloLens and Magic Leap devices reach this functionality by using sensors to create spatial meshes of rooms while cameras and depth sensors track controllers and hands. Users can select holograms through a “pinch” gesture that HoloLens detects. The system enables virtual objects to behave naturally in the real world through real-time tracking and 3D visualization and physics simulation which allows objects to hide behind real items and fall realistically to the floor.

Example of Mixed Reality Devices

1. Microsoft HoloLens:

   1. The device functions mainly for business operations and professional applications.

   2. The system employs sophisticated sensors together with spatial mapping technology to display holograms on actual environments.

   3. The device includes built-in audio and voice and gesture controls which enable users to interact without using their hands.

   4. The platform provides strong development tools which drive its adoption in healthcare engineering and educational applications.

   5. The self-contained structure of this device operates independently from external computers yet its professional focus and high price restrict its availability to professional markets.

2. Magic Leap:

   1. The product focuses on creating lightweight designs combined with innovative optics to merge digital content into real-world experiences.

   2. The Lightfield display technology creates a natural depth perception by projecting digital images directly into your field of view.

   3. The product targets enterprise users and creative fields through its ability to deliver immersive interactive experiences.

   4. The platform offers developers a powerful development environment to create distinctive applications which span gaming entertainment and professional visualization and design.

   5. The technology keeps advancing through efforts to minimize hardware size and improve comfort for extended use periods.

The Future of Mixed Reality (MR) & Conclusion

The technology of MR continues to develop at a fast pace. The advancement of hardware technology combined with better environmental understanding by software will drive MR application growth. The technology shows potential to revolutionize education by allowing holographic tutors in classrooms and boost productivity through virtual monitors accessible through glasses and enable social interactions by displaying avatars alongside real people.

The successful combination of virtual and real elements remains a complex technical hurdle. The successful integration of virtual and real elements requires precise technical execution because any misalignment or latency will destroy the immersive experience. The current generation of AR glasses provides restricted viewing areas and their price reaches into thousands because the HoloLens 2 represents the top end of the market. The obstacles to MR adoption will gradually fade away in the future.