How One Man’s Limited Sight Gives Him Limitless Vision in Virtual Reality Design

by Rina Diane Caballar
- Sep 11 2018 - 5 min read
The virtual reality experience of Simon Che de Boer's ‘Nefertari: Journey to Eternity.’
Simon Che de Boer possesses just 5 percent of his eyesight, which gives him a unique perspective when designing virtual reality experiences such as ‘Nefertari: Journey to Eternity.’ Courtesy RealityVirtual.

In the Valley of the Queens in Egypt lies the tomb of Nefertari, an ancient Egyptian queen and one of the favored wives of Pharaoh Ramesses II. The tomb’s walls are covered in intricate paintings, hieroglyphic inscriptions, and portraits of the queen wearing a white gown and gold headdress. Discovered in 1904, the historical site has undergone extensive restoration but remains fragile.

To digitally preserve Nefertari’s tomb, Simon Che de Boer and his New Zealand–based virtual reality (VR) company RealityVirtual developed Nefertari: Journey to Eternity, a hyperrealistic virtual tour experience that takes viewers into the Egyptian tomb. “I think it’s a really good use case of preserving the past—resurrecting sites, environments, and scenes that humanity has lost or could lose due to corrosion or the elements,” Che de Boer says.

To replicate the tomb’s elaborate details, Che de Boer captured nearly 4,000 photographs of the site, then combined photogrammetry (the science of making measurements from photographs) with deep learning methods for processing and visualization. “Essentially, we’re able to feed the computer all this data that we captured from the location, and the computer looks at all the photographs and teaches itself to understand what it’s looking at,” he says.

A closeup image of an Egyptian tomb with fine details rendered through photogrammetry.
Che De Boer used photogrammetry to capture details from some 4,000 photographs to re-create the ancient tomb in extreme detail. Courtesy RealityVirtual.

Through deep learning, Che de Boer and his team could increase image resolution without sacrificing quality. More important, deep learning allowed Che de Boer to remove “errors” from images. For instance, exit signs were painted out and replaced with hieroglyphics from the spaces around them. “When we find an area that’s missing because we didn’t get enough photos or something was blocking it, the computer has a fundamental enough understanding to paint it out and fill it with elements from the surrounding environment,” he says.

Che de Boer is no stranger to pushing boundaries: He creates VR experiences despite having approximately 5 percent vision. “With Nefertari’s tomb, I could never really appreciate how awesome it was,” he says. “I could never get close enough to see great detail. Virtual reality allowed me to get more intimate with the experience than I was able to in the real world.”

Because of his low vision, Che de Boer has strong spatial and situational awareness, a fitting skill for VR development. “When I walk anywhere, I have to memorize exactly where I’ve been,” he says. “For my entire life, I’ve been mapping out environments just to get around. That’s a good skill to have in photogrammetry because you need to know where you’ve been and where you’re going.”

A portrait of Simon Che De Boer.
Simon Che De Boer. Courtesy RealityVirtual.

Che de Boer’s interest in VR led to a passion for regaining what was lost in the past—not only for the world but also for himself and his daughter. In 2014, they lost their home in a fire. After this devastating experience, all Che de Boer wanted was to rebuild the place he called home. “The whole reason I got into [VR] was because I wanted to see if I could re-create my home,” he says. “I wanted to find a way to reproduce it and allow myself a sense of closure.”

Using his love of working with computers and his interest in computer graphics, Che de Boer began to explore VR. “Because I didn’t have a background in this, I didn’t realize what the limitations were,” he says. “I jumped into the scene without any formal training or even having any fundamental understanding of what I was doing. I set out an agenda; I wanted to capture ultrarealism.”

To achieve such realistic experiences in the virtual world, Che de Boer focused on the imperfections. “For me, it’s all about the feeling of what something creates,” he says. “And I know from the real world that the reason we feel [something is] so real is because of its imperfections—the perfection of imperfection.”

Che de Boer’s process challenges the limits of what most people perceive as possible for VR development. Instead of starting with a few thousand points of detail for a live scene and refining it before adding more points, Che de Boer and his team apply a brute-force approach. They use Autodesk Maya for 3D modeling and employ clever memory- and asset-management techniques to bake anywhere from 1 billion to 40 billion points of detail into a scene. Once they’re satisfied with the level of detail, they use Autodesk Meshmixer to work on the refinements and efficiencies—filling holes, removing noise, and cleaning up artifacts.

Detail is an essential part of the VR experience, one that matters most to Che de Boer. “I think a lot of people are so overwhelmed by detail that they don’t realize what’s actually there,” he says. “For me, I can look at an image and see how it feels on an artistic level. Lighting, mood, and all that stuff is really important to me. It’s the ability to take a step back and see the big picture where others might be getting too much detail and take it for granted.”

Che de Boer wants to continue documenting heritage sites, digitally preserving them for years to come. He’s looking to virtually re-create New Zealand’s ChristChurch Cathedral as it existed before it was damaged by a 2011 earthquake. He’s also working on a number of locations that he can’t yet disclose. “These are locations that everyone knows about but only few get to access,” he says. “We’ll hopefully democratize experiences that most people would never get an opportunity to have in their lifetime.”

As an advocate for what he calls “slice of life” experiences, Che de Boer envisions a future in which virtual reality can satisfy people’s thirst for new experiences. “Sometimes people just want to be moved and be in awe of the environment they’re placed in—like a beach, a coliseum, or a tomb,” he says. “If the realism is so good, then it’s like actually being there.”

More Like This

Success!

You’re in.

Get smart on the future of making.

Subscribe to our newsletter.