It takes more than a global pandemic to slow Gary Yost down. Leading the original team that created 3D Studio and 3D Studio MAX (today known as Autodesk 3ds Max), Gary - now a prolific filmmaker - has been involved with virtual reality for some time now as co-founder of The WisdomVR Project, a non-profit dedicated to creating immersive cinematic experiences and developing a free online curriculum to help make VR filmmaking more accessible to the wider world. He was in the midst of delivering nine virtual reality experiences to Oculus when the COVID-19 epidemic shook up our world and gave him new inspiration: why not utilize the unique abilities of virtual reality to tell the story of COVID-19 in a way that traditional filmmaking could not?
The result of these efforts was Inside COVID19, at once a narrative-driven film about emergency physician Dr. Josiah Child's attempt to prepare five different hospitals to combat the virus and then, after he contracts it himself, an immersive 3D-dive into the molecular drama unfolding inside Dr. Child's body, animated in crisp 6720x6720 resolution by Andy Murdock using 3ds Max.
We had the pleasure of speaking with Gary about virtual reality, Inside COVID19, and his approach to building immersive experiences.
With Ram Dass capturing the first WisdomVR experience in 2018. (RD passed away in 2019)
Yost (right) with WisdomVR co-founder Adam Loften
On the Benefits of VR Filmmaking
Very few people are as enthusiastic and optimistic about the potential of storytelling in virtual reality as Gary Yost. "I think that the pandemic and creation of the COVID vaccines is our modern-day equivalent to the moon landing in the 1960s, in terms of a scientific achievement," he told us. For him, VR provides a level of immersion that makes the medium so compelling for telling this complex narrative. "We’re creating a cutting-edge documentary about a 21st-century pandemic and you’ll only understand how powerful it is once you're in the headset and everywhere you look, you’re surrounded by this tremendously compelling story. You can't look away.
VR technology is particularly good at overcoming traditional problems of scale. "We always used to say, back in the early days of Max development, that 3D animation was perfect for showing super tiny things and really huge things." And there's no better example than Inside COVID19 since for so many people, a major barrier to understanding the pandemic is conceiving of microscopic viruses, far too small for the human eye, as a real threat. The film Yost’s team has made cuts through all the misinformation by presenting the highly complex molecular processes in a comprehensible and visually compelling format.
Setting up the ZCAM V1 10-camera array used to capture stereoscopic 360 video
"I've had a lot of people tell me, after they've seen it in the headset, that they wish they could just stand on a street corner and hand it to people, ask them to put the headset on and watch, because it's just that powerful. Some people have even had to take the headset off because it was too scary for them. Once you're inside the cell and watching the infection process, it's genuinely terrifying; you're right there inside the cytoplasm watching the virus reproduce; you’ve never witnessed anything like it.”
On Making Inside COVID19
"I had been working in VR filmmaking for three years and had already started The WisdomVR Project as a way to tell important stories in this immersive medium when Oculus approached us with funding to do nine experiences during the winter of 2019/2020. Then the pandemic hit, and we shifted our focus towards using our skills and the VR tools to share the COVID-19 experience through the eyes of this amazing doctor, Josiah Child."
With WisdomVR subject Del Seymour, founder of Code Tenderloin, SF
In many ways, it was the perfect opportunity to showcase the potential of VR, since, remember, its biggest strength is in compressing large-scale visuals and blowing up small-scale ones. "As we started to figure out how to tell the story, it became obvious to us that it operated on multiple levels - not just the personal level, but also the molecular level; how we really get sick. And then there's this planetary level, about how all of these complex global issues; environmental justice, social and public health equity have come together around the pandemic."
The intricacies of how coronaviruses attack the immune system are well understood by scientists but conveying that knowledge to the general public - who don't necessarily have the requisite background information - is a serious challenge. So long-time 3ds Max animator Andy Murdock set about visualizing that process at the molecular level. "Andy, with his years of experience creating scientific animations for National Geographic and the Discovery Channel, was perfect for this task, and with rendering support from the Chaos Cloud we took on a massive technical challenge and finished the job in record time. Now, along with another of my 3ds Max co-inventors, Rolf Berteig, we’re using Max and Unity on the Oculus Quest to create an interactive simulation of how the immune system works, so that people with headsets can manipulate these processes themselves, greatly helping to understand how the body adapts, how the infection process works, and also how the vaccines work to stave off infections. Many people think of vaccines as scary and to get them over their fear, our game-like simulations let them actually touch and feel the vaccine as it interacts with the immune system at the molecular level, making the unknown both known and accessible."
SARS-CoV-2 virion floating over a patch of epithelial cells - 3ds Max
Gary and his team are now working closely with experts, including epidemiology and immunology professors based at the Johns Hopkins Bloomberg School of Public Health, to vet their information and to make expert decisions concerning the graphics themselves. "Obviously, the human body is incredibly complex, so we have to simplify things, especially for the new generation of mobile VR headsets like the Quest 2. It's always a question: what to show, what not to show?"
The result is that Inside COVID19 is an inside look into the human body under siege, and the success of the film is evident in the praise it's receiving, recently winning the People’s Choice award at the 2021 FIVARS (Festival of International Virtual and Augmented Reality Stories) as well as the interest of various organizations that are using VR as an instructional tool. “Right now, the Johns Hopkins Center for Teaching and Learning is using our Inside COVID19 VR documentary to explore how this technology can be used as a tool for teaching public health students and for communications about the pandemic. That’s tremendously exciting and personally gratifying.”
SARS-CoV-2 virion about to be grabbed by an ACE2 receptor (endocytosis) - 3ds Max
Ribosomal translation of SARS-CoV-2 proteins in epithelial cytoplasm (with organelles) - 3ds Max
The Future of VR Storytelling
Gary isn't done with VR - not even close. If anything, the success of Inside COVID19 has only increased his ambitions for the technology. “There are dramatic advances in human achievement occurring now, but these magnificent advances--in vaccine development--are taking place at a microscopic level, thus limiting the public’s ability to fully appreciate the science that is reshaping our shared global experience. But there’s good news: Immersive VR technologies make it possible to bring the story of COVID vaccines to life in powerful, easily understandable ways that will help the next generation of students around the world understand the extraordinary science--and meet the extraordinary scientists--making these vaccines possible."
"I want to do a series of VR documentaries about some of the major players in the creation of the vaccines," he told us, before naming scientists, researchers, and virologists across the world who have caught his attention. It would be a way of spotlighting these remarkable people, but with secondary benefits as well. "It's also a way of inspiring young people who are interested in STEM. Back in the mid-1990s, when we originally envisioned the toolset for 3ds Max, we never dreamt that one day we’d be able to create 3D animations and films that people could become fully immersed in as if they had become embodied in an entirely new universe.”
Gary's ultimate goal, however, is the stated mission of his WisdomVR Project: to teach young people how to use these VR tools, so they can tell stories in their own communities. "We acknowledge that there are an infinite number of stories, and we can only tell this tiny, tiny number, so our long-term goal, as an educational organization, is to share this knowledge and empower other people to tell their own stories."
For more on Gary Yost