POV: Nathaniel Dirksen on the breakneck pace of "visceral" VR

By - - Maya
20 mins
Last modification: 21 Nov, 2017

In the race to establish a foothold in VR storytelling, Baobab Studios is a clear frontrunner. Founded just two years ago, by Maureen Fan of Zynga and Eric Darnell of Dreamworks, they’ve secured the financial backing of Hollywood heavyweights and Silicon Valley investors, earning them early comparisons with Pixar. Here, their Director of Engineering, Nathaniel Dirksen, discusses the tools and technology they use, and what Baobab are doing to keep up with the breakneck pace of VR’s evolution.



I’m a Director of Engineering at Baobab Studios. That means that I need to have a big picture understanding of everything we do technologically, and of the production pipeline and how all of the pieces fit together. It includes everything from modeling to animation, game engine integration, and rendering, not to mention all of the ancillary helper tools that keep things running smoothly. Given that I've been here a long time, I've got an advantage in doing all that, since I originally wrote a lot of the pieces.

As we've grown, I spend more time supervising, overseeing, and mentoring others as they're working on our systems, providing the broader context of goals, helping to ensure that everything will work well together up and downstream, and will provide an excellent experience for our users.  We're not so big that I don't get hands-on with a lot of the work. I tend to jump in wherever help is needed. I also do a lot of forward-looking planning based on that knowledge, like bidding out Engineering tasks, prioritizing dev needs, scoping projects, handling assignments for sprints, and making sure that all of the Engineering work gets done, and making sure folks have the tools they need to do their jobs.


"One of the things we always focus on is improving artist workflow."


There are a lot of people from the film side here at Baobab, and we hold ourselves to a super high–quality standard regarding the visuals that we're producing. One of the things we always focus on is improving artist workflow.

Film production is well understood, and there are a ton of great tools for doing film production. And it’s the same with gameplay – game engines are great for making games. But when you're creating narrative experiences, when you’re integrating film technology into the game technology, getting those pieces to work together is difficult. Creating tools in the game engine to light more cinematically, quickly and easily, and tools to bring in top–quality character animation that looks as though a film renderer rendered it is an ongoing challenge. Merging these two worlds and making them work together is imperative to making the impactful, narrative-driven content we want to be known for.



Because Baobab puts such an emphasis on empathizing with the character in our experiences and because we want nuance of expression and acting, most of our animators come from a film background. The modeling, the rigging, and the animation techniques are very similar to what you'd use for film, and we want the team to have tools that they're comfortable with, know well, and will allow them to achieve what we're after.

When I say that it's challenging to bring film technology, like Maya, together with game engines, it's because it's hard to get all of the nuanced expression that you'd get out of a good character rig being used by an excellent animator into game engines. They don't necessarily have all that technology built in – but we want that same expressiveness in the game engines that you can get in Maya, right?

“If we didn't have [Maya], we couldn't achieve the same level of empathy with the characters we create. And we have to have that.”

Maya is a tool that’s made to be at the front of a pipeline, that’s made for animating characters and having them expressed in fun, nuanced, understandable ways. If we didn't have that, we couldn't achieve the same level of empathy with the characters we create. And we have to have that. I think it's important to connect to our character once you put on the headset and find yourself next to them.

Further, Maya’s expandability/scriptability is hugely useful. On the Engineering side, it’s super helpful to easily run automated scripts and build our plugins for stuff like custom exporters. Without that expandability, it'd make the integration between the animators working in Maya and the game engines we use to present the content much more difficult.


I find the whole aspect of being able to put yourself into a different space and exploring that space to be truly exciting. You do get a different experience in VR than you do in a video game or a movie. That reaction that you can bring about in the viewer is so different.

And it's little things in VR. Even just scaling you up so that you feel like you're a giant is amazing. Yes, you can have a film raise the camera way up and give the audience a giant's eye view, but it’s not the same visceral feeling you get in VR. And, when characters look at you, interact with you, or respond to you in VR, the feeling is very different than if that were to happen in film. In film, you're the passive observer; there's always that fourth wall. If the character breaks that wall by turning to you and looking directly into the camera, it feels weird, right? Whereas with VR, it feels natural. It's visceral, it's cool, and you don't get that with other media.


When it comes to interaction in narrative experiences, it’s complicated but also fascinating because, in some sense, the two are opposed. In VR, you want to see the results of what you've done, which can make it hard to have a strong narrative arc and narrative timing then – you're giving more and more control to the user. To have a strong narrative arc in VR is an ongoing experiment and something we'll continue to explore for a long time because I don't think there's an obvious answer. I think it's an answer that'll change over time as people use VR and become more experienced.


“It's amazing how quickly it's all moving…it’s just pure excitement.”


I've been doing this for two years now, and there's always new stuff constantly coming out. When I started, it was Rift headsets and Xbox controllers. Now, you have hands in the scene and all the inside-out tracking so you can move around in an untethered fashion. It's amazing how quickly it's all moving.

In film, everyone knows how to make a 3D movie at this point, right? Sure, there will always be opportunities for advancement, gradual improvements to the pipeline and techniques, but it’s nothing compared to the pace of VR. And we're almost used to it now. If you don't have two or three new headsets in a year, it's like, "Oh, what's going on?" I'm sure it'll slow down eventually and become more settled, but for right now, it's just pure excitement. All our plans are based on this breakneck development pace.



Even with all the cool new tech, no one would say that we've reached the epitome of VR technology yet. I would love to get to way further, way faster. I would love if you could have great hand tracking with an untethered headset that didn't weigh too much and still have a ton of rendering power. The technology not being quite where we want it does, sometimes, bring about interesting creative restrictions though. The storybook look of our “Rainbow Crow” makes for a fantastic environment but it’s also one that we can render quickly. I think that a lot of these unique looks are really exciting. I enjoy the fact that the limitations of rendering technology still means that people have to be creative with their visual approaches. We get a lot of interesting visual designs as a result.

"Even with all the cool new tech, no one would say that we've reached the epitome of VR technology yet."

It’s the same with the hardware restrictions. There’s not much of a creative use in having a wire that ties you to a computer, right? We haven't found a way to make good use of being tethered to a box. Certainly, I'm excited to see the mobile headsets get six stop tracking, six-degree freedom so that you can track your position in addition to your orientation. I think that's going to make them so much more immersive than when it's just rotationally tracked. It'll make the 360 video approach was less viable, but the overall experience will be far superior.


We use VR as a production tool. Our animators are working in Maya, in a 2D fashion, typically, but they will preview their work in VR regularly because what works in 2D, often, once you get into VR, it doesn't even look the same. It's not like sheets that work when you've done it two cameras, but even staging and composition can feel very different when you're in this space.

It's the same thing with the previs process. We still do storyboards, and scripts, and all of that, like you, normally do. But then, at the point we're doing any early pre-visualization, we want to get things into VR so we can understand what the story is going to be like when you see it in the correct medium, right? We try to get them into VR as quickly and as early in the process as possible. I think it's essential to producing good results.

“Coming to Baobab, moving into VR, all of it has been rewarding.”


Baobab Studios is a very comfortable environment for me; it didn't feel like much of a cultural shift to come here from DreamWorks Animation – apart from it being a smaller company, obviously. Baobab is full of people from DreamWorks and those who aren’t are still the same sort of people: they're super passionate, super enthusiastic, super smart individuals. Coming to Baobab, moving into VR, all of it has been rewarding.

Huge thanks to Nathaniel and Baobab Studios for sharing your journey to VR with us.

Baobab's INVASION is available now for Samsung Gear VR, Daydream, Vive, Oculus and Playstation VR. An extended sneak peek of ASTEROIDS is available for Samsung Gear VR and Daydream, with a full episode launching in December.

For more from Baobab Studios, see:

Baobab Studios invades VR

Journey to VR POV: Eric Darnell of Baobab Studios on the emerging language of VR storytelling

Look behind the scenes of Baobab's INVASION!


Posted By
Published In
  • Maya
  • AR
  • VR
  • Film & VFX
To post a comment please login or register