Fine Art in VR

Vincent Van Gogh has not only influenced Expressionists and les Fauves, his impact has now reached the realm of virtual reality. Borrowed Light Studio’s The Night Café lets you walk into the iconic painter’s masterpieces.

Mac Cauley walks us through the development of his reimagining of Van Gogh’s world. 

What is The Night Café?

The Night Café is an experience where you can enter into Van Gogh’s paintings in a 3D space. The initial idea was trying to imagine what it would be like to step inside of a painting. That was the inspiration and general idea for it.

Where did this idea come from? Are you a Van Gogh or fine arts aficionado?

I’ve always really liked his work. I do some painting myself and I’ve been inspired by him. My background is actually in animation and filmmaking both live action and animation.

A few years ago when I saw the Oculus Rift Kickstarter, it brought back memories of my childhood experiencing some of the early VR products in malls. You could go and experience very rudimentary VR. Growing up, that was really fascinating to me. It was just like entering a digital fantasy world.

So when I saw the Kickstarter for the Oculus Rift, it got me thinking about all possibilities that could be created for it and pretty quickly, I thought how cool would it be to see a painting of Van Gogh’s in that kind of medium? So I just sort of went from that idea. I tried to figure out how that would look and how big of a space it should be and what kind of movement you should be able to move freely through the experience.

What products are in your pipeline?

I use Maya for modeling, animation, UV mapping, and a lot of core 3D functionalities. I use Mudbox for sculpting but more for the texturing. I occasionally use ZBrush for sculpting and Unity for all my game engine functionality.

Tell me about creating the Van Gogh environment.

The main reasons I picked The Night Café as the main painting to use is just because I knew that I needed a space that was contained and The Night Café happens to be one of the few interior settings that Van Gogh painted. It could be more realistically created and completely detailed so that you could explore every nook and cranny.

I had a few different style requirements that I wanted to meet. One was I wanted it to feel very organic. Van Gogh’s paintings and brushstrokes are very sweeping and organic feeling.  I wanted all of the models, not just the textures, but the models themselves to have that feeling. That meant individually modeling the objects from scratch and using various sculpting tools to push and pull the vertices in ways that made it look more organic. The first model I would put very rigidly and straight within Maya. Then I would bring it into Mudbox and push and pull some of the vertices to give it that organic look. And from there I would bring it back into Maya and unwrap the UVs. 

I would often have to optimize the topology of the model, specifically for the characters. I never went fully high poly because I knew this was going to be for real time and it was low poly and a lot of the detail was going to come from textures as well. I modeled sort of relatively low poly but I still wanted to optimize the master, so I used the Quad Draw tool in Maya to re-draw some of the polygon and vertices. 

Then I went to the unwrapping tool. I could jump right into Mudbox and bring it in for the texture painting aspect. The really nice thing about Mudbox is that it allows you to paint directly to bitmaps, which is something that some other software doesn’t allow for. The newer UV unwrapped tool was really helpful. It automatically unwrapped a lot of the UV’s so I could quickly get them in an optimal layout. And then tweak them to fit perfectly.

I really appreciated all the control that Mudbox gives you over the brushes and bringing in your own alphas. I made all my own alphas with the brushes to try and mimic some of the painting and pastel style brushstrokes. So I would create an alpha in Photoshop, bring it into Mudbox and then I would tweak some of the brush settings like the pressure sensitivity and the variations were useful in giving it a more organic painted look. Also, using a Wacom tablet in addition to Mudbox is useful for getting the realistic looking brush strokes.

What approach did you use for the animation?

Animation has always been something that’s important to me. I knew it was going to be very important to have animations that bring the characters to life in a way that’s different from the painting. You actually get a sense of his work because they are so alive in his brush strokes.

But, there’s only so much you can do without having actual animation. So, I wanted to bring things to life with various particle animation. I tried to achieve a certain level of realism in the self-portrait I did where he’s sitting in a chair smoking his pipe. I had him walk over to the window and, and looking out at the stars. I thought that would fit the mood of the piece. For the piano player that I used a sort of ambient style animation where there’s always constant music playing throughout the experience. I wanted it to be this diegetic sound, a sound that’s coming from the piano so that everything feels like this small moment in his life. So, yeah, the animations were a key feat of bringing Van Gogh’s world to life.

Borrowed Light Studios used Mudbox and Maya to create The Night Café. Take a deep dive into Van Gogh’s iconic world for free on Steam.