Creating a Virtual Reality Tribute to

Vincent van Gogh

Overview
-

Inspiration

Creative Approach

Technical Workflow

Tools

Learnings

The trailer for the experience can be watched here:

https://www.youtube.com/watch?v=jBOL5yakREA

recording gear VR videos is tricky, I used a paid app called Recordable

Inspiration

Eureka!

no eureka moment, the idea was an evolution

original plan was for a film, fictional painter inspired by van gogh, went into the woods and battled a mythical bear

I usually know when I have an idea I want to pursue when it keeps popping back into my head.

 stumbled upon an artist named Alexa Meade that made me believe an idea like this could work
doesnt require toon shading techniques, just carefully painted objects and characters

Le Caf de Nuit (The Night Caf)

1888

I selected The Night Cafe because it was perfect for the technical requirements.

It is an interior with a limited amount of space for movement.

many classic elements of VG style including radiating lights, skewed perspective, unusual combination of colors

interiors are preferable because they are limited no looking into infinity

Selecting the paintings:

not all the paintings were from van gogh

looking for objects to fill the spaces that needed to be imagined

colors, shapes, and theme all informed painting selection

 mapping out the things I wanted to achieve with the demo

initially the experience was going to have a lot of interactivity

ultimately decided to focus on the visual experience due to time constraints / deadline

 I think the focus on atmosphere over interaction worked in the projects favor for this experience

although interaction could definitely add immersion as well

Research

=?

What kind of lamp is this?

What is an arc lamp? Is this an oil lamp?

 there are a lot of oil lamps.

I also learned other interesting facts.

 Thomas Crapper popularized the flush toilet in the late 1800s

Research = Authenticity

researching history and visual references creates a more authentic world

this charming place is called Bandits Roost Alley and inspired what you see through the window in the back

Pre-Production

 laying out the floor plan was very important

helped to figure out where people could stand and what viewpoint they would have

 here is the added section compared with the original painting

 trying to complement van goghs color palette was a challenge

I wanted to create a shift in tone in the added room but it still needed to feel like a cohesive whole

I wanted it to feel more serene in that added room. you should feel a sense of ease as you go from the clashing colors of
the main room to this area

 heres the final look in the experience

Asset Production Workflow

Design Rules

Everything should look unique

Things should blend into each

other

Each object and character should

convey mood

1) Everything should have a unique look. Things shouldnt feel duplicated.

2) All the objects and characters should blend into each other as if brush strokes are intertwining.
3) On a more abstract level, everything should work towards a particular mood of being lost in a moment in time

Software Used
Modeling base mesh, UV layouts, rigging, animation
Sculpting details, painting textures
Game engine
Sketching, texture clean-up
Sculpting characters

Sketching

the first step is to sketch out the object

it is important to understand the form of what you will be modeling

Modeling

low polycount very important

total vertices in scene under 100k

it was important to give each of these props a sense of flow

first I constructed them with rigid forms

keeping a low poly count is a consideration for every object considering this is a gear VR game

Modeling

low polycount very important

total vertices in scene under 100k

I then used mudbox to warp the form to fit the painted style

Modeling

characters had a higher poly count so I had to use them sparingly

Texturing

Handpainted in Mudbox

No overlapping UVs

textures were all hand painted in Mudbox using a wacom tablet

UV layout was very important, making sure there were no overlapping UVs so every surface area could be painted uniquely

Texturing

characters typically took longer to paint because they incorporated more details

Samples of the modeling process can be seen here:

https://www.youtube.com/watch?v=jD-uM-Z6SN0

texture projection wouldnt work because of the surfaces you dont see from the painting
a lot of trial and error and making creative choices about how to best render non dynamic style lighting in a painterly style

Rigging
RapidRig = AWESOME!

rigging is often tedious

I discovered RapidRig on this project
reduced rigging time from 6+ hours to 1
caveat: you still have to paint skin weights

Animation
-

Kinect mocap is cool but low

quality

Hand keyframing is slow but worth

it

Fewer animations overall

Animation adds life to a scene

After doing testing with 2 kinects to get mocap data I discovered the animations were unusable

feet drift too much

automated filters to remove noise kill the life in the animation
had to keyframe it even if it meant less animation
animation helped to give life to the scene

Particle Tests

I did some very rough tests using particles to draw the color instead of shaders (wavy, flowing look)

would likely not have worked on gear VR

it was very tricky to achieve the details I wanted
messy and would have taken much longer to develop

Specular Bumped Shading

next I tested Specular Bumped shaders that incorporated normal maps

normal maps dont look as good in VR because they subtle lighting dierences you would see in each eye are not accurate
this style made it feel more like you were walking around in a painted box than inside the painting

Flat Shading

Finally I settled on a flat shader

looked the best because the colors were vibrant and matched the paintings color palette exactly
very good for performance (no normal maps, no per pixel lighting)

Controls
-

Simplified controls

Oculus SDK character controller

was 90% there

adding crouching for more control

over viewpoint

after going back and forth on controls I decided simpler is better.

not everyone has a gamepad for Gear VR

even my gamepad wasnt working properly (cost $60 bucks!)

the SDK controller had most of what I needed but I had to modify it slightly, it was having issues using tap and hold to
move forward, built in rotation had to be disabled
swiping down to crouch gave slightly more freedom to the player so they could see the environment from more vantage
points

Optimizations
-

Texture atlasing

Mesh batching

Texture/Audio compression

Reducing particle counts

Triggers to manually hide/show

objects (homebrew occlusion
culling)

I used scripting to manually handle some optimizations

particle eects were one of the biggest performance hits because they use alpha cutout shaders
manually triggering hide/show on these helped with frame rate issues

Learnings

Characters in VR are really cool

Particles are awesome (obviously)

Movement is tricky

The Note 4 is pretty capable

Stylized worlds can still be

immersive

as soon as I added characters to the environment that became my favorite thing to go check out
theres something about a moving, character that seems to have some life
particles are really popular in VR. the blooming particle lamps are usually the first thing people comment about when they
try it.
Movement is very tricky and I went through several iterations on control, speed, acceleration but settled on the most subtle
(slow, no acceleration, simple controls, strictly head rotation turning).
you might isolate select players because they cant handle any VR movement but it is a trade o.
the Note 4 handled this better than I expected, it was originally going to be a DK2 demo
stylized worlds without any shadows or per pixel lighting can still create an immersive experience that allows your mind to
day dream

Thank you!