Did you know that you can capture depth data on your iPhone 7 plus?
You might be asking yourself... depth data? What’s that?
It’s essentially a black and white image that reflects depth information.
The lighter  values mean the object is closer.
The darker values mean the object is farther back
This depth map is often used by CG artists and visual effects guys like myself.
We call this z-depth, data along the z axis
With depth data you can do things like change the focus,
add a foggy background
and achieve parallax effects like this.
The iPhone 7 Plus has two cameras on the front.
These two cameras essentially work like our two eyeballs.
Simply by comparing two images and subtracting the difference in parallax, the processor can calculate depth.
In portrait mode, the iPhone knows how to blur out the background thanks to this depth data.
The cool thing is that now with iOS 11 you can actually capture raw depth data as an image
using the iPhone 7 Plus.
At the moment I couldn’t find apps in the store
however, you can get developer code from apple’s developer program.
Just head on to the Apple Developer website.
Here is What you will need
A Mac that has the latest OS
You’ll need Xcode 11
An iPhone 7 plus with a USB lightning cable
First we are going to install Xcode
And download the three sample AVCam developer apps
Your going to get AVCam, wiggle me, and AVCam photo filter
First simply load the project file in Xcode
Plug in your phone
choose your user
And push play
Do this for all three developer apps
Now the apps should be on your phone.
Let’s start with AV cam
Turn on "Depth Data Delivery"
and turn on "Live Photo Mode"
Now take a picture.
This looks like a normal photo.
Don’t be fooled though
Embedded inside this pic is the depth data.
With a depth data app like wiggle me we can make use of it.
Open the wiggle me app and Load the picture.
Woah 3d...
You can pinch to zoom and move your phone around to see what happens.
That my friends is the power of depth data.
Now I am going to use AVCam photo filter to capture raw depth data.
And see what I can do with this image in After Effects and Zbrush
Load AVCam photo filter, Turn on depth,
Slide the bottom slider all the way up
Now at first It’s going to look all glitchy.
choose smooth, And voila, you have depth data
With a tripod, snap a depth photo
Next slide the bottom slider all the way down
And snap a normal photo
Now we will use these two images to create a 3d version of this photo
In After Effects I will be using Volumax Pro which can be found on videohive.net.
This is an awesome After Effects template made by Cream Motion
Essentially it takes a 2d image and turns it into 3d using this depth map
Just open the project, put your image in
“Your Picture Comp”
make sure to cover all the entire canvas
Copy the picture and paste it into
“2 Your Displacement Map.”
And now you can move and animate this image in pseudo 3d.
Using depth data
Let’s take it one step further
and actually make a real 3d model from this depth data.
First let’s crop these photos into nice squares.
We are going to use 4096 x 4096 pixels
Render those guys out
Next we’re going to open Zbrush and make a plane
Hit "T" on the keyboard
And select “Make Polymesh 3d”
Under “Texture” load the color image and apply the texture to the plane
Next in the alpha selector, load your zdepth map image
Under "Masking" choose "Mask by Alpha"
Now if we turn off our texture we can see that this geometry has been masked based off the Z-depth data
Let’s deform the geometry using the depth data as our stencil.
We will use the “offset” deformation along the z axis
There, now have this 3d image
But you’ll notice these artifacts
That’s because there is not enough bit depth in this image
Until this capturing technology improves to include more depth data, we will have to simply blur the mask.
So let’s blur the mask, lower our poly count, and deform again.
There, much smoother
Now we have full 3d control over our image.
Although it is not perfect, there is definitely room for development
This depth map capturing technology is good enough to do some basic parallax and blur effects.
But once we can capture more depth information
we will be able to manipulate things like lighting, shadows, and even capture our real environments into 3d virtual space.
But until then... I hope you enjoyed this video
please like share and subscribe!
Thanks for watching the Dusty Gamebox
