7/30/2019 22 SingleView
1/43
Scene Modeling for a Single View
15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005
Ren MAGRITTEPortrait d'Edward James
with a lot of slides stolen from
Steve Seitz and David Brogan,
7/30/2019 22 SingleView
2/43
Classes next term
15-465 Animation Art and Technologythis is the learn Maya and collaborate with artists to make a movie
class. It will likely NOT be offered next year so it is important that
the students understand that as it has been offered every
year for a while now.
http://www.cs.cmu.edu/~jkh/aat/
15-869C Special Topics in Graphics: Generating Human Motion
An advanced class on techniques for generating natural
human motion. This is a one-time offering.
15-864 Advanced Computer Graphics: Graduate Class
http://www.cs.cmu.edu/~djames/15-864/index.html
7/30/2019 22 SingleView
3/43
Breaking out of 2D
now we are ready to break out of 2D
And enter the real world!
7/30/2019 22 SingleView
4/43
Enough of images!
We want more of the
plenoptic function
We want real 3D scene
walk-throughs:Camera rotation
Camera translation
Can we do it from a singlephotograph?
on to 3D
7/30/2019 22 SingleView
5/43
Camera rotations with homographies
St.Petersburgphoto by A. Tikhonov
Virtual camera rotations
Original image
7/30/2019 22 SingleView
6/43
Camera translation
Does it work? synthetic PP
PP1
PP2
7/30/2019 22 SingleView
7/43
Yes, with planar scene (or far away)
PP3 is a projection plane of both centers of projection,so we are OK!
PP1
PP3
PP2
7/30/2019 22 SingleView
8/43
So, what can we do here?
Model the scene as a setof planes!
7/30/2019 22 SingleView
9/43
Some preliminaries: projective geometry
Ames Room
http://www.illusionworks.com/html/ames_room.htmlhttp://www.illusionworks.com/html/ames_room.html7/30/2019 22 SingleView
10/43
Silly Euclid: Trix are for kids!
Parallel lines???
7/30/2019 22 SingleView
11/43
(0,0,0)
The projective plane
Why do we need homogeneous coordinates? represent points at infinity, homographies, perspective
projection, multi-view relationships
What is the geometric intuition? a point in the image is a ray in projective space
(sx,sy,s)
image plane
Each point (x,y) on the plane is represented by a ray (sx,sy,s)
all points on the ray are equivalent: (x, y, 1) (sx, sy, s)
(x,y,1)
y
xz
7/30/2019 22 SingleView
12/43
Projective lines
What does a line in the image correspond to inprojective space?
A line is a plane of rays through origin
all rays (x,y,z) satisfying: ax + by + cz = 0
[ ]
=z
y
x
cba0:notationvectorin
A line is also represented as a homogeneous 3-vector l
l p
7/30/2019 22 SingleView
13/43
l
Point and line duality
A line l is a homogeneous 3-vector
It is to every point (ray) p on the line: l p=0
p1p2
What is the intersection of two lines l1 and l2 ? p is to l1 and l2 p =l1 l2
Points and lines are dual in projective space given any formula, can switch the meanings of points and
lines to get another formula
l1
l2
p
What is the line l spanned by rays p1 and p2 ? l is to p1 and p2 l =p1 p2
l is the plane normal
7/30/2019 22 SingleView
14/43
Ideal points and lines
Ideal point (point at infinity) p (x, y, 0) parallel to image plane
It has infinite image coordinates
(sx,sy,0)y
x
z image plane
Ideal line l (a, b, 0) parallel to image plane
(a,b,0)y
x
z image plane
Corresponds to a line in the image (finite coordinates)
7/30/2019 22 SingleView
15/43
Vanishing points
Vanishing point projection of a point at infinity
Caused by ideal line
image plane
cameracenter
ground plane
vanishing point
7/30/2019 22 SingleView
16/43
Vanishing points (2D)
image plane
cameracenter
line on ground plane
vanishing point
7/30/2019 22 SingleView
17/43
Vanishing points
Properties Any two parallel lines have the same vanishing point v
The ray fromC through v is parallel to the lines An image may have more than one vanishing point
image plane
cameracenterC
line on ground plane
vanishing point V
line on ground plane
7/30/2019 22 SingleView
18/43
Vanishing lines
Multiple Vanishing Points Any set of parallel lines on the plane define a vanishing point
The union of all of these vanishing points is the horizon line
also called vanishing line
Note that different planes define different vanishing lines
v1
v2
7/30/2019 22 SingleView
19/43
Vanishing lines
Multiple Vanishing Points Any set of parallel lines on the plane define a vanishing point
The union of all of these vanishing points is the horizon line
also called vanishing line
Note that different planes define different vanishing lines
7/30/2019 22 SingleView
20/43
Computing vanishing points
Properties P is a point at infinity, v is its projection
They depend only on line direction Parallel lines P0 + tD, P1 + tD intersect at P
V
DPP t+= 0
+
++
+
++
=
0/1
/
//
1
Z
Y
X
ZZ
YY
XX
ZZ
YY
XX
tD
DD
t
t
DtP
DtPDtP
tDP
tDPtDP
PP
= Pv
P0
D
C i i hi li
7/30/2019 22 SingleView
21/43
Computing vanishing lines
Properties l is intersection of horizontal plane through C with image plane
Compute l from two sets of parallel lines on ground plane
All points at same height as C project to l
points higher than C project above l
Provides way of comparing height of objects in the scene
ground plane
l
C
7/30/2019 22 SingleView
22/43
F ith i hi i t
7/30/2019 22 SingleView
23/43
Fun with vanishing points
T i t th Pi t (SIGGRAPH 97)
7/30/2019 22 SingleView
24/43
Tour into the Picture(SIGGRAPH 97)
Create a 3D theatre stageoffive billboards
Specify foreground objects
through bounding polygons
Use camera transformations tonavigate through the scene
Th id
7/30/2019 22 SingleView
25/43
The idea
Many scenes (especially paintings), can be representedas an axis-aligned box volume (i.e. a stage)
Key assumptions:
All walls of volume are orthogonal Camera view plane is parallel to back of volume
Camera up is normal to volume bottom
How many vanishing points does the box have? Three, but two at infinity
Single-point perspective
Can use the vanishing point
to fit the box to the particularScene!
Fitting the box volume
7/30/2019 22 SingleView
26/43
Fitting the box volume
User controls the inner box and the vanishing pointplacement (# of DOF???)
Q: Whats the significance of the vanishing pointlocation?
A: Its at eye level: ray from COP to VP is perpendicular
to image plane. Why?
7/30/2019 22 SingleView
27/43
High
Camera
Example of user input: vanishing point and back
face of view volume are defined
7/30/2019 22 SingleView
28/43
High
Camera
Example of user input: vanishing point and back
face of view volume are defined
7/30/2019 22 SingleView
29/43
Low
Camera
Example of user input: vanishing point and back
face of view volume are defined
7/30/2019 22 SingleView
30/43
Low
Camera
Example of user input: vanishing point and back
face of view volume are defined
C i f h i i bdi id d b d
7/30/2019 22 SingleView
31/43
High Camera Low Camera
Comparison of how image is subdivided based
on two different camera positions. You should
see how moving the vanishing point correspondsto moving the eyepoint in the 3D world.
A h l f i i hi i
7/30/2019 22 SingleView
32/43
Left
Camera
Another example of user input: vanishing point
and back face of view volume are defined
A th l f i t i hi i t
7/30/2019 22 SingleView
33/43
Left
Camera
Another example of user input: vanishing point
and back face of view volume are defined
A th l f i t i hi i t
7/30/2019 22 SingleView
34/43
Right
Camera
Another example of user input: vanishing point
and back face of view volume are defined
Another example of user input: vanishing point
7/30/2019 22 SingleView
35/43
Right
Camera
Another example of user input: vanishing point
and back face of view volume are defined
Comparison of two camera placements left and
7/30/2019 22 SingleView
36/43
Left Camera Right Camera
Comparison of two camera placements left and
right. Corresponding subdivisions match view
you would see if you looked down a hallway.
2D to 3D conversion
7/30/2019 22 SingleView
37/43
2D to 3D conversion
First, we can get ratios
left right
top
bottom
vanishing
pointback
plane
2D to 3D conversion
7/30/2019 22 SingleView
38/43
Size of user-defined back plane must equal
size of camera plane (orthogonal sides) Use top versus side ratio
to determine relativeheight and width
dimensions of box
Left/right and top/bot
ratios determine part of
3D camera placement
left right
top
bottomcamera
pos
2D to 3D conversion
Depth of the box
7/30/2019 22 SingleView
39/43
Depth of the box
Can compute by similar triangles (CVA vs. CVA)
Need to know focal length f (or FOV)
Note: can compute position on any object on the ground Simple unprojection
What about things off the ground?
DEMO
7/30/2019 22 SingleView
40/43
Now, we know the 3D geometry of the box
We can texture-map the box walls with texture from theimage
Foreground Objects
7/30/2019 22 SingleView
41/43
g j
Use separate billboard foreach
For this to work, threeseparate images used: Original image.
Mask to isolate desired
foreground images.
Background with objectsremoved
Foreground Objects
7/30/2019 22 SingleView
42/43
g j
Add verticalrectangles foreach foregroundobject
Can compute 3Dcoordinates P0,P1 since they areon known plane.
P2, P3 can becomputed asbefore (similartriangles)
Foreground DEMO (and video)
7/30/2019 22 SingleView
43/43
g
Top Related