Panoramic Stereo Videos with a Single Camera

Rajat Aggarwal∗ Amrisha Vohra* Anoop M. Namboodiri

Kohli Center on Intelligent Systems,
International Institute of Information Technology- Hyderabad, India.
{rajat.aggarwal@research, amrisha.vohra@research, anoop@}.iiit.ac.in

Abstract

We present a practical solution for generating 360◦

stereo panoramic videos using a single camera. Current
approaches either use a moving camera that captures mul-
tiple images of a scene, which are then stitched together
to form the final panorama, or use multiple cameras that
are synchronized. A moving camera limits the solution to
static scenes, while multi-camera solutions require dedi-
cated calibrated setups. Our approach improves upon the
existing solutions in two significant ways: It solves the prob-
(a) (b)
lem using a single camera, thus minimizing the calibration
problem and providing us the ability to convert any digital
camera into a panoramic stereo capture device. It captures
all the light rays required for stereo panoramas in a single
frame using a compact custom designed mirror, thus making (c)
the design practical to manufacture and easier to use. We Figure 1: (a) 3D printed model of the proposed coffee-filter mirror
analyze several properties of the design as well as present (b) Image of the proposed mirror placed in a scene (POVRay) (c)
panoramic stereo and depth estimation results. Stereoscopic panorama (red-cyan anaglyph) recovered from the
image in (b).

1. Introduction
The ability to capture and transmit high quality omni-
directional stereo videos enables several applications such
as virtual tourism, remote navigation, and immersive en-
tertainment. However, accurate omnidirectional stereo re-
quires the capture of a four-dimensional light-field. Peleg et
al. [17] showed that if we assume the two eyes are restricted
to move along a horizontal circle (panoramic stereo), one
can create independent 2D panoramic images for each eye
that closely reproduces the stereo views for a human. This
result theoretically enabled the capture of omnidirectional
stereo videos. However, practical systems for capturing
stereo panoramic videos are just emerging [1].
In recent years, display devices for stereo panoramas
have become ubiquitous with virtual reality (VR) headsets
that uses smart phone displays such as Google Cardboard
c
and Samsung GearVR
. Stereo panoramic videos allows ing the visual quality of the captured images.
∗ Equal Contribution
one to stand at any location in the world and look around
as if they were present in the real environment. However,
the capture of stereo panoramas currently requires a camera
that moves along a circle or a complex synchronized multi-
camera setup. We aim to develop a simple solution that
can capture 360◦ stereo panorama using an existing digi-
tal camera, thus making the creation process of immersive
VR content accessible to a much larger population. The
primary challenge here is to capture all the light rays cor-
responding two sets of cameras (left and right eye views)
arranged along the same circle to a single sensor without
causing blind spots or occlusions in the optical system. We
show for the first time that such an optical system is possi-
ble and practical using a set of custom designed reflective
surfaces that we refer to as the Coffee Filter Mirror (See
Fig. 1). We derive the surface equations of the mirror petals
and discuss the optimization of its parameters for maximiz-
As mentioned before, most of the existing approaches in-

13755
volve either multiple cameras to capture various perspective
directions [12, 1, 5], or a single camera that rotates along a
horizontal circle to acquire the images [17, 19, 20]. The
problem with moving or multiple cameras is that it makes
the calibration and camera positioning difficult thus mak-
ing the system bulky and delicate to use. They also give
rise to visible artifacts like motion parallax, visible seams,
synchronization errors and mis-alignments [13]. Moving
cameras also limit their use to static scenes and require ex-
tensive post processing to get the left and right panoramas.
In comparison, our approach has the following advantages:
1. Simplicity of Data Acquisition: In the multi-camera
systems like Google Jump [1] or the system proposed
by Amini et al. [5], all the cameras need to be syn-
chronized using an electronic system to ensure that the
images are captured at the same time. The use of a
single camera eliminates the need for synchronization
and reduces the size of the device making it easy to be
used and handled. Data is acquired in the form of a
regular image or video and may be stored in standard
formats.
2. Ease of Calibration and Post Processing: Our ap-
proach solves the problem without any moving parts,
thus simplifying the calibration process. As we ex-
plain later, simple binary patterns can be used to cal-
ibrate the relative configuration of the camera and the
mirror. This also simplifies the post-acquisition de-
warping process to obtain the left and right panoramas.
The whole process of data acquisition and post pro-
cessing can easily be done on a smartphone, making it
a panoramic stereo video capture and display device.
3. Adaptability to Various Applications: Our custom
designed mirror can be easily used as an attachment
to any consumer camera to convert it into a stereo
panoramic capture device. The size of the mirror can
be adapted according to the application and the field of
view can be controlled.
4. Complete omni-directionality: Our design can be ex-
tended using a concave lens to capture mono images of
the top or bottom region. We can thus generate a com-
plete 360◦ x 270◦ view of the world captured using a
simple set up.
We describe in detail the design of the proposed cata-
dioptric omni-stereo system. We also show that our system
works efficiently for acquiring both 3D images and videos
and for both static and dynamic scenes. We discuss the op-
timization of the design parameters and present the recon-
structed stereo panoramas.
2. Related Work
Last two decades have seen significant advances towards
achieving omnistereo imaging. The first set of approaches
3756
generates a pair of panoramic images with a vertical off-
set. While such a geometry results in simple epipolar lines
leading to fast disparity estimates, they are not suitable for
human stereo perception as our brain expects a horizontal
disparity. Gluckman et al. [12] proposed the use of two om-
nidirectional cameras, each consisting of a parabolic mir-
ror, telecentric optics and a conventional camera, that were
aligned vertically along the same axis. Kawanishi et al. [15]
proposed a more complex setup that replaces each omnidi-
rectional camera in the above device with 6 cameras and
a hexagonal mirror. While this increases the resolution of
the omnidirectional images, the disparity remains vertical
as before. Lin et al. [16] created omnidirectional stereo us-
ing two cameras and a conical mirror, which limits the ver-
tical FOV. Yi and Ahuja [23] improved on the above designs
by capturing stereo omnidirectional panoramas with a sin-
gle camera. Their system uses a mirror-lens combination to
create two light paths that reflect off different positions in
the mirror forming a stereo pair with a vertical disparity.
Acquiring a horizontal disparity panoramic stereo re-
quires one to record a 3D light-field that is the equivalent
of all cameras with centers along a horizontal circle. Pe-
leg et al. [17] showed that this may be compressed to two
360◦ panoramas, one for each eye, without affecting the vi-
sual perception. They proposed the use of a single camera
that rotates in a circular trajectory to acquire the 3D light-
field. Strips of images are then extracted from left and right
ends of the frames that are stitched to create the right and
left eye panoramas respectively. While this method works
quite efficiently for static scenes, it is affected by artifacts
like visible seams and vertical parallax for moving objects
or uneven camera motions. These errors cause major mis-
perceptions when viewed in 3D as explained by Held and
Banks [13]. Couture et al. [8, 9] proposed the use of a ro-
tating stereo camera pair and stitching complete frames in-
stead of small strips, which can generate panoramic stereo
video textures. Use of rotating cameras limits the system
to small repetitive motion and suffers from the perception
artifacts like visible seams and vertical parallax.
Richardt et al. [20] proposed a flow based blending ap-
proach that helps reducing these visual artifacts and works
well for images captured using hand held cameras. How-
ever, the use of SFM makes this approach computationally
expensive when applied to high resolution images.
Peleg et al. [17, 19] also explored creating panoramic im-
ages using a perspective camera and a spiral mirror. They
could capture the 360◦ panorama using three such setups,
each acquiring 132◦ . They also proposed the use of a fres-
nel lens cylinder around an omnidirectional camera for each
eye. Both systems were complex and hard to manufacture
and use because of its large size and delicate structure.
To reduce visual artifacts and capture dynamic scenes,
synchronized multi-camera setups were recently proposed.
 Principle Camera Name Device Type of Perceived Artifacts Image
setup scenes Resolution
Omnistereo [17] Rotating Static Mis-alignments, stitching artifacts & Medium
Single Camera based Vertical parallax
Megastereo [20] Rotating Static No artifacts but extensive run time for High
SFM
Coffee-filter Fixed Dynamic No stitching artifacts and horizontal Medium
(proposed) disparity errors. Very less vertical
disparity errors.

Multiple camera Omnistereo, fresnel 2 Fixed Dynamic Chromatic aberrations, difficult to Low
based, needs Lens solution[17] cameras manufacture and use , Self-occlusions
camera Panoramic Stereo 2 Rotating Dynamic Visible seams and vertical parallax High
synchronization Video Textures [8, 9] Cameras textures
Google Jump [1] 16 Fixed Dynamic No stitching artifacts but setup is High
Cameras expensive and bulky
Omnipolar [6] 6 Fixed Dynamic Self occlusions , setup is expensive and High
Cameras bulky
Hexagonal Pyramidal 12 Fixed Dynamic Vertical disparity errors High
Mirrors [15] cameras

Table 1: Omnistereo device/approach comparison based on device setup complexity, Type of scenes that the camera can handle, Perceived
artifacts in the omnistereo experience and resolution of the omnistereo panoramas.

P
Scene
point
Couture and Roy [6] have proposed a setup which uses 6
L
cameras with fisheye lenses. Large occlusions are visible
in the produced omnistereo panoramas due to the huge b V V
R
curvature of the lenses. Also to reduce depth distortions, Viewing circle

the number of cameras needs to be increased, which in Image surface

Right eye views Left eye views
turn makes the system bulkier. Tanaka and Tachi [22] also
proposed a method to capture omnistereo video sequences. (a) (b)
Their rotating optics system consisted of prism sheets, Figure 2: (a) The scene point P viewed by L and R eyes forming
circular or linear polarizing films, and a hyperboloidal a viewing circle with diameter equal to baseline (b) Arrangement
of mirrors for capturing right and left views. Tangential rays are
mirror. The latest and most effective solution is a simple
captured by placing a mirror normal to the viewing circle.
extension of the idea by Peleg et al. [17], where the rotating
camera is replaced by 16 static cameras along a circle. This
circle, as shown in Fig. 2a. The diameter of the viewing
virtual reality camera was introduced at the Google I/O
circle is equal to the baseline. For each viewpoint, the set
conference, called ‘Google Jump’ [1]. All the cameras are
of tangential rays in the clockwise direction account for the
synchronized to take frame-aligned videos, which are then
left eye views, and the set of tangential rays in the anti-
stitched to form a complete 360◦ video. Although several
clockwise direction account for the right eye views. To ac-
systems have been proposed in the past most of them
curately capture stereo information, the camera should be
either suffer from limited FOV, or are limited by their size,
able to capture all the rays tangential to the viewing circle.
calibration and alignment issues and cost of manufacturing.
For this purpose, we propose a special mirror design for
Our goal is to reduce this to a simple catadioptric system
omnistereo viewing, called coffee filter mirror, owing to the
using a custom designed mirror and a single camera. We
similarity in shape. Using the image captured by this mir-
also provide a theoretical foundation of the design choices
ror we generate panoramas for left and right eye views with
and demonstrate results for both static and dynamic scenes.
appropriate disparity. The disparity in these images is used
Table 1 shows the more detailed comparison of our device
to perceive depth when seen in 3D using a VR headset such
with other omnistereo devices/approaches.
as Google cardboard [2].
3. Design Overview
Consider the arrangement of two flat mirrors P1 P2 and
3D information can be extracted from two images taken P2 P3 as shown in Fig. 3b, P B is the tangent to the view-
by two cameras horizontally displaced by a baseline. For a ing circle V and represents the direction of the rays that are
complete 360◦ view, the two cameras, analogous to the two required to be collected for the right eye view. Using a mir-
eyes, can move around the center of a circle called viewing ror P1 P2 , normal to the rays falling in the direction P B ,

3757
 β A petal
β face of a petal
β P3
β B
V
tangents to
the viewing Cmin
circle Cmax
β
β consecutive areas of the scene. However, this is only pos-
viewing circle: V
(a)
Figure 3: (a) A general horizontal cross-section of the proposed
coffee filter mirror.(b) Combination of the two arrangements as
shown in Fig. 2b to capture both eye views in a single design.
the incident rays can be reflected to the camera placed at
the center of the design. P1 P2 provides a horizontal field
of view to the eye and is analogous to the strip width in the
image-based approaches as mentioned in [17]. Each such
mirror is referred to as a face. Multiple faces, when ar-
ranged in the configuration shown in Fig. 2b, captures all
the tangential rays required for constructing the panorama
for a single eye. These faces are arranged at equal angular
separation such that Pi (even i) lies on the circle Cmax and
Pj (odd j) lies on the circle Cmin . Rays that are falling in
the direction of P B correspond to the view of the world as
seen from the right eye, as shown in Fig. 3b.
Our design is motivated from the idea to capture both
eyes’ views in a single device for omnistereo imaging. Be-
tween two consecutive faces that are catering to same eye
view, we introduce a similar face that captures a second eye
view.We combine the two arrangements of Fig. 2b in a sin-
gle design, as shown in Fig. 3b, such that rays for both eyes’
views are reflected to a single camera. The combination of
one left and one right face is referred to as a petal in rest
of the paper. Different number of petals can be used de-
pending upon the application requirements. Fig. 3a shows
a general horizontal cross-section of the proposed device to
explain the structure and the nomenclature that would be
used further in the paper.
4. Design Details
In order to provide a better 3D experience to the user,
it is important that the horizontal FOV of each face of the
mirror is sufficient enough to avoid any stitching artifacts
and mis-alignments. Also, vertical FOV should cover the
appropriate height of the world and the resolution must be
uniform across all the regions of the captured scene. In this
section, we explain how these factors affect our choice of
design parameters.
4.1. Horizontal Field of View
Horizontal field of view refers to the amount of the scene
captured by a face in the horizontal direction. In our design,
this is directly dependent on the number of petals, say n.
P2
P1 P To cover the complete 360◦ view, each face should cover
at-least 2π
n FOV. As shown in Fig. 4a, face P1 P2 , which is a
flat mirror, views only the region parallel to its length. The
O b
best arrangement of the next face for the same eye view,
P3 P4 would be such that the FOV of the two mirrors cover
sible if the mirrors are arranged linearly. FOVs can not in-
(b) tersect at any point if flat mirrors are arranged in a circular
manner, as shown in Fig. 4a. Hence, certain areas of the
world will be left unseen by the camera and will be missing
from the captured images.
Overlap between the
two FOV
P2
Flat mirrors P1
changed to
curved P3 P4
(a) (b)
Figure 4: (a) Flat mirrors have limited FOV, causing blind spots (b)
Curved mirrors increase the FOV and the overlap between consec-
utive faces of an eye.
Field of view of each face is increased by opting for a
horizontal curvature for the face as shown in Fig. 4b. Center
of each curved face lies on the tangent to the viewing cir-
cle. Increasing the curvature of the faces, increases the over-
lap between the FOVs of two consecutive same eye view’s
faces. This overlap is advantageous while de-warping the
captured image into the panoramas, as it resolves the prob-
lem of missing regions. However, the increase in curvature
also increases the inter-reflections between the neighboring
faces.
4.2. Inter reflections
Due to the curvature in the faces in the proposed design,
the FOV of the adjacent faces aligned at an obtuse angle
β, overlap. As shown in Fig. 5, a ray originating from the
camera that strikes the face P2 P3 , may get reflected multi-
ple times because of the FOV overlap between faces P2 P3
and P3 P4 . These inter-reflections will cause mis-captured
information at the camera sensor,thus decreasing the reso-
lution of the final de-warped panoramas. Hence, an optimal
amount of overlap is kept such that inter-reflections are kept
to a minimum, while also solving the problem of missing
regions as explained in Section 4.1.
4.3. Vertical Field of View
In the past, several catadioptric systems using various
combinations of flat and curved mirrors have been used to
increase the amount of the scene captured by the camera
in the vertical direction. For better quality and perception
3758
 P2 P P
FOV1
L β/2 β/2

rc
F

l
C
Inter- +
P1 A
C D B A
reflections B E
FOV2 θ/2θ/2 E

O b
P3

Rmin
R max
V
Cmin
P4
Cmax

Figure 5: Inter-reflections cause wrong world points to be captured
due to FOV overlap between each pair of left and right eye view
mirrors
of the de-warped images, it is important that the resolution
is uniform both horizontally and vertically. In a flat mir-
ror, vertical FOV is equal to the height of the mirror and
hence the resolution remains constant.Our proposed design
is such that at each horizontal cross section the faces can be
confined within a circle. The radii of these circles increase
from 0 to Rmax . As we need uniform resolution, we see
that a parabolic curvature in the vertical direction is more
desirable as compared to a hyperbola or straight line(see
Fig. 6)1 . There have been several approaches proposed to
attain uniform resolution on some planes or uniform angular
resolutions [7, 11, 14]. However, there work focuses mainly
on creating sensors with uniform resolution along the radial
line, whereas our mirror is designed to capture stereoscopic
views. Therefore, to simplify the set up and the dewarp-
ing process, at present, We seek optimality in terms of the
size, complexity, and minimizing inter-reflections. Creat-
ing a device that can optimally capture stereoscopic views
with uniform radial resolution is an aspect that needs to be
explored in future.
Differential angle (deg)
(a) (b)
Figure 7: Geometry of the petal surface used to obtain optimal
design parameters.
for left and right eye. The design of the mirror is symmetri-
cal, and all the petals are of same size and dimensions. The
distance between the two extreme points of a mirror surface
i.e AP as shown in Fig. 7a is referred as petal length, de-
noted by l. Each petal, say Pi , where i = 1 to n is bounded
by a circle Cmax with radius Rmax , and inside by a circle
Cmin with radius Rmin . V is the viewing circle with radius
equal to b. From Fig. 7a, OA = Rmin and OP = Rmax .
From △OAP and △OBP , by sine rule we get the relations
as, sin(l θ ) = Rmax
(θ+β)
Rmin
= sin( β
2 sin(π− ) 2 ) 2
Note that ∠AP O = ∠BP O = β2 and ∠AOP =
∠P OB = θ2 , because each face is symmetrical and ori-
ented at equal separation. ∴ ∠OAP = π − θ2 − β2 . LD is
the perpendicular bisector of the chord AP and is tangent to
the viewing circle V . OD = b is the radius of the viewing
circle. In △OCD and △CLP , we get LP = CP cos( β2 ),
implies CP = LP sec( β2 ) = 2l sec( β2 ).
In △P LC, ∠LCP = π2 − β2 . ∠OCD = ∠LCP ,
being vertically opposite angles. Hence, ∠COD = π2 −
∠OCD = β2 . Also, OC + CP = Rmax , which gives,

Parabola
4
OC = Rmax − 2l sec( β2 )
Cone
Hyperbola

β β
2 In △OCD, OD OC = cos 2 , which gives OC = b sec( 2 ).
β β
0 Using this we get, Rmax − 2l sec( 2 ) = b sec( 2 ) and
0 2 4 6 8 10 Rmax = (b + 2l ) sec( β2 ). Combining all these, we get:
Radial Length
Figure 6: Uniformity of vertical resolution in terms of differen- 2b sin( θ+β
2 )
tial angle of the incident rays along the radial length for a cone, Rmax = (1)
parabola and hyperbola. sin( θ+2β
2 )
sin( β2 )
Rmin = Rmax (2)
5. The Mirror Surface sin( θ+β
2 )

We now derive the equation of the mirror surface. Mul- sin( θ2 )

tiple factors can be varied to make the device adaptive to
specific applications. We derive the expressions for only
one petal AP B as shown in the Fig. 7a. and the same ex-
pressions hold for all n petals rotated by 2π/n. Circular
surfaces AP and P B are used to capture the right and left
eye view respectively. Let us consider the angle between the
chords of these two faces as β. Each petal subtends an angle
θ at the center, where θ = 2π
n . Hence, we get n views each
1 Please see Supplementary Material for more mathematical details
l = Rmax (3)
sin( θ+β
2 )
5.1. Optimizing the design parameters
In our proposed design, disparity and mirror size can be
altered depending upon the application requirement. Size
of the mirror is proportional to Rmax . In order to have
a compact mirror design that generates human perceivable
stereo panoramas, the design parameters need to be opti-
mized. The minimum value of the outer radius of the coffee

3759
 P

filter mirror i.e. Rmax is dependent upon β. At petal an- rc

/2 A
gle, βopt = π−θ 2 , Rmax is minimum, and hence we get the
/2
minimum size of the device. (xc,yc) /2
2
In Fig. 7a, Let ∠P BE be α, the angle between two d /2 (xd,yd)
1

petals. In △OBP , ∠OP B = β2 , ∠P OB = θ2 and

∠OBP = π − θ+β θ+β
2 . Therefore, ∠P BF = π −(π − 2 ) =
θ+β
2 . ∠P BE = 2∠P BF , which means α = θ + β. Con-
sider Fig. 7b where O′ is the center of curvature of the face
P B. P O′ and O′ B are the radii of curvature i.e rc and
∠P O′ B = 2γ is the angle subtended by each face at the
center of curvature. In △P O′ B, ∠A = π − (θ + β), which
implies, γ = π2 − ∠A = (θ + β) − π2 . In order to have small
mirror size, γopt = (θ + βopt ) − π2 . Therefore, the optimal
horizontal angular field of view γopt = θ2 and is indepen-
dent of the obtuse angle ∠P BE between nearby faces.
O′ C is the perpendicular bisector of P B, CB = 2l .
l z = mφ r 2
In △O′ CB, r2c = sin γ. Radius of curvature rc can be
optimized by using the optimal value of γ. Therefore,
l
rc = 2 sin θ , is the optimal radius of curvature. It is to be
2
noted that these centers of curvature lie on a circle.
To avoid wastage of pixels due to inter-reflections, as ex-
plained in Section 4.2, it is important to collect the maxi-
mum scene information in the captured image. Each face
covers 2θn angular FOV, thus a total of n such faces for
each view covers complete 2π FOV. For no missing regions,
FOVs of two faces for the same eye views should be cov-
ering consecutive areas of the scene. This is achieved by
aligning one face in the direction of O′ P and the next face
for the same eye view, in the direction BE. Hence the ob-
tuse angle between the two faces P B and BE is π+θ 2 . The
amount of inter-reflections depends upon the angle between
two consecutive petals, α, which depends upon the sam-
pling angle of the mirror 2π
n . Ideally, the amount of inter-
reflections reduces down to zero, when the FOV of two con-
secutive faces do not intersect at all. However, this way,
some of the scene regions will be left uncovered in the FOV
of some faces and hence not imaged at all. In order to ac-
count for these inter-reflections, we introduce a small angle
δ such that the angle of curvature becomes 2γ + δ. This
makes sure some overlap is there, so that some redundant
information is captured, which can be used while dewarp-
ing. However, the value of δ is kept sufficiently low, such
that inter-reflections are also reduced to a huge extent.
5.2. Resultant Mirror Surface
In this section, we obtain the surface equations of the
proposed coffee filter mirror in terms of polar coordinates φ
and r. As explained earlier, the surface of the coffee filter
mirror is such tha each face is paraboloidal vertically and
at each horizontal cross section the faces can be confined
within a circle. The radii of these circles increase from 0
to Rmax . Let us consider the central axis of the mirror to
be the z axis. Then the surface equation can be written as a
c
1
(0,0)
Figure 8: Parameters of the mirror petal.
function of x and y axis:
z = f (x, y) = mφ (x2 + y 2 ), (4)
where mφ is the slope of the parabola for a given φ. Let
x2 + y 2 = r2 , where r is the radial distance in the XY plane
and φ is the angle of the radial line, then:
(5)
Eqn 5 represents the petal surface of our custom de-
signed mirror centered around origin. Consider the upper-
most and widest cross section of the mirror at z = zmax ,
such that zmax = mφ r1 2 , mφ = zmax r12
. Let (xc , yc ) be
the center of the circle of curvature of a face of a petal and
(xd , yd ) be the point which lie on the curvature, r12 = k 2 r2
such that xd = kx and yd = ky. rc be the radius of the
circle of curvature for a face. Combining this with Eqn 5,
we get mφ = zkmax 2 r 2 which implies, z =
zmax
k2 . It is to be
2 2 2
noted that xc + yc = dc , Calculating distance from center
of the curvature and the point on the curvature we have:
(xd −xc )2 +(yd −yc )2 = (kx−xc )2 +(ky−yc )2 = rc2 (6)
p
(xxc + yyc ) + (xxc + yyc )2 − r2 (d2c − rc2 )
=⇒ k =
r2
Since, mφ = zmax /k 2 r2 ,
 2
r
mφ = zmax p
(xxc + yyc ) + (xxc + yyc )2 − r2 (d2c − rc2 )
as mφ = zmax /k 2 r2 . Also, it can be derived from
the Fig. 8, (xc , yc ) = (xd , yd ) + rc (cos(θ1 + θ2 + β2 +
θ2 ), sin(θ1 + θ2 + β2 + θ2 )), where θ2 = tan−1 ( 2rl c ).
From this and Eqn 5 we get,
!2
r2
z = zmax p
(xxc + yyc ) + (xxc + yyc )2 − r2 (d2c − rc2 )
(7)
Therefore, Eqn 7 gives the equation of the paraboloidal
surface of the mirror. Note that the slope mφ at every point
is a function of r. Any incident ray I coming from the world
point falls on the mirror surface at the point P (r, φ) and
ˆ φ))n(r,
is reflected in the direction R̃ = 2(Ĩ · n(r, ˆ φ) − Ĩ,
where n(r, φ) is the direction of normal vector at P . n is the

3760
normal vector to the tangent plane containing the tangents in
both horizontal and the vertical direction. For a horizontal
plane, the direction of normal vector nh at point (r, φ) is
T

given by yc − r sin φ r cos φ − xc 0 where (xc , yc )
are the center of curvature. Similarly, the normal vector nv
in vertical direction is obtained by dx
 dy dz T

which
dr dr dr
 T
is obtained as cos φ sin φ 2mφ r . Hence, the normal
vector to the tangent plane at point (r, φ) is obtained by
n = nˆh × nˆv .
5.3. Calibration and De-warping
Images captured by the camera using the coffee filter
mirror are of the form as shown in Fig. 1b. Captured images
varies with the orientation and viewing angle of the camera.
However, for stereo vision to be perceivable, camera’s view-
ing axis must be aligned with the central axis of the device.
To calibrate our device, we use a generic non-parametric
camera calibration method as proposed by Posdamer and
Altschuler [18]. We project structured light binary patterns
onto a display surface and project both normal and inverse
binary sequence patterns. The obtained calibration images
together will be used to compute a mapping from 3D world
coordinates to 2D image coordinates which is used for de-
warping the captured scene image into left and right eye
panoramas.
6. Epipolar Geometry and Stereo
To find the center of projections of the proposed coffee-
filter mirror, we find the trajectory of the viewpoints by
finding the intersection of the reflected rays for every pair
of adjacent points. Let a point lying on a radial line with
single φj is denoted by pi,φj . Then the reflected rays for
each pair of adjacent points pi,φj and pi+1,φj intersect at
the ci,φj . The set of all such points form a line as shown in
Fig. 10a. Lφj = {ci,φj }∀i. We take the average of these
intersection points c′φj , to find the average viewing circle
for complete 360◦ view. Locus of c′φj for φ for a single
face is a straight line parallel to central axis of the camera.
Let c′′k be the average center for a single face. Locus of c′′k
for all n faces is viewing circle, as shown in Fig. 10b. In
order to have a mirror with diameter of the viewing circle
equal to the baseline of human eye, diameter of the upper-
most cross-section must be twice the human baseline. With
the change of diameter of the cross-section from top to bot-
tom, both linear and angular disparities change. However,
for panorama to be perceivable, linear disparities are more
important than the angular disparities. The change in linear
disparities are avoided by interpolation during dewarping.
The combination of mirrors and a conventional camera be-
have as a non-central catadioptric camera. For details on
such classification, readers are directed to [21].
Let us consider a point in 3D world defined by (X, Y, Z)
3761
which is imaged by a mirror surface at point P (r, φ), then
the ray coming from X to P is viewed by some other mirror
surface at location P ′ (r′ , φ′ ). The set of such points form
an epipolar curve for the point P . Each point is then trans-
formed into the corresponding image coordinate using the
calibration explained in previous section. Epipolar curve
for a point in the left face is found by minimizing the dis-
tance between the reflected rays from a point in a left face
P to every other point in it’s right face P ′ . Thus, for each
φ in the mirror surface, we find the rφ which intersects the
reflected ray from point P such that,
|[PP′ , Ir,φ , Ir′ ,φ′ ]|
=0 (8)
|Ir,φ × Ir′ ,φ′ |
where Ir,φ represents the direction of reflected ray from
mirror surface. Since, the design behaves as a non-central
camera, every point has different epipolar constraints. We
calculate stereo disparity between the left and right views
by finding the correspondences along these epipolar curves.
7. Results and Discussions
In order to test our design, we have modeled the cof-
fee filter mirror using POVRay [4], a freely available ray
tracing software tool that accurately simulates imaging by
tracing rays through a given scene. We have used two 3D
scene datasets [10, 3] to demonstrate how the proposed mir-
ror is used to create stereo panoramas (red-cyan anaglyph)
as shown in Fig. 1c and Fig. 9. Our mirror can also be used
to capture dynamic scenes and create 3D stereo videos. As
shown in the results, the proposed coffee filter mirror is able
to capture 103◦ FOV in vertical direction and 360◦ in the
horizontal direction. The image of the simulated scene is
captured using the simulated proposed setup and a virtual
camera at a resolution of 5000 x 5000 and is dewarped into
two left and right panoramas of resolution 512 x 8192.
Fig. 11a shows a stereo depth map of the POVRay scene
‘average office’[10] computed using the coffee filter mir-
ror. The qualitative comparison of the depth map with the
ground truth is shown in Fig. 11b. Fig. 12a and Fig. 12b
shows left and right panoramas respectively created using
12 petals of the initial manufactured prototype of the coffee-
filter mirror. Additional results including anaglyph images,
videos of dynamic scenes and stereo depth maps may be
found at the project website 2 .
We have created a physical prototype of the proposed
mirror (Fig. 13(b)) with 24 petals, capturing 48 different left
or right eye views. Each face subtends an angle θ = 15◦ at
the center of the coffee filter mirror. The total height of the
mirror zmax is kept as 5 cm and the radius of the hole rmin =
2.7cm. We have kept b = 6.5 cm which is equal to the aver-
age value for human baseline. This optimal petal angle βopt
2 http://cvit.iiit.ac.in/research/projects/panoStereo/
 Figure 9: Red-Cyan anaglyph panorama obtained by using the proposed set up using POVRay dataset [3].
6 4
ABCDE
5
2
4

3 0
Z
Y

E
D
C
B
2 A
−2
1 (a)
0 −4
0 2 4 6 8 10 −4 −2 0 2 4
X X

(a) (b)
Figure 10: (a) Lines in black represents the locus of the intersec-
tion of consecutive rays for different radial lines lying on a single (b)
face (b) Red and blue curves represent the caustic curve for left Figure 12: Left and right panoramas extracted from 12 petals of
and right faces respectively. an initial prototype of the mirror shown in Fig13(b).

ters between faces that causes the artifacts in Fig. 9. How-

ever, the ratio of the vertical shift to the horizontal shift is
(a) very small and does not create any visible artifacts beyond
a depth of d. The value of d turns out to be 70 cm when the
rendering cylinder is kept at 200 cm, for images captured
at 5000 × 5000 resolution. These could also be corrected
like other approaches by depth estimation at the dewarping
(b)
Figure 11: (a) Comparison of reconstructed depth as obtained us- stage.
ing the proposed set up with the ground truth depth map.

comes out to be 82.5◦ , the optimal values of Rmax = 9.77

cm, Rmin = 8.571 cm and l = 1.696 cm. To mitigate the
effects of inter-reflections in the adjacent faces, as explained
in Section 4.2, we introduced a small angular overlap be-
tween two adjacent petals as δ = 2◦ . This means each petal
captures with a redundancy of 1/15◦ , since θ = 15◦ . The (a) (b)
resulting panoramas are shown in Fig. 12. Degradations in Figure 13: (a) Mockup of proposed device attached to a consumer
quality of images are due to imperfections of the mirror sur- cellphone camera (b) Set up created using initial manufactured
face that was created by SLA printing. prototype of coffee filter mirror.

In our design, the horizontal shift in camera centers be-

tween faces is very small (≈ 2mm) reducing the blind spot
to within 5 cm of the device. The shift decreases along
the height of the mirror. To simplify the post-capture setup,
we avoided depth computation step at dewarping. This re-
tains a small disparity difference from the top of the image
to its bottom. This difference is visible on close inspection
and may be corrected with depth estimation. As our goal
was human perceivable stereo and the artifacts are imper-
ceivable while using stereo glasses or HMD, we avoided
this step. There are radial and vertical shifts in camera cen-
8. Conclusions
We have proposed a simple practical solution to captur-
ing 360◦ stereo panoramas using a single digital camera for
immersive human experience. As the resolution of sensors
increase, the quality of the panoramas also increase. We de-
rived the optimal parameters of the design and experimental
results show that we can avoid most visual artifacts in the
panoramas. While designed with human consumption in
mind, the stereo pairs could also be used for depth estima-
tion.

3762
References [19] Y. Pritch, M. Ben-Ezra, and S. Peleg. Optics for omnistereo
imaging. In Foundations of Image Understanding, pages
[1] Google Jump, https://www.google.com/get/cardboard/jump/. 447–467. 2001. 2
1, 2, 3
[20] C. Richardt, Y. Pritch, H. Zimmer, and A. Sorkine-Hornung.
[2] Google Cardboard, https://www.google.com/get/cardboard/.
Megastereo: Constructing high-resolution stereo panoramas.
3
In Computer Vision and Pattern Recognition, IEEE Confer-
[3] http://www.ignorancia.org/en/index.php?page=Childhood. ence on, 2013. 2, 3
7, 8
[21] P. Sturm, S. Ramalingam, and S. Lodha. On calibration,
[4] POVRay, http://www.povray.org/. 7
structure from motion and multi-view geometry for generic
[5] A. S. Amini, M. Varshosaz, and M. Saadatseresht. Evalu- camera models. In Imaging Beyond the Pinhole Camera.
ating a new stereo panorama system based on stereo cam- 2006. 7
eras. International Journal of Scientific Research in Inven-
[22] K. Tanaka and S. Tachi. Tornado: Omnistereo video imaging
tions and New Ideas, 2, 2014. 2
with rotating optics. Visualization and Computer Graphics,
[6] V. Chapdelaine-Couture and S. Roy. The omnipolar cam-
IEEE Transactions on, 11(6):614–625, 2005. 3
era: A new approach to stereo immersive capture. In Com-
[23] S. Yi and N. Ahuja. An omnidirectional stereo vision system
putational Photography, IEEE International Conference on,
using a single camera. In Pattern Recognition. Eighteenth
2013. 3
International Conference on, 2006. 2
[7] T. L. Conroy and J. B. Moore. Resolution invariant surfaces
for panoramic vision systems. In Computer Vision, 1999.
The Proceedings of the Seventh IEEE International Confer-
ence on, volume 1, pages 392–397. IEEE, 1999. 5
[8] V. Couture, M. S. Langer, and S. Roy. Panoramic stereo
video textures. In Computer Vision, IEEE International Con-
ference on, 2011. 2, 3
[9] V. C. Couture, M. S. Langer, and S. Roy. Omnistereo video
textures without ghosting. In 3D Vision, International Con-
ference on, 2013. 2, 3
[10] F. Devernay and S. Pujades. Focus mismatch detection in
stereoscopic content. In IS&T/SPIE Electronic Imaging. In-
ternational Society for Optics and Photonics, 2012. 7
[11] S. Gächter, T. Pajdla, and B. Micusik. Mirror design for an
omnidirectional camera with a space variant imager. In IEEE
Workshop on Omnidirectional Vision Applied to Robotic Ori-
entation and Nondestructive Testing, pages 99–105, 2001. 5
[12] J. Gluckman, S. K. Nayar, and K. J. Thoresz. Real-time om-
nidirectional and panoramic stereo. In Proceedings of Image
Understanding Workshop, 1998. 2
[13] R. T. Held and M. S. Banks. Misperceptions in stereoscopic
displays: a vision science perspective. In Proceedings of
the 5th symposium on Applied perception in graphics and
visualization, 2008. 2
[14] R. A. Hicks and R. K. Perline. Equi-areal catadioptric sen-
sors. In null, page 13. IEEE, 2002. 5
[15] T. Kawanishi, K. Yamazawa, H. Iwasa, H. Takemura, and
N. Yokoya. Generation of high-resolution stereo panoramic
images by omnidirectional imaging sensor using hexagonal
pyramidal mirrors. In Pattern Recognition. Fourteenth Inter-
national Conference on, 1998. 2, 3
[16] S. Lin and R. Bajcsy. High resolution catadioptric omni-
directional stereo sensor for robot vision. In Robotics and
Automation. IEEE International Conference on, 2003. 2
[17] S. Peleg, M. Ben-Ezra, and Y. Pritch. Omnistereo:
Panoramic stereo imaging. Pattern Analysis and Machine
Intelligence, IEEE Transactions on, 23:279–290, 2001. 1, 2,
3, 4
[18] J. Posdamer and M. Altschuler. Surface measurement by
space-encoded projected beam systems. Computer graphics
and image processing, 18:1–17, 1982. 7

3763