Modeling of Clouds from

a Single Photograph
Yoshinori Dobashi (Hokkaido University)
Yusuke Shinzo (Hokkaido University)
Tsuyoshi Yamamoto (Hokkaido University)

Overview
Introduction
Related

Work
Proposed Method
Results
Conclusion

Introduction
Realistic

Display of Clouds

Synthesizing images of outdoor scenes

Realistic density distribution
Long research history
Modeling

of clouds

Procedural approach
Physically-based approach
outdoor scene

Introduction
Procedural

approach

Use of simple heuristically-defined rules

Low computational cost
Difficult to adjust parameters
Physically-based

approach

Simulating physical process of cloud formation

Realistic clouds
High computational cost

Our Approach
Image-based

approach

Use of a single photograph

Not to reconstruct the same clouds
Using the photo as a guide to synthesize
similar clouds
Three types of clouds

cirrus

altocumulus

cumulus

Overview
Introduction
Related

Work
Proposed Method
Results
Conclusion

Related Work

Procedural modeling

Fractals [Vos83]
Textured ellipsoids [Gar85]
Metaballs + noise function [Ebe97]
Spectral synthesis [Sak93]
Real-time modeling/rendering system [SSEH03]

Difficult to adjust parameters

[Gar85]

[Ebe97]

[SSEH03]

Related Work

Physically-based modeling
Numerical solution of atmospheric fluid dynamics
[KH84, MYDN01, MYDN02]
Controlling cloud simulation [DKNY08]

High computational cost

[KH84]

[MYDN01]

[MYDN02]

[DKNY08]

Related Work
Image-based

modeling

Use of infrared satellite images [DNYO98]

Not applicable to photo taken from the ground

No cloud types

satellite image

synthesized clouds

Overview
Introduction
Related

Work
Proposed Method
Results
Conclusion

Overview of Our Method

cirrus
altocumulus
cumulus

Overview of Our Method

cirrus
altocumulus
cumulus
Calculation of cloud image
(common process)

Calculation of Cloud Image

Overview

cirrus

intensity

opacity

altocumulus

intensity

opacity

cumulus

intensity

opacity

(input photographs)

(cloud images)

Calculation of Cloud Image

Processes

1. Removing cloud pixels

input image

2. Interpolating sky color

sky image

3. calculation of intensity/opacity

intensity

opacity

Calculation of Cloud Image

Processes

1. Removing cloud pixels

input image

2. Interpolating sky color

sky image

3. calculation of intensity/opacity

intensity

opacity

Calculation of Cloud Image

Processes

1. Removing cloud pixels

input image

2. Interpolating sky color

sky image

3. calculation of intensity/opacity

intensity

opacity

Calculation of Cloud Image

Processes

1. Removing cloud pixels

input image

2. Interpolating sky color

sky image

3. calculation of intensity/opacity

intensity

opacity

= cloud
image

Removing Cloud Pixels

Use

of chroma to identify cloud pixels

Clouds are generally white (or gray)

Remove pixel if chroma < threshold

Input photograph

removed cloud pixels

Calculation of Sky Image

Interpolation

of sky colors by solving

Poisson equation:

I sky ( pc , ) 0

pc: cloud pixel

= R, G, B

cloud pixels pc

sky image

Calculation of Cloud Image

Intensity

of clouds (single scattering)

sun
sky

viewpoint

cloud

Calculation of Cloud Image

Intensity

of clouds (single scattering)

Isun

viewpoint

Isun

sun
sky

cloud

Calculation of Cloud Image

Intensity

of clouds (single scattering)

Isun

viewpoint

Isun
Isky

sun
sky

cloud

Isky

Calculation of Cloud Image

Intensity

of clouds (single scattering)

Isun

viewpoint

Icld

Isun
Isky

sun
sky

cloud

Isky

I cld ( p, ) ( p) I sun ( ) ( p) I sky ( p, )

(p: pixel, = R, G, B)

Calculation of Cloud Image

cloud intensity () & opacity
() by minimizing:

Computing

store these in cloud image

I ( p, ) ( ( p ) I

sun

( ) ( p ) I sky ( p, )) min

unknowns
input image

sky image

Overview of Our Method

cirrus
Modeling of cirrus
altocumulus
cumulus

Modeling of Cirrus
Cirrus

clouds

Thin and no self-shadows

Two-dimensional texture

input photograph

Modeling of Cirrus

Use of cloud image as 2D texture

Removing effect of perspective transformation by
specifying cloud plane
input image
cloud image
cloud plane
cirrus cloud texture

Overview of Our Method

cirrus
altocumulus
Modeling of altocumulus
cumulus

Modeling of Altocumulus
Altocumulus

Thin but with self-shadows are observed

Using Metaballs to define three-dimensional
density distribution
metaball
metaball
position
radius
center
density

input photo

field
function

Generating Metaballs
Converting

cloud image into binary image

cloud image

binary image

Generating Metaballs
Distance

transform of binary image

Distance from a white pixel to the nearest

black pixel

binary image

distance image

Generating Metaballs
Generating

2D metaballs at white pixels

Use distance for radius of metaball

binary image

Generating Metaballs
Generating

2D metaballs at white pixels

Use distance for radius of metaball

distance
binary image
image

Optimizing Metaball Density

Minimizing

difference between cumulative

density and cloud intensity
center density
N

p 1

field function [Wyvill90]

l 1

( p ) ql f (rlp )

cloud image

min

cumulative density at pixel p

Optimizing Metaball Density

Example

cloud image

cumulative density image

Specifying Cloud Plane

Interactive

specification

Orientation of cloud plane and viewing angle

cloud plane

Projecting Metaballs
Projecting metaball center onto cloud plane
Scaling metaball radius in proportion to distance
from viewpoint

cloud plane

2D metaballs
viewpoint

Computing Density Distribution

Calculating

bounding box of all metaballs

Subdividing bounding box into grid
Computing density at each grid point

bounding box

density distribution

Overview of Our Method

cirrus
altocumulus
cumulus
cumulus

Modeling of Cumulus
Cumulus

Generating surface shape

Calculating densities inside surface shape

cloud image

surface shape

3D density
distribution

Computing Surface Shape

Converting

cloud image into binary image

cloud image

binary image

Computing Surface Shape

Distance

transform of binary image

Extracting medial axes
Pixels where distances are local maxima
medial axis

binary image

distance image

Computing Surface Shape

Use

distance at medial axis as thickness

of clouds
Propagate thickness by optimization
medial axis

cloud image

thickness image
colorization by optimization [Levin 04]

Computing Surface Shape

Constructing

surface shape

Assuming symmetric shape with respect to

image plane

front view

side view

Computing Density Distribution

Calculating

bounding box of surface

shape
Subdividing bounding box into grid
Calculating density at each grid point

Overview
Introduction
Related

Work
Proposed Method
Results
Conclusion

Results
Computer

CPU: Intel Corei7 (3.33 GHz)

Main memory: 4GB
GPU: NVIDIA GeForce GTX 295
Computation

time

within 10 seconds

Results
Cirrus

input image

cloud texture

input image

cloud texture

Results
Altocumulus

input

input

synthesized
synthesized

Results
Cumulus

input

input

synthesized

synthesized

Results
Modeling

process

Results
A

cloud scene

Conclusion
Modeling

of clouds from a photograph

Methods for three types of clouds

(cirrus, altocumulus, cumulus)
Generating realistic 3D clouds similar to the
input photograph
Future

Work

Improving shapes of cumulus clouds

 END