UNIVERSITÀ DEGLI STUDI DI ROMA LA SAPIENZA
FACOLTÀ DI INGEGNERIA
Tesi di Laurea Specialistica in
INGEGNERIA INFORMATICA
Algoritmi per l’illuminazione interattiva di
materiali traslucenti deformabili su GPU
Relatore
Prof. Marco Schaerf
Candidato
Giacomo De Martino
Correlatore
Ing. Marco Fratarcangeli
Anno accademico 2005/2006
Sintesi
Introduzione
L’interazione della luce con i materiali nel mondo reale è un processo molto
complesso da interpretare, a causa di vari fattori: la duplice natura della luce (onda e
particella), le diverse fonti di luce e le interazioni tra i vari oggetti. Affinché Il
disegno di una scena in computer grafica risulti realistico, non può tenere conto solo
di alcuni fattori e trascurarne altri. Molti modelli matematici sono stati creati per
riprodurre digitalmente vari tipi di materiali. Mentre nella computer grafica
destinata agli effetti digitali per i film si possono usare algoritmi che impiegano
minuti per disegnare un fotogramma completo, per altre applicazioni, come
videogiochi o guide interattive virtuali, servono tecniche molto veloci, capaci di
reagire ai cambiamenti imposti dall’esterno sulla scena, a tempo di esecuzione. Negli
ultimi anni il divario qualitativo delle immagini prodotte negli effetti digitali di post
produzione rispetto alla grafica interattiva si è assottigliato, grazie anche alla
evoluzione esponenziale della potenza di calcolo e alla programmabilità delle unità
di processamento grafiche (GPU) .
Illuminazione in computer grafica
La scienza che studia e misura la luce è la Radiometria. Alcune grandezze
radiometriche fondamentali (vedi Fig. 4) sono la Radianza e L’Irraggiamento: la
prima è la quantità di luce che illumina un punto di superficie mentre l’altra è la
quantità di luce in un singolo raggio. L’irraggiamento dipende dalla Radianza
contenuta in un infinitesimo di angolo solido centrato nella normale della superficie
modulato per un fattore di esposizione verso la sorgente di luce. Integrando i vari
contributi in tutte le direzioni si ottiene l’irraggiamento totale.
L’illuminazione di ogni punto in una scena dipende dal trasporto di luce, cioè
dall’interazione della luce con i vari oggetti. Un approssimazione di questo
fenomeno è l’equazione di rendering che rappresenta la quantità di luce che lascia
un punto di una superficie derivante da rapporti di equilibrio tra emissioni e
riflessioni (vedi Eq. 3). Il termine più importante nell’equazione è la funzione di
i
distribuzione di diffusione bidirezionale (BSDF). Materiali differenti hanno diversa
BSDF: ad esempio i materiali metallici tendono a riflettere la luce in specifiche
direzioni, mentre quelli diffusi spargono la luce in tutte le direzioni (vedi Fig. 10,11).
Più è complessa questa funzione e più è difficile riprodurre certi materiali.
Diffusione sottosuperficiale
I materiali più difficili da riprodurre in computer grafica sono quelli
traslucenti come la cera, le foglie, il latte, il marmo e la pelle.
Quando un raggio di luce colpisce l’interfaccia di un materiale traslucente,
non viene completamente riflesso, una grande percentuale penetra nell’oggetto e
colpisce le particelle di cui è composto. Ad ogni interazione il fascio di luce perde
energia e cambia direzione di propagazione e lunghezza d’onda (quindi colore).
Queste interazioni diffondono la luce all’interno dell’oggetto facendogli perdere la
direzionalità. Parte della luce riemerge poi in un altro punto della superficie
dell’oggetto a seconda dei valori di assorbimento e dispersione caratteristici del
materiale.
Per simulare corpi traslucenti bisognerebbe calcolare il trasporto della luce
attraverso il volume dell’oggetto, cosa computazionalmente molto costosa.
Recentemente è stato proposto un modello analitico per il calcolo della
diffusione sottosuperficiale da Jensen [42][46], che individua due componenti:
singola dispersione e dispersione multipla, la prima può essere calcolata
esattamente (vedi Eq. 28), mentre per la seconda si utilizza un approssimazione con
dipolo (vedi Eq. 25,26). Il calcolo è suddiviso in due fasi: nella prima si valutano gli
integrali su un insieme di punti campioni dell’oggetto, mentre poi per ogni punto
dell’ immagine si valuta una funzione di illuminazione di due sorgenti luce una sopra
e l’altra sotto lo strato confine del materiale (dipolo). Oggi questa tecnica è diventata
uno
standard
per
le
produzioni
digitali
fotorealistiche.
Recentemente
l’approssimazione dipolo è stata sfruttata nel rendering in tempo reale grazie all’uso
delle mappe di profondità. Di solito queste mappe servono per disegnare le ombre in
una scena; poiché la dispersione della luce dipende da quanto penetra nel corpo
illuminato, si è pensato di usarle per calcolare lo spessore degli oggetti. Ad ogni
fotogramma da disegnare su schermo si calcolano dal punto di vista della luce: la
mappa di profondità dell’oggetto, le normali della superficie e la “trasmissività”
ii
secondo la legge di Fresnel (vedi Eq. 44) dell’oggetto. Nella seconda parte
dell’algoritmo dal punto di vista dell’osservatore, l’illuminazione di un punto
dipende dalla quantità di luce trasmessa dai punti adiacenti sulla superficie in
direzione della sorgente di luce. L’approssimazione con dipolo è un modello
analitico che necessita solo di coefficienti caratteristici del materiale per essere
valutato, facili da collezionare per ogni materiale.
Materiali traslucenti eterogenei e la pelle umana
Alcuni materiali traslucenti non sono omogenei, bensì composti da strati, ad
esempio le foglie o la pelle umana. Le foglie sono suddivise in tre strati ed esibiscono
maggiormente caratteristiche di singola dispersione. Per simulare l’interazione con
luce si utilizzano varie mappe di colore e maschere, una per ogni strato (vedi Fig.
26). Ad ogni fotogramma viene calcolato il colore, lo spessore, la normale e il raggio
rifratto nel punto da illuminare; poi usando l’equazione (vedi Eq. 41) si ottiene
l’attenuazione esponenziale della luce. La pelle invece è suddivisa in: epidermide e
derma, ricoperti di una patina oleosa, ognuno con differente densità, indice di
rifrazione e colore.
Attraverso vari studi sulla fenomenologia della pelle [44], sono stati misurati
dei valori utili per implementare tecniche alternative rispetto alla approssimazione
dipolo singolo e multiplo, più adatte ad applicazioni real-time, e direttamente
applicabili nella produzione artistica della prossima generazione di videogiochi. Le
caratteristiche osservate della pelle sono: la maggior parte della luce che colpisce la
pelle entra negli strati inferiori e subisce processi di diffusione multipla riemergendo
in un'altra posizione rispetto a quella di ingresso. L’ingresso e l’uscita dipendono
dalla microstruttura dell’interfaccia oleosa che separa la pelle dall’aria, il colore
dipende dalla differente concentrazione di melanina ed emoglobina negli strati
inferiori. Dal punto di vista artistico più sono dettagliate le mappe da applicare ai
modelli tridimensionali e meglio vengono modellate le imperfezioni cutanee
conferendo maggior realismo. Viene prima calcolato l’irraggiamento sub-superficiale
tramite l’uso di mappe di colore, di perturbazioni delle normali della geometria e
funzioni di illuminazione con un rendering nello spazio della tessitura. Vengono
aggiunte le ombre della sorgente di luce principale e il contributo di illuminazione
ambiente modulato da una mappa di occlusione ambientale. Con il risultato di
iii
questo passo si creano varie versioni sempre più sfocate dell’irraggiamento subsuperficiale per formare una gerarchia di texture a rappresentare gli strati della
pelle. Usando una combinazione lineare con vari pesi per ogni canale di colore
dell’irraggiamento, si ottiene l’effetto di morbidezza della pelle e la riemersione di
piccole quantità di luce in zone ombreggiate. Per la specularità è meglio usare una
funzione di illuminazione con caratteristiche di conservazione dell’energia e
reciprocità per avere maggiore riflessione nelle zone del viso con angoli di incidenza
tangenziali rispetto alla sorgente di luce. E’ importante applicare una correzione di
gamma allo spazio di colori utilizzato; questa tecnica presa in prestito dal modo degli
effetti digitali permette di correggere il difetto dei dispositivi di visualizzazione non
lineari che smorzano le intensità di colore riducendone il relismo.
Risultati e direzioni future
Nella fase di preparazione della tesi la maggior parte delle tecniche elencate
in questa esposizione sono state implementate in OpenGL tramite Shader, cercando
di unificare i punti di forza.
Le tecniche di approssimazione in tempo reale che usano mappe di
profondità soffrono del problema di campionamento per le basse risoluzioni delle
mappe intermedie utilizzate durante il rendering, generando qualche artefatto
visivo.
Il rendering della pelle umana è sicuramente più affascinante, e grazie a
schede video più performanti si potrebbe anche aggiungere il calcolor dell’occlusione
ambientale e la luce indiretta senza precomputazioni. Il modello di illuminazione
della pelle è buono per ambienti chiusi e poco illuminati poichè manca dell’effetto di
penetrazione della luce nelle zone sottili della pelle come le orecchie. Dal punto di
vista qualitativo è possibile aggiungere effetti di illuminazione ad alto range
dinamico (HDR) e image based lighting utilizzando armoniche sferiche; queste
tecniche complementari aumentano il realismo di ogni scena da disegnare.
L’implementazione del rendering della pelle umana è stata testata su un calcolatore
di fascia medio-alta ed eseguita a circa 30 fotogrammi al secondo senza particolari
ottimizzazioni. Gli algoritmi si sono dimostrati scalabili sia per risoluzione delle
mappe e dei buffer intermedi, che per complessità di operazioni matematiche
eseguite ad ogni fotogramma.
iv
Sempre di più vedremo nelle applicazioni interattive un amento nella varietà
dei materiali riprodotti, a differenza di qualche anno fa quando avevano tutti un
aspetto “plasticoso”.
v
Interactive Shading Algorithms of
Deformable Translucent Material on GPU
Master Thesis
Author:
Giacomo De Martino
Table of Contents
1 Introduction
5
1.1 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 GPU Evolution Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 The OpenGL Graphics Hardware Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Shading Theory Background
13
2.1 The Rendering Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Bidirectional Scattering Distribution Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2.1 Overview of the BxDF Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3 The BRDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3 Perceiving Translucent Materials
25
3.1 Light Scattering in Participating Media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 Subsurface Scattering Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.2.1 Single vs Multi-Scattering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.3 Interactive Subsurface Scattering Image-Space Evaluation . . . . . . . . . . . . . . . . . . . . 35
3.3.1 Fast Absorption Only Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4 Multi-Layered Translucent Material
47
4.1 Rendering Plant Leaves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.2 Leaves Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.2.1 Epidermis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.2.2 Mesophyll . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.2.3 Veins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.3 Visual Representation Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4.4 Human Skin Rendering Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.5 Anatomy of Human Skin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53
4.5.1 The Epidermis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.5.2 The Dermis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.5.3 The Hypoderms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56
4.6 Travel of Light Through Skin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.7 Asperity Scattering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57
4.8 The Kubelka-Munk Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2
4.8.1 Why Not Use Compositing? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4.8.2 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . .61
4.8.3 Reflection and Transmission from a Single Layer . . . . . . . . . . . . . . . . . . . . . . 63
4.8.4 Composition of Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64
4.9 Multiple Dipole for Light Diffusion in Multi-Layered Translucent Material . . . . . . 65
5 Skin Rendering Using Texture-Space Diffusion Layer Compositing
68
5.1 Overview of the Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.2 Implementation Algorithm Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.2.1 Diffuse Component . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.2.2 Specular Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.2.3 Gamma Correction . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 85
6 Conclusions and Future Work
87
6.1 Results Gallery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
References
92
3
Acknowledgments
Thanks to Prof. Marco Scharef who gave me chance doing computer graphics thesis.
Thanks to Ing. Marco Fratarcangeli, who helped me with hints, suggesting papers
and dedicating time.
Thanks to Andrew Hwang for providing me great 2D/3D assets [14].
Thanks to remedy for gDEBugger license, which helped me to find nasty bugs in the
code.
Thanks to Gianluca Pepe for his testing and critiques during development.
4
Chapter 1
Introduction
In computer graphics, rendering of a scene, that is drawing some objects on a
2D screen, can be done off-line or on-line. Off-line rendering involves frame by
frame pre-computation taking order of magnitude of minutes (even hours)
depending on shading complexity. On-line rendering can be accomplished either
interactively or in real-time: both techniques enable external interaction at run-time,
which incorporates a change of the camera, change of the environment and/or
change of the lighting situation; real-time techniques can be applied in videogames
using current generation hardware, these applications demand at least 60 frames
per second as an absolute minimum, while interactive ones on the other side need a
frame rate typically situated in the range of 10-20 frames per second and can be
used for 3D presentations or e-learning. Actually lighting simulation by quantum
mechanics laws is unthinkable for off-line rendering, and to greater reason for
interactive rates, so often in computer graphics we achieve eye pleasing results using
some approximations. When we talk about deformable objects, we are shrinking
interactivity requirements, in facts many interactive algorithms take advantage of
long pre computation before run-time. Pre computed values have strictly geometry
correlation in the scene being rendered. Objects deformation induces changes in
rendering parameters, which need to be updated for each frame interval. The GPU
acronym stands for Graphics Processing Unit, i.e. video card processor. In the
techniques being examined in this work much rendering load is accounted by GPU,
both for memory storage and computation cycles. The advantage instead of
rendering on CPU is due to higher tasks parallelization and
degree of
programmability of modern GPU hardware. This is achieved by specialized
programs called Shaders: little programs written in specific programming languages,
compiled for running directly on the GPU. Translucent materials are the ones that
permit light penetrate them but not completely pass through (more later); for
instance glass is a transparent material because we can see things on the other side,
while grape is translucent. So this thesis will dive into several cutting-edge
5
techniques aimed to render interactively all kind of translucent material such as
marble, wax, skin and leaves.
1.1 Thesis Structure
The following chapters are structured as follows:
•
Chapter 1 introduces the topics covered in this thesis: we start explaining the
title, then a brief overview on graphics processing unit from early days to nowadays,
emphasizing the overcome in processing power of GPU against CPU. We talk about
key stages of OpenGL 3d graphics pipeline.
•
Chapter 2 contains relevant theory background about digital illumination,
introducing radiometry and fundamental quantities, and the problem of rendering.
Then we take account classic way to approximate light radiation transport in a scene.
•
Chapter 3 introduces the phenomenological appearance of translucent
material and their difference against matte objects. Then we introduce a generalized
analytical solution to model subsurface scattering and the most adopted
implementation technique in production pipeline. We present a set of image-space
techniques about interactive rendering of translucent material.
•
Chapter 4 extends the subject to more complicated multi-layered translucent
material such as skin and plant leaves. We give an overview about leaves and skin
anatomy, introducing early rendering techniques.
•
Chapter 5 presents full explanation of real-time skin shading technique
suited for virtual character animation, with advantages, motivations and caveats.
•
Chapter 6 underlines results achieved in this thesis and future work
directions. Finally we present a set of screenshots taken from developed demos.
6
1.2 GPU Evolution Overview
[3] CPUs are, by design, general purpose. CPUs execute applications (for
example, word processors and accounting packages) written in general-purpose
languages, such as C++ or Java. Because of the GPU’s specialized design, it is much
faster at graphics tasks, such as rendering 3D scenes, than a general-purpose CPU
would be. New GPUs process tens of millions of vertices per second and rasterize
hundreds of millions or even billions of fragments per second. This is
overwhelmingly faster than the rate at which a CPU could process a similar number
of vertices and fragments. However, the GPU cannot execute the same arbitrary,
general-purpose programs that a CPU can.
Prior to the introduction of GPUs, companies such as Silicon Graphics (SGI)
and Evans & Sutherland designed specialized and expensive graphics hardware. The
graphics systems developed by these companies introduced many of the concepts,
such as vertex transformation and texture mapping. These systems were so
expensive that they did not achieve the mass-market. Today, GPUs are far more
powerful and much cheaper than any prior systems.
The First-Generation GPUs (up to 1998) are capable of rasterizing pretransformed triangles and applying one or two textures. They also implement the
DirectX 6 feature set. these GPUs completely relieve the CPU from updating
individual pixels. They lack the ability to transform vertices of 3D objects; instead,
vertex transformations occur in the CPU, and they have a quite limited set of math
operations on hand.
The Second-Generation GPUs (2000) offload 3D vertex transformation and
lighting (T&L) from the CPU. Although the set of math operations expanded in this
generation, the possibilities are still limited: this generation is more configurable,
but still not truly programmable.
The Third-Generation GPUs (2002) provides vertex programmability rather
than merely offering more configurability, they let the application specify a sequence
of instructions for processing vertices. Considerably more pixel-level configurability
is available. This generation is Transitional (DirectX 8.x).
The Fourth-Generation GPUs (2003) provide both vertex-level and pixellevel programmability. This level of programmability opens up the possibility of
offloading complex vertex transformation and pixel-shading operations from the
7
CPU to the GPU. Due to high computational power (see Fig. 1 and 2), general
purpose computation becomes feasible at faster rates then with CPU system. The
DirectX 9 feature set brought conditional statement and dynamic branching.
• GPU Observed GFLOPS
• CPU Theoretical peak GFLOPS
Figure 1: Processing power comparison; note the exponential evolution in
processing power for GPU in last years. (source nVidia)
Recently with the introduction of DirectX 10 Api [12], graphics pipeline
became very different from the traditional one, as the Shader portion in now
completely unified. Instead of an instruction making its way through the vertex
shader and then moving on to the pixel shader, instructions are recognised in the
Input Assembler which then forward the instruction onto the shade. A new degree of
programmability was added by Geometry shaders.
Most important features set are true geometry instancing and textures array,
predicted draw expanding on early Z to further reduce redundant overdraw, new
HDR formats, increased number of temporary and constant registers, integer
instruction set, all belonging to Shader Model 4.0. specification. See Fig. 4 for a
Shader Model features summary.
8
2 core
GPU
CPU
30
200
GFLOPS
GFLOPS
CPU to
CPU to
GPU to
GPU
system
graphics
memory
memory
8 GB/s
30 GB/s
1 GB/s
Figure 2: current chip, and interconnection computational power values expressed
in GigaFlops, (billions of floating point instructions per seconds) for a medium level
system.
1.3 The OpenGL Graphics Hardware Pipeline [11]
A pipeline is a sequence of stages operating in parallel and in a fixed order.
Each stage receives its input from the prior stage and sends its output to the
subsequent stage. Here we examine the situation in recent OpenGL pipeline (see Fig.
3).
The 3D application sends the GPU a sequence of vertices batched into
geometric primitives: typically polygons, lines, and points. there are many ways to
specify geometric primitives, via immediate-mode commands or through vertex
arrays. Every vertex has a position but also usually has several other attributes such
as a color, one or multiple texture coordinate sets, and a normal vector. The normal
vector indicates what direction the surface faces at the vertex, and is typically used in
lighting calculations.
Vertex transformation is the first processing stage in the graphics hardware
pipeline.
Vertex transformation performs a sequence of math operations on each
vertex. These operations include transforming the vertex position into a screen
position for use by the rasterizer, generating texture coordinates for texturing, and
lighting the vertex to determine its color.
The transformed vertices flow in sequence to the next stage, called primitive
assembly and rasterization. After vertex transformation, vertices are assembled into
primitives, according to the topology (e.g., TRIANGLES, QUAD_STRIP) provided by
9
the call to glBegin(). Primitives are points, lines, triangles, quads, or polygons, but
instead decompose them into triangles as permitted by the spec. After initial
primitive assembly, a geometry shader is executed on each individual point, line, or
triangle primitive, if one is active. It can read the attributes of each transformed
vertex, perform arbitrary computations on entire primitives as input, along with
being able to generate new or modify existing primitives and output them without
the need for CPU intervention- the latter I known as data amplification. In much
the same way that the Geometry Shader can output more vertex data than it
originally received, it can also output less vertex data than it received, too; this is
known as data minimization.
These emitted vertices are themselves assembled into primitives according to
the output primitive type of the geometry shader.
These primitives may require clipping to the view frustum (the view’s visible
region of 3D space), as well as any enabled application-specified clip planes. The
rasterizer may also discard polygons based on whether they face forward or
backward.
The term pixel is short for “picture element.” A pixel represents the contents
of the frame buffer at a specific location, such as the color, depth, and any other
values associated with that location. A fragment is the state required potentially to
update a particular pixel.
This process is known as culling. Polygons that survive these clipping and
culling steps must be rasterized. Rasterization is the process of determining the set
of pixels covered by a geometric primitive. Polygons, lines, and points are each
rasterized according to the rules specified for each type of primitive. The results of
rasterization are a set of pixel locations as well as a set of fragments.
There is no relationship between the number of vertices a primitive has and
the number of fragments that are generated when it is rasterized. For example, a
triangle made up of just three vertices could take up the entire screen, and therefore
generate millions of fragments!
The term “fragment” is used because rasterization breaks up each geometric
primitive, such as a triangle, into pixel-sized fragments for each pixel that the
primitive covers. A fragment has an associated pixel location, a depth value, and a
set of interpolated parameters such as a color, a secondary (specular) color, and one
or more texture coordinate sets.
10
Conventional vertex attributes
Generic vertex
Fixed function vertex processing
Vertex shader
begin/end state
Position, color, other vertex data
Primitive assembly
Geometry shader
Primitive assembly
Output primitive type
Color clamping
clipping
Flat shading
Perspective divide
Final color processing
Viewport transformation
Facing determination
Two-sided coloring
rasterization
textures
fragment shader
framebuffer
Figure 3: Overview of recent changes in OpenGL rendering pipeline [11].
These various interpolated parameters are derived from the transformed
vertices that make up the particular geometric primitive used to generate the
fragments. You can think of a fragment as a “potential pixel.” If a fragment passes
the various rasterization tests (in the raster operations stage), the fragment updates
a pixel in the frame buffer.
11
SM 1.1
SM 2.0
SM 3.0
SM 4.0
(2001)
(2002)
(2004)*
(2006)
128
256
≥512
4+8**
32+64**
≥512
constant
≥96
≥256
≥256
registers
8
32
224
12
12
32
2
12
32
16
16
16
16
4+2***
8+2***
10
32
render targets
1
4
4
8
samplers
8
16
16
16
Feature
instruction
slots
tmp registers
input registers
textures
4
8
16
2D tex size
16
2Kx2K
≥64K
16x4096
4096
128
8Kx8K
integer ops
9
load op
9
sample offsets
9
transcendental
ops
9
9
9
9
9
derivative op
flow control
9
static
stat/dyn
stat/dyn
9
9
dynamic
Figure 4: Shader model feature comparison summary, *specification released in
2002, hardware in 2004; **texture load + arithmetic instructions; **texture + color
registers; dashed line separates vertex shader (above) from pixel shader (below).
12
Chapter 2
Shading Theory Background
[1] One of the primary goals in graphics is generating compelling images of
real objects. There are three big challenges that must be overcome to achieve
compelling images. First, real objects exist in general lighting environments (area
lighting). Second, the light from the environment can be shadowed and reflected by
the objects in the scene, e.g. global illumination. Finally, the surface material
properties in the scene all have to be represented and modelled. Addressing one of
these challenges to the exclusion of others will not result in realistic images. For
example, extremely realistic reflectance properties on an object lit by a single
directional light with no shadows or interreflections will not look realistic.
Computer graphics is fundamentally about light and its interaction with
objects. Light is electromagnetic radiant energy (we concern ourselves with the
visible part of the spectrum -wavelengths from 380 to 780 nanometers). We will
stick strictly to geometric optics, which treats light as linear rays (as opposed to
physical optics, which treats light as waves, or quantum optics, which treats light as
dual waves/particles). This works well for most scenes, in which the scale of objects
is much larger than light wavelengths.
The physical measurement and study of light is radiometry. The fundamental
quantity in radiometry is power (energy over time), which is radiant flux ( Φ ),
measured in Watts. For example, the light pouring through a window (in all
directions) would be measured as radiant flux. Radiosity ( B ) is the light exitant
(going out of) a single point (in all directions), and is measured in Watts per square
meter. Radiance ( L ) is the light in a single ray, and is measured in Watts per square
meter per steradian. Irradiance ( E ) can be seen as the opposite of radiosity – it is
the light going into and illuminating a surface point. It includes light hitting that
point from all incident (incoming) directions. It is also measured in Watts per
square meter.
13
Radiant Flux
Φ
Radiosity
B
Radiance
Irradiance
L
E
Figure 4: Radiometric quantities [1].
Note that although the radiosity and irradiance diagrams in figure 4 show a
cone of directions for clarity, they are measured over a full hemisphere (centered on
the surface normal).
An important relationship between radiance and irradiance is:
E = ∫ Li cos θi dωi
(1)
dE = Li cosθi dωi
(2)
Ω
Ω
N
dωi
θi
Figure 5: Radiance and Irradiance relationship [1].
14
ωi
This means that if we take a tiny patch of incident directions with solid angle
dωi (small enough to be represented by a single incident direction ωi without loss of
accuracy), the radiance incident from ωi , times dωi , times the cosine of the angle
θi between ωi and the normal N gives us the contribution of that patch to the
irradiance. If we integrate this over the hemisphere Ω (centered on the normal), we
get the total irradiance. The cosine is there because radiance is defined relative to an
area perpendicular to the ray, and irradiance is defined relative to an area parallel to
the surface. Another way of looking at it is that the same irradiance, coming in at a
more oblique angle, contributes a smaller amount to the irradiance because it is
‘spread out’ more. Note that although we talk about radiance incident from ωi , the
direction actually points outwards.
2.1 The Rendering Equation
[13] In computer graphics, the rendering equation describes the flow of light
energy throughout a scene. Based on the physics of light, it provides theoretically
perfect results under the physical assumption that only the particle phenomena of
light are present, in contrast to the various rendering techniques, which
approximate this ideal.
The physical basis for the rendering equation is the law of conservation of
energy. Assuming that L denotes radiance, we have that at each particular position
and direction, the outgoing light ( Lo ) is the sum of the emitted light ( Le ) and the
reflected light. The reflected light itself is the sum of the incoming light ( Li ) from all
directions, multiplied by the surface reflection and cosine of the incident angle.
15
Figure 6: A visual representation of rendering equation.
Mathematically, the equation is expressed as follows:
( )
( )
(
) ( )(
)
Lo x, ω = Le x, ω + ∫ fr x, ω ′, ω Li x, ω ′ ω ′ ⋅ n d ω ′
Ω
(3)
Broken down:
( )
L (x, ω )
Lo x , ω
e
∫ ...d ω′
is light outward at a particular position x and in direction ω .
is light emitted from the same position and direction.
is an infinitesimal sum over a hemisphere of inward directions.
Ω
(
fr x, ω ′, ω
)
is the proportion of light reflected at the position (from
inward direction to outward direction). This function is known as the
( )
(ω′ ⋅ n)
L i x, ω ′
BRDF (more on this later).
is light inward from the position and direction ω ′ .
is the attenuation of inward light due to incident angle.
Two noteworthy features are: its linearity — it is composed only of
multiplications and additions, and its spatial homogeneity — it is the same in all
16
positions and orientations. These mean a wide range of factorings and arrangements
are (easily) possible for solutions.
The rendering equation is the key academic/theoretical concept in rendering.
It serves as the most abstract formal expression of the non-perceptual aspect of
rendering. By connecting outward light to inward light, via an interaction point, this
equation stands for the whole 'light transport' in a scene. All more complete
algorithms can be seen as solutions to particular formulations of this equation.
2.2 Bidirectional Scattering Distribution Function
[13] One of the main mathematical tools for describing surface materials is
called the BSDF (Bidirectional scattering distribution function). A BSDF models how
a material reflects light, and though it has several simplifying assumptions, it can
represent a broad class of materials.
However in practice this phenomenon is usually split into the reflected and
transmitted components, which are then treated separately as BRDF (Bidirectional
reflectance
distribution
function)
and
BTDF
(Bidirectional
transmittance
distribution function).
For example, wood is a material, representable by a BRDF, in which
reflectance is not homogenous since it varies over the surface due to pigment, grain
and possibly the amount of varnish. Additionally, varnished wood exhibits strong
reflection of the environment at grazing angles and almost none at incident angles,
i.e. looking at the surface head on. Some materials like plastic are isotropic, meaning
that there is no visible grain on the surface. Other materials having a visible grain,
such as hair and brushed metal, are termed anisotropic.
17
Figure 7: BSDF: BRDF + BTDF [13].
BSDF is a superset and the generalization of the BRDF and BTDF. The
concept behind all BxDF functions could be described as a black box with the inputs
being any two angles, one for incoming (incident) ray and the second one for the
outgoing (reflected or transmitted) ray at a given point of the surface. The output of
this black box is the value defining the ratio between the incoming and the outgoing
light energy for the given couple of angles. The content of the black box may be a
mathematical formula which more or less accurately tries to model and approximate
the actual surface behaviour or an algorithm which produces the output based on
discrete samples of measured data. This implies that the function is 4 (+1)
dimensional (4 values for 2 3D angles + 1 optional for wave length of the light),
which means that it cannot be simply represented by 2D and not even by a 3D
graph. Each 2D or 3D graph, sometimes seen in the literature, shows only a slice of
the function.
Some tend to use the term BSDF simply as a category name covering the
whole family of BxDF functions.
The term BSDF is sometimes used in a slightly different context, for the
function describing the amount of the scatter (not scattered light), scatter being
simply a function of the incident light angle. An example to illustrate this context:
for perfectly lambertian surface the BSDF(angle)=const (see Fig. 11). This approach
18
is used for instance to verify the output quality by the manufacturers of the glossy
surfaces.
2.2.1 Overview of the BxDF Functions
-BSDF (Bidirectional scattering distribution function) is the most general
function.
-BSSRDF (Bidirectional surface scattering reflectance distribution function)
describes the relation between outgoing radiance and the incident flux, including the
phenomena like subsurface scattering (SSS).
-BRDF (Bidirectional reflectance distribution function) is a simplified
BSSRDF, assuming that light enters and leaves at the same point (see Fig. 8).
-BTDF (Bidirectional transmittance distribution function) is similar to BRDF
but for the opposite side of the surface. (see Fig. 7).
Figure 8: BRDF vs. BSSRDF [13].
19
2.3 The BRDF
A BRDF (Bidirectional Reflectance Distribution Function) is a function which
relates incident irradiance to exitant reflected radiance. More precisely:
fr (ωi, ωe ) =
dLe(ωe )
dE (ωi )
(4)
Where ωi is the direction to the incident irradiance, and ωe is the direction
to the exitant reflected radiance.
N
ωe
θi
ωi
dωi
Figure 9: solid angle and BRDF [1].
Thus for every pair of directions (incident and exitant), the BRDF gives us
the ratio between incident irradiance and exitant radiance. Since the incident
direction and the excitant direction are both 2D quantities (a common
parameterization is to use two angles: elevation θ relative to the surface normal and
rotation φ about the normal), the BRDF is a 4D function. For most types of surface
materials, the BRDF does not vary if the incident and exitant directions are rotated
around the normal (while keeping the relative rotation between them). This reduces
the number of variables from 4 to 3. BRDFs with this property are called isotropic
BRDFs, and those without it are anisotropic BRDFs (two examples are brushed
metal and hair).
If we take this equation and combine it with the previous relation between
radiance and irradiance, we get:
20
fr (ωi,ωe ) =
dLe(ωe )
Li (ωi ) cos θi dωi
(5)
Which gives us:
dLe (ωe ) = fr (ωi,ωe ) L(ωi ) cos θi dωi
(6)
Or
Le (ωe ) =
∫ f (ω ω ) L (ω ) cos θ dω
r
i,
e
i
i
i
i
(7)
Ω
This means if we want to know the contribution of light from a tiny patch of
incident directions (characterized by dωi and ωi ) to the reflected light exitant in a
direction ωe , we take the value of the BRDF at ωi and ωe , times the incident
radiance from ωi , times dωi , times cos θi . If we integrate this over the hemisphere
(centered on the normal), we get the total exitant radiance in direction ωe . This is
the reflection equation.
In real-time rendering, a simplified model of incident radiance is used
(environment maps allow complex incident radiance, but they are a special case). In
this model the incident radiance is equal to an ambient constant LiA from all
directions, except for a small number of directions in which we have directional or
point lights. Those lights are characterized by their irradiance contribution E (if they
were physical lights, this would be equal to their solid angle times their radiance).
Since radiance is constant but the solid angle varies as the square of the distance
between the light source and the surface point it illuminates, this causes the
irradiance contribution to follow an inverse square law. This incident radiance
model enables converting the integral into a sum over the light sources, which
simplifies the reflection equation:
Le(ωe ) =
∑ f (ω , ω ) E cos θ + L ∫ f
r
il
e
l
l
i
iA
Ω
21
⎛
r⎜
⎝
ωi, ωe ⎞⎟⎠ cos θi dωi
(8)
This can also be written
Le(ωe ) =
∑ f (ω , ω ) E cos θ + L
r
il
e
l
i
l
iA
RA(ωe )
(9)
Usually, for the purpose of simplicity, we ignore the dependency of RA (the
ambient reflectance) on ωr and assume that it is constant. This may introduce
significant errors in many cases, but does simplify the equation further:
( )
(
)
Le ωe = ∑ f r ωil , ωe El cos θ i + LiA RA
l
(10)
Not every arbitrary 4D (or 3D for isotropic BRDFs) function can be a BRDF.
To be physically plausible, a BRDF has to follow certain rules:
Reciprocity: fr (ωi, ωe ) = fr (ωe, ωi )
(11)
This means that the surface reflects light the same when the directions are
reversed, and is a basic law of physics (Helmholtz reciprocity). It is also required for
bidirectional ray tracing algorithms to work, since they implicitly depend on this
property.
Energy Conservation: ∀ωe :
∫ f (ω , ω ) cos θ dω
r
i
e
i
i
≤1
Ω
If the BRDF does not obey this condition, then there will be more reflected
light energy than incident light energy which is a violation of conservation of energy.
It is also required for many global illumination algorithms to converge.
Note that the BRDF in this form cannot model certain classes of physical
phenomena, such as effects relating to light entering the surface at one point and
leaving at a different point (subsurface scattering / translucency), light which enters
the surface from one side and is emitted out the opposite side (transmittance). Note
22
(12)
that the BRDF usually is dependent on wavelength, but it cannot model effects in
which the light changes wavelength (fluorescence). Finally, due to energy
conservation it cannot model surfaces which emit light on their own
(phosphorescence).
Most real objects are not made out of one homogeneous pure material with
no scratches or blemishes. Most surfaces have BRDFs which are shift-variant (which
vary from point to point on the surface). This adds an additional two dimensions to
the BRDF which now depends on a surface parameterization as well.
BRDF data can be measured from real surfaces, which is useful in comparing
or fitting BRDF models. The device most commonly used for this is a
gonioreflectometer, which measures the reflectance of a surface under multiple
incident and exitant directions. Usually only one point on the surface can be
measured, so shift-variant BRDFs cannot be captured with this device. There are
large collections of BRDF data measured from various surfaces which are available
online.
Later we will describe a BRDF model which has energy conservation and
reciprocity.
To quantify the amount of light reflected by a surface we can use the ratio of
reflected to incident flux. This quantity is known as the reflectance, ρ of the surface.
Each surface exibits different reflection profiles (see Fig. 10).
(a)
(b)
Figure 10: a specular surface (a) perfect specular reflection(mirror), (b) glossy
specular.
23
(a)
(b)
Figure 11: a diffuse material (a) Lambertian (ideal) diffuse, (b) general diffuse
reflection.
24
Chapter 3
Perceiving Translucent Materials
[4] Many materials that we commonly encounter are translucent, including leaves
and fruit flesh; textiles and papers; various stones, such as agate or marble; soaps;
wax; some types of glass; milk and human skin. Many image cues allow us to tell
that a surface is translucent rather than opaque. When light strikes a translucent
material, it enters the body of the object, scatters, and re-emerges from the surface.
The light that bleeds through translucent objects gives them a characteristic softness
and glow. However, although light is visible through translucent objects, form is
generally not.
Figure 9: a collection of real world translucent material (source Google).
25
A recent research about
some of the factors that influence perceived
translucency using a combination of psychophysics and image analysis brought to
these results [4].
Highlights occur when light is specularly reflected from the surface of an
object. Note that specular reflections are caused by the interface between two
materials of different refractive index. many translucent materials that we
commonly encounter are somewhat glossy (e.g. plastic, wax or marmalade). This
means that the human visual system may ‘expect’ translucent materials to exhibit
specular reflections. Interestingly, we have found that highlights can contribute to
the visual impression of translucency.
Interestingly, hue, saturation and intensity can all vary as a function of the
distance travelled by a ray through a translucent material.
We know that colour is not necessary for the perception of translucency,
because a black-and-white photograph can nevertheless yield a vivid impression of
translucency.
When illuminated translucent objects become ‘filled’ with light. An important
consequence of this is that points on the surface that do not receive any direct
illumination (i.e. they are in shadow) can nevertheless receive light from within the
body of the object.
In translucent materials light scatters below the surface of the object. This
phenomenon, called subsurface scattering, causes light to spread beneath the
surface and re-emerge in a region around the point of illumination. In contrast,
traditional models for light scattering based on the Bidirectional Reflectance
Distribution Function (BRDF) assume that materials are opaque, and that all light is
reflected from the point that is illuminated.
A full simulation of translucency requires solving the radiative transport
equation, Jensen et al. [42] simplify the light scattering by assuming that the
translucent material is homogeneous. In this case the scattering of light can be
approximated by a diffusion equation as well as a term for single scattering. These
terms form a Bidirectional Scattering Surface Reflectance Distribution Function
(BSSRDF).
The parameters in the BSSRDF are the refractive index of the material, the
phase function (Jensen et al. [42] use the Henyey-Greenstein phase function), and
the scattering and absorption coefficients. The absorption and scattering coefficients
26
specify the probability that a photon will be absorbed or scattered when travelling a
given distance within the material. Changes in translucency can influence apparent
softness, realism and even how edible an object looks.
3.1 Light Scattering in Participating Media
[14] Natural phenomena like fog, fire, water, beams of light through clouds
and subsurface scattering can’t really be described through “surface” models. To
perform a full simulation of these volumetric effects it is necessary to solve equations
of participating media.
However, in case of subsurface scattering quite many
simplifications can be made. When a photon enters a participating media it can
either continue unaffected through the medium or it can interact at a given location.
When a photon interacts with a medium one of two things can happen: photon gets
either scattered or absorbed. Scattering means that light heading in one direction is
scattered to other directions due to collisions with particles, while absorption
provokes the reduction of radiance due to conversion of light to another form of
energy (e.g. heat). The probability of a photon being either scattered or absorbed as
it moves through the medium is given by the scattering coefficient, σs , and the
absorption coefficient σa .
The combined loss in radiance L in the direction ω due to both outscattering and absorption is:
(ω ⋅ ∇)L(x, ω ) = −σ (x ) L(x, ω )
(13)
σt = σs + σa
(14)
t
where
As we move through the media there will also be a gain in radiance due to inscattering of light. This is given by following equation:
(ω ⋅ ∇) L(x, ω ) = σ (x ) ∫ p(x, ω′, ω ) L
s
Ω 4π
27
⎛
i ⎜⎜
⎝
⎞
x, ω ′ ⎟⎟ d ω ′
⎠
(15)
where the incident radiance Li is integrated over all directions of the sphere
(
)
Ω 4π . p x, ω ′, ω is a material property called phase function: when modelling
scattering within the material, we can use phase functions to describe the result of
light interacting with particles in the layer. In general, each particle will have a
different scattering profile. This name is misleading since the scattering profile has
no connection with the phase of the incident light waves and would be more
appropriately called a scattering function. A phase function describes the scattered
distribution of light after a ray hits a particle in the layer.
(
)
Phase functions are in general four-dimensional functions p ωi, ωo of the
two directions. However, with the exception of media like ice, that have an oriented
structure to them, particles in most media are randomly oriented. Under this
situation, the phase function can be reduced to a function of the phase angle
between the two directions:
(
p (θ ) = p ωi ⋅ ωi
)
(16)
We will use the Henyey-Greenstein phase function, which is a commonlyused parameterized phase function. It takes an asymmetry parameter g called
mean cosine of scattering direction, that ranges from -1 to 1, which spans the range
of strong retro-reflection to strong forward scattering (see Eq. 16 and Fig. 10).
g (x ) =
∫ p(x, ω′, ω )cos θ d ω′
Ω 4π
28
(17)
Figure 10: different asymmetry parameter g for Henyey-Greenstein phase function,
backscattering(left), isotropic scattering (center), forward scattering (right).
The Henyey-Greenstein phase function is:
p(cos θ ) =
(1 + g
1 − g2
2
− 2 g cos θ
)
32
To compute overall scattering from a layer with known phase function and
attenuation and scattering coefficients, it’s in general necessary to simulate all of the
multiple scattering within the layer.
The choice of a single scattering phase function is a compromise between
realism and mathematical tractability. Figure 11 illustrates Henyey-Greenstein phase
functions p(θ ) . In case of isotropic scattering the photon is scattered in random
direction without a history where it came from. For isotropic scattering the phase
function is constant and looks like figure 11 (a).
Figure 11: The Henyey-Greenstein phase function (a) g=0, (b) g=0.9.[14]
29
(18)
There also can be a gain in radiance due to emission Le from the medium,
e.g. because of flames:
(ω ⋅ ∇ )L(x, ω ) = −σ L (x, ω )
a
e
(19)
By combining equations [13],[15] and [19] it is possible to find the total
change in radiance per unit distance. After integrating the both sides of combined
equations for a segment of length s the commonly known volume rendering
equation can be formed:
( )
L x, ω =
∫
S
e
−τ x , x + s ω
0
∫
S
0
e −τ ( x , x′ )σa ( x ′)Le ⎛⎜⎝ x ′ ⎞⎟⎠ dx ′ +
e −τ ( x , x′ )σs (x ′)
(
∫πp(x′, ω ′, ω )L (x′, ω ′)d ω ′dx′ +
i
(20)
Ω4
)L(x + sω , ω )
Where the optical depth τ (x, x′) is given by:
x′
τ (x, x′) = ∫ σt ⎛⎜⎝ t ⎞⎟⎠dt
(21)
x
3.2 Subsurface Scattering Solution
The most advanced and accurate, yet computationally feasible model for
calculating subsurface scattering is the BSSRDF method introduced by Jensen
[6][42][46]. This method relies on the observation that the scattering of light is
formed of two components: single and multiple scattering. Physically single
scattering is interpreted as illumination from a single scattering event and multiple
scattering as illumination due to integer number of scattering events. The solution
for single scattering will be exact while multiple scattering is solved with a dipole
30
point source approximation. For example, human skin’s multiple scattering term is
very high, whereas its single scattering term is very small. A substance such as
marble or wax has a much larger contribution from the single scattering effect, and
consequently less contribution from the diffusion (multiple) component. When a
laser is shot through a leaf there is a strong focused beam of light that enters and
then exits in a direct manner. This shows the single scattering of light as it passes
through the material. We will see later the complete structure of leaves.
The formulation assumes a homogeneous participating medium, the
properties of which are characterized by the absorption coefficient σa , the scattering
(
)
coefficient σs and the phase function p ωi , ωo . The extinction coefficient is
σt = σa + σs .
3.2.1 Single Vs Multi-scattering
(
)
The BSSRDF, S , relates the outgoing radiance, Lo xo, ωo at the point xo in
( )
direction ωo , to the incident flux Φi xi, ωi at the point xi from direction ωi :
(
) (
) ( )
(22)
dLo xo, ωo = S xi, ωi; xo, ωo dΦi xi, ωi
Given a BSSRDF, the outgoing radiance is computed by integrating the
incident radiance over incoming directions and area, A :
(
) ∫ ∫ S (x , ω ; x , ω )L(x , ω ) (n ⋅ ω )d ω dA(x )
Lo xo, ωo =
i
i
o
o
i
i
i
i
i
i
(23)
A 2π
where Lo is the outgoing radiance at point xo in direction ωo , L is the
incident radiance at point xi in direction ωi , and S is the BSSRDF. Jensen et al.
[42] define the BSSRDF as the sum of a single scattering term S (1) and a diffuse
multiple scattering term Sd :
(
)
(
) (
S xi, ωi; xo, ωo = S (1) xi, ωi; xo, ωo + Sd xi, ωi; xo, ωo
31
)
(24)
Using a dipole source approximation, they derive Sd as:
(
)
Sd xi, ωi; xo, ωo =
1
π
( ) (
) (
Ft η , ωi Rd xi − xo Ft η , ωo
)
(25)
where Ft is the Fresnel transmittance, η is the relative index of refraction,
and Rd is the diffuse reflectance computed by:
Rd =
α′
4π
⎡ ⎛⎜
⎛
1 ⎞⎟ e −σtrdr
1 ⎞⎟ e −σtrdv ⎤
⎜
⎟
+
v ⎜σtr +
z
⎢ zr ⎜⎜σtr + ⎟⎟
⎥
⎜
dv ⎟⎠ d v2 ⎥
dr ⎠ d r2
⎝
⎢⎣ ⎝
⎦
(26)
Where:
σ ′s = (1 − g )σs and σ ′t = σa + σ ′s are reduced scattering and extinction
coefficients, α ′ = σ ′s σ ′t is the reduced albedo,
g is the mean cosine of the scattering angle,
σtr = 3σaσ ′t is the effective extinction coefficient,
dr = r 2 + z 2 r and dv =
r 2 + z 2 v are the distances from illumination point
xi to the dipole source,
r = xo − xi is the distance between xi and xo ,
zr = 1 σ ′t and zv = (1 + 4 A 3) are the distances from xo to the dipole source.
Here A = (1 + Fdr ) (1 − Fdr ) and Fdr is a diffuse Fresnel term approximated
by
Fdr = − 1.440 η 2 + 0.710 η + 0.668 + 0.636
The diffusion approximation requires an expensive integration of light
transport from all points on the surface (see Fig. 12). Jensen et al. [42] introduced a
two-pass approach to accelerate the integration process remarkably. The key idea is
to decouple the computation of incident illumination from the evaluation of
32
(27)
BSSRDF, making it possible to reuse illumination samples. Specifically, in the first
pass they compute irradiance values for uniformly sampled points on the surface.
Figure 12: Dipole source approximation for multiple scattering [47]
As we said single scattering can be interpreted as illumination from a single
scattering event. Mathematically this means the first order solution to volume
rendering equation. Single scattering occurs only when the refracted incoming and
outgoing rays intersect as shown in Fig. 13. According to Jensen [42][46] the total
outgoing radiance L 0 (1) due to single scattering is computed as an integral over path
length s along the refracted outgoing ray:
Lo
(1)
(x , ω ) = σ ∫ ∫ Fp(ω , ω )e−σ (s
∞
o
o
i′
s
2π 0
o
t
i′
+ s)
L xi, ωi ds d ωi
(
)
Where:
ω i′
and ωo′ are the refracted incoming and outgoing directions,
( ) ( )
F = Ft η , ωi ⋅ Ft η , ωo is the combined Fresnel transmittance,
s′i and s are the scattering path lengths along ω ′i and ω ′o ,
and p is a normalized phase function.
33
(28)
When sampling the illumination, it is difficult to estimate s′i accurately since
that requires finding the point of refraction xi for arbitrary geometry. In practice, if
we assume that the surface at xi is locally flat and illumination is distant, a good
approximation of s′i can be found [42] [46] by:
s′i = si
ωi ⋅ ni
⎛1⎞
1 − ⎜⎜ ⎟⎟
⎝η ⎠
2
(29)
⎛1 − ωi ⋅ ni 2 ⎞
⎜
⎟
⎝
⎠
where si is the observed path length as if the incident ray is not refracted.
The single scattering component (Eq. 28) is derived from previous work by
Hanrahan and Krueger [41] and is computed by integration along ω ′o .
(a)
(b)
Figure 13: Single Scattering optics, (a) real; (b) approximation
34
(a)
(b)
Figure 14: Rendering example of translucent objects using dipole approximation(a)
standard BRDF, (b) BSSRDF model. [42]
3.3 Interactive Subsurface Scattering Image-Space
Evaluation
As stated in [9], the following approaches are based on the fact that
subsurface scattering although a global effect, is largely a local one due to its
exponential falloff, which limits the volume it can affect, therefore even though the
light does not necessarily exit an object at the same point where it enters, as
required by a BRDF model, it will for all practical purposes exit within a short
distance of its entry point. In Jansen et al. [42][46] they showed that most part of
translucent materials can be almost entirely simulated by only considering multiple
scattering.
The main rationale behind a possible combination of local illumination
model with sss effects is based on the phenomenological observation that the effects
are well localized. First the scattering within one object will have very little effect on
the appearance of another object; the influence between different objects can be well
described by the reflectance values of their surfaces only. Second, even within the
same object, the sss will have very little effect on another surface point if the
35
distance between the two points is large. This property is a result of the exponential
falloff of light intensity due to absorption and scattering within the material.
e − σs r
α′ ⎡
e− σs v ⎤
⎥
⎢ zr (1 + σsr )
Rd ( xi, xo ) =
+ zv (1 + σsv )
sr 3
4π ⎢
sv 3 ⎥
⎦
⎣
′
zr = 1 σt zv = zr + 4 AD
sr = xr − xo , with xr = xi − zr ⋅ Ni
sv = xv − xo , with xv = xi − zv ⋅ Ni
A=
1 + Fdr
1 − Fdr
Fdr = −
1.440
η
2
+
0.710
η
+ 0.668 + 0.0636η
σ = 3σaσ ′t
D = 1 3σ ′t ,
σt = σa + σs ′ , α ′ = σ ′s σ ′t
σ ′ = reduced scattering coefficient (given)
σa = absorption coefficient (given)
η = relative refraction index (given)
Ft (η , ω ) = Fresnel transmittance factor
Figure 15: Recap of quantities and equations describing the BSSRDF
diffusion term [10]
We derived the subsurface scattering reflectance function using Jensen’s
method, skipping the single scattering term:
(
)
S xi, ωi; xo, ωo ≈
1
π
( )
( )
Ft η , ωo Rd ( xi; xo ) Ft η , ωi
(30)
Where Rd is evaluated as reassumed in Figure 15.
Then substituting Eq. 28 in shade equation Eq. 21, for a given surface point
we get:
36
L( xo, ωo ) =
1
π
Ft (η , ωo ) B(xo )
(31)
B( xo ) = ∫ E (xi ) Rd ( xi, xo ) dxi
(32)
S
E ( xi ) =
∫ L(x , ω ) F (η , ω )(N ⋅ ω ) d ω
i
i
i
i
i
i
(33)
Ω
This means the evaluation of subsurface scattering in highly scattering
material can be decomposed into three phases [10].
In the first stage, the radiances from lights are collected over the surface; the
light incident at a surface point xi is scattered into the material according to the
Fresnel term Ft . For an irradiance impulse I ( xin ) from a point or parallel light
source this is simply:
(
)
( )
E ( xin ) = Ft η , ωin N (xin ) ⋅ ωin I ωin
(34)
Where η is the optical density of the material. The Fresnel term Ft can be
well approximated as proposed by Schlick [2] (see later).
An evolution [5] mimics most kinds of light sources by integrating the
contributions from point lights, directional lights and the ambient lights. This
procedure is called the gathering of irradiance.
Second, light diffuses through the material. This diffusion process is
approximated by the diffuse sub-surface reflectance function Rd ( xin, xout ) with
xin, xout ∈ S , which can be compared with the geometric term in Radiosity. Rd
describes the transport of incident light at xin through the object to xout . This 4Dfunction does not only depend on xout − xin , but also on the angle between xout − xin
and the surface normal in xin :
B( xout ) = ∫ E ( xin )Rd ( xin, xout )dxin
S
37
(35)
Note that the evaluation of the integral B( xo ) over the entire object’s surface
is the most expensive part of sss evaluation, the solution proposed was either
perform Montecarlo integration using randomly distributed samples or using a
spatial hierarchy. An even more efficient techniques is using importance sampling in
image-space as implemented by Mertens et al. [21].
In this integral it is assumed that the path from xin to xout is completely
within the object, which is in general the case only for convex objects. So errors
occur for concave objects, however, in practice this kind of error is often visually not
important. Finally the light leaves the object, again weighted by Ft :
L(xout , xin ) =
1
π
Ft (η , ωout )B( xout )
(36)
The simulation path is also depicted in Fig. 16.
Figure 16: Computing translucency by integration.[10]
A series of papers like [10], [5] and [24] with its evolution [40] exploit depth
buffer to compute distance light travels inside objects and then evaluate dipole
source approximation. Depth map is a floating point texture containing distance of
nearest objects’ pixels from light point of view, and is commonly used by modern
renderer for generating shadows. These techniques separates simulation is in two
stages (see Fig. 17).
38
Figure 17: [10], First stage compute irradiance samples of light surface(left), second
stage compute radiance leaving the object by filtering these irradiance samples
(right).
The first phase (see Eq. 34) is computed during the generation of a collection
of intermediate maps sometimes called Translucent Shadow Maps (TSMs) [10] or
Diffusion Source Maps (DSMs) [5]. Additionally to depth, this set of maps stores
irradiance E ( xin ) and the surface normal N ( xin ) with every pixel (see Fig. 18). The
surface normal is required later to compute Rd . Note, that E ( xin ) is wavelength
dependent, thus we store a red, green and blue color component which allows us to
use arbitrary textured surfaces. We can also store coordinates for each pixel avoiding
matrices transformation for later Rd calculation [figure 16].
Figure 18: from left to right: irradiance map, depth map, light space position map,
normal map, final rendering result.
Besides point light, the irradiance from ambient lights are determined by
using sphere harmonic method, In a pre-processing stage, we compute the basis and
39
coefficients of the sphere harmonic functions and record them as a cube map. To
avoid spherical harmonic knowledge, ambient cube map can be pre-computed as a
180° cosine angle filter kernel with edge fix-up enabled using utility like CubeMap
Gen from ATI [56]. The visual differences between the two methods is negligible for
real-time rendering.
Figure 19: (left) original light probe, (center) spherical harmonic, (right) cosine filter
Having this information, the integral in Eq. 35 can be computed during
rendering of the user's view as a filter with weight given by Rd . This is denoted as
the scattering operation [5], where the color of each visible surface point depends on
a variable number of lightened source points. This makes the implementation with
current graphics hardware nontrivial because the rendering pipeline is designed as
amenable for gathering operations, where the destination of each pixel is defined
before rasterization. With an attempt to convert the scattering operations into a
gathering procedure in GPUs, one may implement it at the fragment level. This
scheme requires lots of texture lookups at the complexity of the image resolution.
This inefficiency can be overcome with hierarchical sampling scheme.
The filter weights heavily depend on Δz , i.e. the distance light travelled
trough the object. For large Δz the weights are small and decrease only slowly with
(Δx, Δz ) ,
i.e. kernel horizontal and vertical spread. If Δz is small, the central
weights are large and decrease quickly. Generally, the filter must be large enough to
cover all regions where Rd is significantly large . for natural materials this can be in
the order of centimeters.
In [10] they use a filter based on a mip-mapping of generated set of maps;
the filter sample pattern covers 21 sample positions, shown in Fig. 20a, sampling
density decreases with the distance to the origin. The outer samples represent a
40
larger filter area, thus their irradiance, depth and normal values area are taken at
coarser mip-map levels.
Alternatively (see Fig. 20b) for each handled surface location, in [5] they
sample twelve points (shown in red color) on its near neighbourhood uniformly, and
eight points (shown with green and blue colors) for moderately far regions; the black
point denotes the underlying surface location.
The patterns reflect the fact that Rd decreases rapidly with the sample
distance. The pattern is based on heuristics; as shown above, the filter value
distribution heavily depends on Δz so there is no globally optimal pattern.
Pattern A
Pattern B
Figure 20: different type of filter kernel [10].
For speeding up computation time, avoiding Rd computation, given a (u, v ) texture
coordinate for current shaded pixel a 7 x 7 texels filter kernel can be chosen, where
the neighborhood of
(u, v )T serves
as input. The incoming radiance at the
neighboring texels only contributes to the outgoing radiance, if the texel's depth
values lies within a certain proximity to d. If the stored depth value at (u , v ) is
T
41
significantly smaller than d, then the currently processed surface point is shadowed
by the surface part represented by this texel. All contributing texels are summed up
using constant, precomputed weights, ignoring depth difference. In the rendering
passes for the local response we also calculate the local illumination depending on
the Fresnel term for reflected light.
Layer 0
0
depth
Layer 1
1
0
depth
Layer 2
1
0
depth
1
Figure 21: Depth peeling strips away depth layers with each successive pass. The
frames above show the frontmost (leftmost) surfaces as bold black lines, hidden
surfaces as thin black lines, and “peeled away” surfaces as light grey lines[16].
All this work is done by a vertex and a fragment program. The number of
required rendering passes depends on the deployed graphics hardware.
Note that if the object is concave we calculate an approximated thickness,
because the information on occlusions are not saved (Figure 21). This can lead to
errors and ghost like effects during the visualization of the object. So we solved this
problem using depth of peeling [16] which is an image space technique that allows to
get for each pixel the second nearest fragment, third nearest, and so on. In this case
we need only the second depth value to check if the point that we are shading
receives or not directly the scattering term. So only one pass of the algorithm is
performed [24][40].
42
Depth map
a
c
b
Figure 22: calculation of thickness: for point b, the thickness is correctly evaluated,
however for point c if we calculate thickness respect light we get errors because the
point is occluded by b. For point like c we do not calculate subsurface scattering in
order to avoid ghosting effects [40].
Another recent paper [42] accounts the computation of B integral (Eq. 35)
in the dipole source approximation using second order Taylor series. After some
elaborate mathematical manipulation, which are omitted here, they simplify Eq. 35
to a form suited for real-time evaluation:
B( xo ) ≈ E ( xo )F 0 +
(
)
1 2
∇ E ( xo ) F 2
2
(37)
Where ∇ 2 E ( xo ) is the Laplacian of E at xo , and the F factors are materialspecific constants, which are calculated as:
∞
F 0 = 2π ∫ Rd (r )rdr =
α ′ ⎛⎜
0
1+ e
2 ⎜⎝
4
− A 3(1−α ′ )
2
⎞ −
⎟e
⎟
⎠
3(1−α ′ )
(38)
and
F2 =
2π
∞
∞
α′
∫ cos θ cos θdθ ∫ R (r )r dr = π ∫ R (r )r dr = 4 [R
3
d
0
0
d
0
43
3
v
+ Rr ]
(39)
With
⎡z
z*3σtr ⎤ − a
*
2 + a* −
R* = ⎢
⎥e * ,
σ
tr
a
* ⎥
⎦
⎣⎢
(
)
a = σtr z
*
*
And * can be either v or r.
Eq. 38 is similar to the inverse of a Laplacian sharpening filter applied to the
irradiance; this is not a coincidence as this blurring effect of the irradiance is one of
the main visual properties of the multiple scattering term.
Outgoing radiance is calculated as the sum of three terms; i) multiple
scattering, ii) single scattering, iii) Fresnel term. The computation is divided in three
rendering passes:
1) render the irradiance image of the object as seen from the observer, in an
Irradiace buffer with the same screen resolution.
2) render the object’s depth image as seen from the key point light source
into a depth buffer, like for the shadow map algorithms.
3) Render the final image summing up the three terms listed above.
In this stage we calculate irradiance Laplacian, using finite differences on
discrete textel grid of the irradiance map generated in 1st pass, using following
5x5 filter kernel:
0
0
−1
h2
0
0
0
−1
h2
−2
h2
−1
h2
0
−1
h2
−2
h2
+ 16
h2
−2
h2
−1
h2
0
−1
h2
−2
h2
−1
h2
0
0
0
−1
h2
0
0
44
Where h denotes mesh size, which a good value is ~2 times mean free paths,
1
σtr
where σtr is the effective extinction coefficient of the medium.
Note that we are not actually sampling irradiance on the surface of the object,
but rather from the object as seen from the camera. The main drawback of this
implementation is that multiple scattering effect does not “wrap around” the object.
A better alternative would be to sample in texture space (see later). We can also use
different kernel size to tailor “performance vs visual quality” ratio. The single
scattering term is evaluated as in Jansen [42] [46] see figure 13: we need the
distance from point(s) {p} on the refracted eye-ray (which is going 'inside' the
material) back to the corresponding point(s) on the surface that is the intersection of
a line from a light-source to {p}. As an approximation of that we use the appropriate
depth from a depth- map D (i.e. a Williams' shadow depth map). This D is computed
in the second pass.
All these techniques suffer from aliasing problems as standard shadow map
because of discreet nature of depth buffer.
Figure 23: Translucent material rendering example from [38].
45
Fast Absorption Only Approximation
3.3.1
[39] For very translucent material, light absorption is one of the most
important factor. To simulate light absorption through homogenous object we need
object thickness and attenuation coefficient σa . If light travels a total distance x
between entering and exiting the medium, the fraction of it that was attenuated is
1− e
−σ a x
i.e. after travelling some distance x, e
−σ a x
of the light remains. The total
distance x through the layer that the ray passes through is z cos θ , where θ is the
angle of the ray with the surface normal N and z is object thickness. If the layer has
uniform attenuation coefficient σa , the fraction of light that exits the bottom of the
layer is:
e
−σ a z
cos θ
(40)
Using depth map from light point of view we have the distance from the light
to the closest surface points where light entered the surface. Then during standard
rendering we evaluate the distance from light to the exit point. Subtracting these
values we do not obtain object thickness but an approximation of the distance light
travelled inside object (see figure 24).
Depth map
Light
do
di
Eye
Object
Figure 24: di is the closest surface point seen from light view, do is the
exitant point of the light ray.
46
Chapter 4
Multi-Layered Translucent Material
In the following chapters we will examine multi-layered translucent material
as plant leaves and skin.
4.1 Rendering Plant Leaves
[27] Rendering large natural outdoor has been always a big challenge for
real-time computer graphics. A large grass and trees landscape would require
million polygon to render correctly, but actually we don’t need hundred of thousand
of “unique” plants and trees, we only need different color and level of detail to fake
human eye.
Recently this problem has been overcome thanks to geometry instancing API
feature. Geometry instancing simply use 1 draw call for rendering many copies of the
same object, saving resources overhead.
Plants are also very complex in their optical behaviour. Many studies were
conducted about leaves’ optics parameter.
4.2 Leaves Structure
[13] A leaf is usually modelled by four layers with different optical properties.
47
Figure 24: Sezione foglia [13]
A structurally complete leaf of an angiosperm consists of a petiole (leaf
stem), a lamina (leaf blade), and stipules (small processes located to either side of
the base of the petiole). The point at which the petiole attaches to the stem is called
the leaf axil. Not every species produces leaves with all of these structural parts. In
some species, paired stipules are not obvious or are absent altogether. A petiole may
be absent, or the blade may not be laminar (flattened). Leaf structure (anatomy)
varies from species to species, in the follow we describe a common generic structure.
A leaf is considered to be a plant organ, typically consisting of the following tissues:
1.
An epidermis that covers the upper and lower surfaces
2.
An interior chlorenchyma called the mesophyll
3.
An arrangement of veins (the vascular tissue).
4.2.1 Epidermis
The epidermis is the outer multi-layered group of cells covering the leaf. It
forms the boundary between the plant and the external world. The epidermis serves
several functions: protection against water loss, regulation of gas exchange,
secretion of metabolic compounds, and (in some species) absorption of water. Most
leaves show dorsoventral anatomy: the upper (adaxial) and lower (abaxial) surfaces
have somewhat different construction and may serve different functions.
48
The epidermis is usually transparent (epidermal cells lack chloroplasts) and
coated on the outer side with a waxy cuticle that prevents water loss. The cuticle may
be thinner on the lower epidermis than on the upper epidermis, and is thicker on
leaves from dry climates as compared with those from wet climates.
The epidermis is covered with pores called stomata, part of a stoma complex
consisting of a pore surrounded on each side by chloroplast-containing guard cells,
and two to four subsidiary cells that lack chloroplasts. The stoma complex regulates
the exchange of gases and water vapor between the outside air and the interior of the
leaf. Typically, the stomata are more numerous over the abaxial (lower) epidermis
than the adaxial (upper) epidermis.
4.2.2 Mesophyll
Most of the interior of the leaf between the upper and lower layers of
epidermis is a parenchyma (ground tissue) or chlorenchyma tissue called the
mesophyll (Greek for "middle leaf"). This assimilation tissue is the primary
location of photosynthesis in the plant. The products of photosynthesis are called
"assimilates".
In ferns and most flowering plants the mesophyll is divided into two layers:
•
An upper palisade layer of tightly packed, vertically elongated cells,
one to two cells thick, directly beneath the adaxial epidermis. Its cells contain many
more chloroplasts than the spongy layer. These long cylindrical cells are regularly
arranged in one to five rows. Cylindrical cells, with the chloroplasts close to the walls
of the cell, can take optimal advantage of light. The slight separation of the cells
provides maximum absorption of carbon dioxide. This separation must be minimal
to afford capillary action for water distribution. In order to adapt to their different
environment (such as sun or shade), plants had to adapt this structure to obtain
optimal result. Sun leaves have a multi-layered palisade layer, while shade leaves or
older leaves closer to the soil, are single-layered.
49
•
Beneath the palisade layer is the spongy layer. The cells of the
spongy layer are more rounded and not so tightly packed. There are large
intercellular air spaces. The light passes through these intercellular air spaces cause
internal scattering of the incident radiation and distribute light within the leaf. This
is important for an optimal absorption of the light by the leaf’s mesophyll.
These two different layers of the mesophyll are absent in many aquatic and
marsh plants. Even an epidermis and a mesophyll may be lacking. Instead for their
gaseous exchanges they use a homogeneous aerenchyma (thin-walled cells
separated by large gas-filled spaces). Their stomata are situated at the upper surface.
Each of these layers cause reflection, refraction and scattering.
Leaves are normally green in color, which comes from chlorophyll found in
plastids in the chlorenchyma cells. Plants that lack chlorophyll cannot
photosynthesize. In cold autumns they sometimes change color, and turn yellow,
bright orange or red as various accessory pigments (carotenoids and anthocyanins)
are revealed when the tree responds to cold and reduced sunlight by curtailing
chlorophyll production.
4.2.3 Veins
The veins are the vascular tissue of the leaf and are located in the spongy
layer of the mesophyll. They are typical examples of pattern formation through
ramification. The pattern of the veins is called venation.
The veins are made up of:
•
xylem, which brings water from the roots into the leaf.
•
phloem, which usually moves sap out, the latter containing the
glucose produced by photosynthesis in the leaf.
External leaf characteristics (such as shape, margin, hairs, etc.) are important for
identifying plant species, and botanists have developed a rich terminology for
describing leaf characteristics.
50
4.3 Visual representation Summary
[36] Leaves show a strong single scattering component , thus can be solved
using Hanran-Krueger analytic function, but it can be approximated furthermore
doing some assumption.
Figure 25: Example of single scattering in leaves as seen from backside
In contrast to other translucent material, plant leaves are usually very thin
and highly textured. Scattering is most forward scattering and the multiscatter term
can be neglected . A simple but fast approximation of subsurface scattering is
obtained computing Irradiance with
(
−s
L = L(1) + Ld = 1 + e i e
− so
) L (x , ω ) ⋅ (N ⋅ ω )
i
i
i
(41)
In this case the single scattering and diffusion approximation by a diffuse
light source. The reflected light is modelled by specular reflection.
51
For implementation as in [36] seven different textures are used in
conjunction: front and back color leaf obtained from a real photo, alpha map used to
model arbitrary shape using single geometric quad. Translucency map representing
leaf seen through light filtering, a thickness map , a normal map for surface normal
perturbation, extinction map models light absorption.
At each rendering step for each image pixel look up tickness and extinction
coefficient and perturbed normal, compute refracted ray,. Compute path length
trough leaf and exponentially attenuate light intensity, if light and viewer are at the
same side of the leaf modulate value by front or back texture map and add specular
lighting, else modulate value by through light texture.
Figure 26: Texture map used. Taken from [36]. From left to right: adaxial diffuse,
abaxial diffuse, translucency, alpha map, thickness map, extinction map, bump
map.
4.4 Human Skin Rendering Challenges
Human skin is the most difficult material to reproduce in computer graphics
for two reasons: first we are used to see skin since we born, so our eyes cannon be
tricked so easily; second, skin is a multi-layered heterogeneous translucent material
where each layer has distinct optical and physical properties. Photorealistic skin
rendering is considered like “holy grail” of computer graphics animation by most
artists [30].
Besides, skin is an alive tissue, so there are many other complementary clues
that determine if skin is real or fake: skin appearance varies not only across
52
individuals, but varies spatially and temporary within a single individual too. Skin
appearance is highly affected by the position of the camera and the light
environment used due to subsurface scattering. The goal to understanding deeper
human skin behaviour is shared by many research areas including medicine,
cosmetics, computer graphics. Computer graphics aims to render photorealistic
images for movie and games.
Skin rendering techniques can be divided into two categories: Reflectance
based models, and image based rendering techniques. The former category can be
further divided into analytical and numerical models. Reflectance models can be
real-time or non real-time. Besides, there exists another approach not only used in
skin rendering, to achieve better results rendering skin: asperity scattering.
4.5 Anatomy of Human Skin
[54] Here is a brief description on the parts and functions of the human skin.
The human skin is composed of three main layers: the Epidermis layer, Dermis
layer, Hypodermis (subcutaneous adipose) layer (see Fig. 29).
Figure 29: Skin cross section: layered nature of skin.
53
4.5.1 The Epidermis
The epidermis layer, providing waterproofing as well as a barrier to infection
contains no blood vessels. The epidermis can importantly be subdivided into a
further 5 layers (from superficial to the deepest);
- Corneum layer, this outermost layer is mostly comprised of flattened dead
cells pushed towards the surface and gradually break apart. It is covered with a layer
of sebum, a oily-waxy material produced by the sebaceous glands.
- Lucidum layer, represents a transition from the underlying Granulosum
layer to the above Corneum layer.
- Granulosum layer, contains lipids whose role is to help prevent fluid loss
from the body.
- Spinosum layer, consists of newly created cells from the underlying layer.
- Basale layer, contributes to the regeneration process of new cells.
Cells form at the innermost basale layer through mitosis, with daughter cells
forming through the higher layers but eventually dieing due to isolation from blood
source.
Due to this, cytoplasm is released, resulting in keratin proteins inserted
through layers as we go higher, ultimately reaching the upper corneum layer,
resulting in a fast slough off. This process, through the layers, takes about 30 days
and as is known as keratinisation. As this keratinised process moves through the
layers, it is responsible for keeping water inside the body and pathogens out,
providing the natural barrier to infections.
Melanin pigments are the most important part of the epidermis considering
optics. These pigments protect the materialof the underlying mitotic cell from ultra
violet rays. All individuals have varying content of eumelanin whereas the
phomelanin is only found in individuals with the corresponding genetic trait.
54
4.5.2 The Dermis
The main function of the dermis is thermoregulation. About 90 percent of
the skin mass comes from the dermis.
The dermis layer contains blood vessels, nerves, hair folicals, smooth muscle,
glands and lymphatic tissue. The connective loose tissue, areolar, has collagen,
elastin and reticular fibers within. Fine erector muscles between the hair papilla and
epidermis, can contract, which results in the forming of goose bumps as the hair
fibre is pulled upright. Sebum is produced by sebaceous glands, used for
waterproofing, antibactericidal action, and lubrication. These glands open via a duct
onto the skin by a pore.
The number of blood vessels, and the colour of the blood mostly influence the
appearance of the skin in this layer (particularly the Caucasians, since they got fewer
eumelanin in their epidermis, that fact allows more light to reach the layer of the
dermis).
4.5.3 The Hypodermis
The hypodermis layer structure actually isn’t part of the skin with its purpose
being to attach the skin to the bone structure and muscles, as well as providing it
with nerves and blood vessels . It consist of two layers, the superficial fascia and the
deep fascia. It plays no role in the optics of the skin.
55
Figure 27: Light penetration through hand, from backside light source.
4.6 Travel of Light Trough Skin
When light rays hits a translucent material such as skin, some of these rays
simply bounce off the surface at the exact same position like the Lambert model.
However, some of the rays get absorbed within the top Epidermis layer of the skin
and die, whereas others will travel further within the skin. Of these rays, some will
change to a slightly different direction as what they entered at when they pass into
the Dermis layer, containing blood vessels, nerves, and smooth muscle tissue. Some
of these rays will hit blood vessels and muscle, get absorbed and die, whereas others
will perhaps bounce and reflect back to the surface.
Of the rays that bounce and reflect, they may exit at a different point at which
they entered, back-scattering. When light enters skin it can exit into a shadowed
area, illuminating that area softly, with colored light effected by the tissue within the
skin (see Fig. 27). This is where we get color bleeding. Light that travels through a
material and exits the others side is known as forward-scattering, seen clearly when
light passes through illuminated ear tissue. Light within skin tends to multi-scatter
isotropic, as light bounces around the layers of the skin constantly illuminating until
its uniform as shown below, even though the skin material is anisotropic. Other rays
56
will travel further into the final layer of the skin, the Hypodermis layer. These rays
will eventually meet the bone where they will die (see figure 28).
Figure 28: optical property of human skin.
4.7 Asperity Scattering
Unlike the above models mentioned in previous chapters, which attempt to
simulate the reflectance properties of the skin itself, asperity scattering is a
rendering technique that tries to simulate the effect of a thin layer of sparse
scattering entities (such as dust or small hairs) on the surface of an object. It can be
used to give a softer appearance to objects that are covered in small hairs or fibers,
such as fabric, some plants, and human skin.
Because the scatterers are sparsely distributed, the asperity scattering term
does not contribute noticeably to the reflected light at angles nearly perpendicular to
the surface. However, when either the direction of incoming light or the viewing
direction is at a grazing angle to the surface, the reflect of many scatters is
compounded, and the result is a soft glow around the edges of the object. This glow
is an important visual cue to suggest that the object is soft or “velvety". Because the
asperity scattering term simulates hair instead of skin, it should be combined with
another local illumination model that simulates the properties of skin.
57
The underlying skin model is used to calculate the shading for the skin itself,
and the asperity scattering term adds a layer of glow on top of that. The angles of the
incident and reflected light determine both the intensity and opacity of the glow, so
that it only makes an appreciable difference at grazing angles.
Figure 29: look at velvety appearance at grazing angle
4.8 The Kubelka Munk Model
[7] An early approach to the problem of modeling the appearance of layered
surfaces was developed by Kubelka and Munk starting in the 1930s [32]. They
considered the problem of determining the resulting overall color from a set of
layers of differently colored paints. For example, consider a white wall with a coat of
blue paint on top of it. For thin layers of blue paint, the resulting color will be a very
light blue, but as the thickness increases, the resulting color will approach to the
color of the blue paint. As such, the color of the underlying wall starts out having a
large contribution to the overall color for thin added layers of blue paint, but given a
sufficiently thick layer on top of it, the base color effectively has no more impact.
58
Figure 30: light scattering within layers of colored paint, some rays are
scattered some absorbed; this process continues until all light is absorbed or
scattered out of the medium.
Media like paint can be modelled volumetrically as collections of small
colored particles. As light passes through the medium, some of it misses all of the
particles, and exits with reduced intensity due to absorption in the medium. The rest
of the light interacts with one or more particles in the medium, and is scattered in a
new direction. The interactions with the particles cause some wavelengths of the
light to be absorbed, so that the particles’ colors change the color of the light that
hits it. Because there are so many particles, these interactions are often modeled
statistically, rather than as individual interactions. In order to model these
situations accurately, it’s necessary to both model the scattering of light within each
layer as well as the scattering between the layers (see Fig. 30). In its most general
setting, solving this problem is analogous to solving the rendering equation to model
global light transport in a geometric scene; all the same issues of modeling multiple
interactions of light come into play.
Kubelka and Munk, however, made a number of assumptions about the
details of these interactions in order to develop a model that could be solved
analytically:
_ They assumed that the composition of each layer was homogeneous; i.e.
having uniform color and scattering properties throughout.
_ They ignored the directional distribution of light: specifically they assumed
that the top layer was irradiated by incident light uniformly from all directions, and
that light exiting each layer after scattering was also uniformly distributed.
The homogeneity assumption is only somewhat limiting–the composition
rules. However, the assumptions about light distribution are more serious. Uniform
59
incident illumination is rare in practice (except for outdoors on an overcast day, for
example); a uniform exiting distribution is not physically plausible but is not too far
off the mark for objects that are not very glossy.
By making the above assumptions, Kubelka and Munk were able to compute
an analytic solution to this limited multiple scattering problem between all of the
layers. The resulting model has been found to be useful in a variety of scientific fields
and is widely used due to its predictive power. It can be extended to model scattering
from more complex objects, and some of the ideas behind it provide a basis for the
more sophisticated approaches described in Section 3.3.
4.8.1 Why Not Use Compositing?
The most common approach to modeling layered surfaces in shaders is to use
alpha blending. Consider simulating tarnish on a copper surface: standard
illumination models would be used to compute a color for pure tarnish and for the
original copper surface, and the two are blended according to the thickness of the
tarnish:
color Base = MaterialRoughMetal(N, copperColor, Ka, Kd, Ks, roughness);
color Tarnish = MaterialMatte(N, tarnishColor, Ka, Kd);
Final_Color = mix(Base, tarnish, tarnishLevel);
Where:
Ka = ambient coefficient
Kd = diffuse coefficient
Ks = specular coefficient
N = surface normal at shading point
And tarnishLevel is determined empirically from some notion of how thick
the tarnish is at the point being shaded.
This approach often works perfectly well and can create convincing images.
However, it has a number of drawbacks grounded in its lack of physical accuracy:
60
•
It doesn’t accurately model the effect of varying thickness. For
example, doubling the layer of the tarnish layer doesn’t double its influence on the
final color computed. Rather, the effect of increasing layer thickness is non-linear, as
once it reaches sufficient thickness the base layer is no longer visible and additional
thickness has no effect. While this particular drawback can be fixed by clamping the
maximum tarnishLevel to one; it’s a symptom of a deeper problem.
•
It doesn’t account for what happens to light as it passes through the
tarnish. For example, the tarnish will likely cause the light passing through it to
become diffused; light entering the top layer is likely to exit the bottom with a
random direction (see Figure 3.2). The result of this is that specular highlights from
the bottom layer should be gradually washed out or eliminated, since much less light
would be coming in from the directions that caused specular highlights to be
generated.
In short, because it completely sidesteps the problem of how light scatters
between the two layers, the results of varying the parameters aren’t intuitive.
4.8.2 The Parameters
Kubelka and Munk assumed that three parameters that describe the lowlevel scattering properties of each layer are known. These are:
•
d, the physical thickness (or depth) of the layer
d
•
σa , the attenuation coefficient of the layer, and
61
The attenuation coefficient describes how much light is absorbed by the layer
as it passes
through. Specifically, the attenuation coefficient of a volumetric medium
describes the differential attenuation per distance traveled in the medium; as such
it’s necessary to know both the coefficient and the thickness of the object in order to
compute how much light it attenuates. If light travels a total distance x between
entering and exiting the medium, the fraction of it that was attenuated is 1 − e − σax .
•
σs , the scattering coefficient of the layer
The scattering coefficient describes how much light is scattered back out by
the object per unit distance of travel through it. As light travels through the medium,
it will interact with the particles inside of it. After such an interaction, it then may
scatter in a different direction than it was originally travelling; some of this light
eventually leaves the medium, having taken on the color of the particles it interacted
with.
In many applications, it’s difficult to determine the values of the σa and σs
parameters for the medium being modelled. However, this doesn’t need to stop us
from using the Kubelka Munk model.
We can estimate reasonable values for the parameters if we know the
reflectance of an infinitely thick sample of the layer R . Fortunately, R is an easy
∞
∞
and intuitive parameter to start from, since the problems we typically use Kubelka
Munk for are along the lines of, “I know that if there’s enough tarnish on this object,
I won’t see the base surface and it’ll be a particular shade of green; what color will it
be with less tarnish?” The Kubelka Munk equations (given in the following section)
can be inverted to show that the ratio of the attenuation and scattering coefficients
can be written in terms of R .
∞
62
(
σa 1 − R∞
=
σs
2 R∞
)
2
(42)
4.8.3 Reflection and Transmission From a Single Layer
Once that we’ve got the scattering and attenuation parameters for a medium
that we’re trying to model, and given the layer thickness, we can start to apply the
KM machinery to make pictures. The Kubelka Munk [32] equations give values for R
and T, the reflection and transmission coefficients of the layer. More specifically,
these give us the fraction of light that is reflected and the fraction that is transmitted
through a particular layer.
The Kubelka Munk equations are:
R=
sinh t
a sinh t + b cosh t
b
T =
a sinh t + b sinh t
(43)
Where:
a = (σs + σa ) σs
b=
a2 − 1
t = bσsd
hyperbolic sine and cosine can be computed using the identities
(
cosh x = (e
)
)2
sinh x = e x − e − x 2
63
x
+ e− x
(44)
4.8.4 Composition of Layers
The Kubelka Munk model really becomes useful when we start to work with
multiple layers. We would like to compute the resulting reflectance from putting all
of them on top of each other. The resulting values should account for all the light
inter-reflection between all of the layers.
Consider the case of two layers with known R and T values, R1, T1, R2, and
T2.Assume for now that the bottom layer is just opaque–e.g. that we’re modeling the
result of tarnish on a dull old metal surface. Thus, R2 might be the diffuse reflection
coefficient of the surface without tarnish, Kd, and T2 would just be zero.
We can compute a new R value for the amount of light reflected from the new
composite surface by considering what happens to the light that passes through the
top layer. For example, some will be reflected from the top layer, without even
reaching the metal surface (R1 of it). Some will be transmitted through the top layer
(T1 of it, in fact), reflected by the base surface (R2), and transmitted back out
through the top (T1). As we consider all of the possible interactions of this form, we
find the infinite series:
R 0 = R1 + T 1R 2T 1 + T 1R 2 R1R 2T 1 + ...
or
R 0 = R1 + T 1R 2T 1 = (1 − R1R 2 )
Similarly, if the base object isn’t opaque (T2 6= 0), we can compute the
transmission coefficient for light passing between the two layers together by
T 0 = T 1T 2 = (1 − R1R 2 )
One important thing to notice about these equations is that the R and T
values completely characterize the scattering behavior of the layer; once we have
computed them, we no longer need to worry about the details of scattering and
attenuation coefficients, layer thicknesses, etc. Again, the translation of these
equations into shading language is straightforward.
64
4.9
Multiple Dipole for Light Diffusion in Multi-Layered
Translucent Material
A slightly modified version of the dipole source approximation by Jensen can
be used to simulate light interaction in multi layered material like paper, leaves and
skin [51].
The dipole approximation was derived for the case of semi-infinite medium.
It assumes that any light entering the material will either be absorbed or return to
the surface. For thin slabs this assumption breaks down as light is transmitted
through the slab, which reduces the amount of light diffusing back to the surface.
This means that the dipole will overestimate the reflectance of thin slabs. Changing
the boundary conditions can account for light scattering in slabs. Using it for human
skin avoids the waxy like appearance(some results in figure 32).
Figure 31: multiple dipole scheme [51].
65
(a) rendering appeared in [22], using [51]
Figure 32: (b)Example of off-line state of the art human skin rendering using [51],
the model is fine scale laser scan, no textures were used.
66
(a) A real-time screenshot of muhammad ali from “Fight night round 3” game
Figure 33: (b) real-time photo from technology demo of “Crysis” game using
hemoglobin-melanine shader.
67
Charter 5
Skin rendering using texture space
diffusion layer compositing
A good reference for accurate physiological modeling of exact properties of a
many layered skin model is [44].
[43] The study reveals that only about 6% of the light that hits skin reflects
off directly.
This happens because of a change in index of refraction between the air and
the oily layer and epidermis. A refraction also happens (which is where most of the
light goes), because of they have an index of about 1.4. Therefore a Fresnel reflection
takes place.
Ratio of reflected to refracted light is highly directional (Only about 2.8%
reflection at normal incidence, approaching complete reflection at tangential
incidence)
The 2.8% comes from the Fresnel reflectance for a smooth dielectric surface
with index of refraction of 1.4 assuming air is the other medium (or a vacuum). The
value is 0.02777778. The true value of total reflectance at normal incidence is
different from this once we replace the Fresnel term (which assumes a perfectly
smooth skin surface) with a BRDF that simulates micro-facets. But we use this value
in computing the Fresnel reflectance term which appears in computation of the
BRDF.
The remaining 94% of light which travels underneath the top most layer gets
scattered and absorbed quite differently in each color band.
The absorption
characteristics of light in tissue is highly color dependant due to melanin and
hemoglobin (and oxy-hemoglobin).
We need to handle the two main features of subsurface light transport: light
can come out somewhere else from where it came in, and depending on where it
went, it’s been colored (absorbed) differently along the way. Looking at the transport
68
theory, many parameters model this behavior: average distance between
scattering/absorption events, layer thickness, mean cosine scattering angle
(probability distribution function of light direction after a single scattering event).
It turns out for skin, that distance between scattering event is quite short,
about 0.02mm against epidermis thickness of 0.25mm, and after a single scattering
event light becomes isotropic.
Thus, by the time light gets about 1/10th through the first skin layer all
directional effects are completely lost.
We only need to keep track of how much total light gets through the first oily
layer and Fresnel-interaction with the top layer.
And then once we estimate the total diffuse light that should get back out due
to all the scattering below and in neighboring regions, again, the oily layer and rough
surface affect how much of that light exits in a given direction. It does not exit
completely uniform on the way back out of the surface. It turns out (see later) for
skin this is pretty negligible, unless the surface is very wet.
Then any light reflection back is diffuse (equal in all directions). (Ignoring
the possible directionality induced by passing through the top oily, rough layer).
5.1
Overview of the process
In each frame we compute Subsurface Irradiance, then several blurred
version of the Subsurface Irradiance, then in the final pass using blending we
evaluate final Subsurface Radiance, we add specular and Rim lighting (see Fig. 34).
69
5 times
Render texture
space light
+
+
Textures
start
Shadow mapping
horizontal
vertical
blur
…
blur
Linear combination
Texture mapping
Rim & specular
Figure 34: block diagram of shading algoritm
5.2 Implementation algorithm details
[15] As we noted earlier, one of the most obvious signs of subsurface
scattering is a general blurring of the effect of lighting because of light enters in one
location and exits in another. In fact, 3D Artists often emulate this phenomenon is
screen space by performing Gaussian blurs of their renders in adobe Photoshop and
then adding a small amount of the blurred image back on top of the original. This
“glow” technique softens the lighting and makes the images look less computer
generated.
It is possible to simulate diffusion in texture space like the technique
introduced in “Matrix Reloaded” movie [45] and proposed first time for real-time
computer graphics by [18][19][39]. We can unwrap the mesh of the object with a
vertex program that uses UV texture coordinates as the screen position of the vertex.
The program simply remaps the [0, 1] range of the texture coordinates to the [-1, 1]
range of normalized device coordinates. We have to be careful that the object has
good UV mapping; that is, each point on the texture must map to only one point of
70
the object, with no overlaps. By lighting this unwrapped mesh in the normal way, we
obtain a 2D image representing the lighting of the object. This technique is useful for
other applications, because it decouples the shading complexity from the screen
resolution: shading is performed only for each texel in the texture map, rather then
for every pixel of the object. Many operation, such as convolutions, can be
performed much more efficiently in image space than on a 3D surface. If the UV
parametrization of the surface is relatively uniform, this is not a bad approximation,
because points that are close in world space will map to points that are also close in
texture space.
To simulate light diffusion in image space, we can simply blur the light map
texture.
One layer skin model looks waxy. This happens if assuming skin as one
uniform material.
The scattering of the epidermis is quite narrow and the scattering of the
lower layers is much broader, so the result is something in between which doesn’t
look right, especially up close. Some implementations models skin with up to 6
distinct layer, but we will use a 3 layer skin model.
To model this, different blurs of different blur widths will be weighted
differently depending on what color of light we consider.
Amount of diffuse reflected light becomes a function of distance from the
entry position. This is called the reflectance diffusion profile of the materials. Often
denotes R(r). So for a flat 2D surface diffusion scattering operation is just a
convolution of the 2D image, but it is not a simple Gaussian box filter and it’s a
different blur for red, green, and blue color channel.
5.2.1 Diffuse Component
The first step is to unfold the mesh in a off-screen render target using UV
texture coordinate in a vertex program. Then the fragment program will compute
subsurface irradiance. To compute Subsurface irradiance we may multiply diffuse
color map by a diffuse lighting term, but this choice will loose high frequency
lighting detail. As second attempt we could just stick N dot L in the blur passes,
combine them at the end, and then multiply by diffuse color.
71
If the diffuse color came from a photograph of real skin, the color bleeding
has already taken place, so this might be ok.
For a synthetic diffuse color map, high frequency detail and color
information will not bleed as it should because of the subsurface scattering.
Eventually we can somehow do both: we can’t multiply N dot L with diffuse
color, blur that, then multiply by diffuse color again; that would apply the diffuse
color twice – resulting in a darkened saturated color.
We want to multiply before and after and have the final color be diffuseCol.
So let’s multiply by the sqrt( diffuseCol ) each time.
This is the only physically plausible scenario. The multiply by Diffuse color
corresponds to the surface absorbing some of the incident light. Doing it completely
before or completely after scattering implies the light somehow only interacts with
this absorbing layer once. But it must pass through it twice (see Fig. 36). Another
similar techniques was used in [34]: they use a single blurred lightmap to
approximate light scattering as it enters the volumetric material, but they used a
variable size Poisson filter kernel; kernel center position is based on parallax eye
vector penetrating inner layer. Then they account for light scattering on its way back
out of the material. The deeper the inner layer is from the surface and the more
edge-on the viewing angle is, the more material you have to look through, and thus
the more blurry it should appear. So they used again Poisson disc filter but this time
to filter inner layer albedo map with kernel centered about parallax offset texture
and radius depending on thickness (see Fig. 35).
Figure35: [34] Poisson filtering example at input point and exitant point.
72
Thin oil layer
Detail absorption
layer
Epidermis
Dermis
Skin layers
Figure36: [43] During absorption light color change to the one of hit particle, thus
we need to take account for this phenomenon using two times textures color values.
As exposed earlier we can choose different BRDF; for lighting we pick the
diffuse part of Ashikhmin-Shirley BRDF [31], which has energy conservation and
reciprocity properties.
D ( L, E , N ) =
5
5
⎛
⎞⎛
⎞
28Rd
(1 − Rs )⎜⎜1 − ⎛⎜1 − N ⋅ L ⎞⎟ ⎟⎟⎜⎜1 − ⎛⎜1 − N ⋅ E ⎞⎟ ⎟⎟
23π
2
2
⎠ ⎠⎝ ⎝
⎠ ⎠
⎝ ⎝
Where:
L is normalized point to light vector
N is normalized normal vector
E is normalized point to camera vector
Rd diffuse coefficient
Rs specular coefficient
Fixing the dot product N ⋅ E = 1 ; and not the usual fast Lambertian diffuse
D ( L, N ) = N ⋅ L
73
(45)
In this stage we add Shadows, using Shadow mapping technique [32].
Shadow mapping requires a preliminary rendering pass of the scene from the
point of view of the light, saving only depth of each pixel of the resulting image in a
“depth texture”.
Figure 37: depth map example: darker points are near light view plane.
Then when we render the scene from eye position, we project “depth
texture” on the scene using projective texture mapping.
Figure 38: shadow map algorithm scheme (source nVidia).
74
At each pixel, the depth sample (from the projected shadow map texture)
with the fragment’s distance from the light. If the latter is greater, the pixel is not the
closest surface to the light source. This means the fragment is shadowed, and should
not receive light (in real life probably it will be caught up by indirect lighting). In
addition we perform Percentage Closer Filtering (PCF); it would be a mistake to
average depth value, instead it averages final neighbourhood coverage values.
Figure 39. Shadow final stage
We also take account for low frequency ambient lighting, meaning the
illumination at each point is a function of other geometry in the scene, which
produce light occlusion and interreflections.
Ambient occlusion is a global illumination technique most often calculated
by casting rays in every direction from the surface. Rays which reach the background
or “sky” increase the brightness of the surface, whereas a ray which hits any other
object contributes no illumination (see Fig. 40).
As a result, points surrounded by a large amount of geometry are rendered
dark, whereas points with little geometry on the visible hemisphere appear light (see
Fig. 40). The result can be considered a simple first order approximation of the
whole rendering equation.
E ( p ) = ∫ N ⋅ ωL(ω )dω
Ω
75
(46)
The ambient occlusion shading model has the nice property of offering a better
perception of the 3d shape of the displayed objects.
Figure 40: Ambient Occlusion representation
Recently some algorithm were introduced to compute AO in real time, but in
our skin shading method, we can bake default face pose occlusion values in texture,
since facial animation will not produce big mesh topology differences.
Figure 41: (a) purely diffuse, Ashikmin-Shirley, surface with a constant ambient
term (b) Enhanced shading with an ambient occlusion term
76
In the past ambient lighting was approximated using constant color value.
Indirect illumination represent secondary light bounces off surfaces; We can
approximate area lighting using cube-map textures. A cube-map is made up of six
textures (see Fig. 42) each one for cartesian axis directions. This technique is largely
diffused for specular environment reflections, We transform standard cubemap for
diffuse lighting using RenderMonkey as a precomputation step. Every object in the
scene is ideally at the center of the cube and we access six cube faces using vertex
normal [29].
Figure 42: example of cubemap, first original st. Peter on the right convolved version
from devebec.com.
After rendering Subsurface irradiance we blur that image several times using
7-tap separable blurs (Gaussians), Each blur operation is 2 passes: blur in x, then in
y using temporary buffer in between.Each time widen taps and repeat, keeping
several blurred versions available for the final render pass
Make sure the blur weights sum to 1.0 (at all steps); these blur weights
should not be confused with the weights used to linearly combine different blurred
images in the final render pass (see later).
One 2D Gaussian blur can be built from the last, since the convolution of a
Gaussian by a Gaussian is a Gaussian. We recursively blur the previous image, by
77
widening the blur kernel (no more than 2X the previous one). This provides a basis
of different blurs for mixing and matching to achieve the desired diffusion profile.
Incoming focused
beam
How much diffuse
light at distance r?
r
Figure 43: diffusion profile r [43].
Original image
horizontal blur
78
Vertical blur
Figure 44: blurring stage (exagerated).
79
If we naively blur uniformly over this 2D texture, we’ll over-blur in some
areas, for instance this will overblur the ears versus what blurring happens on the
nose.
Moving one texel in texture space corresponds to a distance in the real world
that highly varies over the surface.
We can easily compute distortions in U and V directions computing texture
coordinate derivatives in texture space, and correct our blur radii locally.
Pre-compute this and store in a texture map (see appendix A). It can help to
blur it slightly to remove some of the high frequency changes from one triangle to
the next (especially for when it’s used for large blurs).
Store (1 / stretch ) so that we can multiply the blur radius by this value
everywhere on the mesh.
Another side effect of texture space convolution occurs while sampling textel
near seam edges; background texture color will blur into nearby pixels, creating
wrong color seams. It can be overcome using appropriate clearColor while rendering
to offscreen render target, you can edit the UV distortion maps to be black at the
seams, or use an alpha map to encode white where the object exists in UV space, and
black where it doesn’t.
In the final render pass, a texture lookup tells you when a texture lookup is
filtering empty texture space (use this to lerp towards local lighting only).
In the final rendering pass we combine several blur, and physically based
specular term.
In order to mimic various translucent materials, photographs can reveal
some information that can help us match the look. Do so one color channel at a time
(view only the red channel of the photo, and return only red in the shader while the
values are changed. Look at small bumps and how the dark holes get filled in by the
subsurface scattering and look at how light bleeds into dark shadowed areas, then
switch to green, etc).
For human skin: red channel has the broadest amount of scattering by quite
a bit, green channel has more than blue, blue has very little at all.
Our visual system is quite sensitive to the subtleties of scattering in skin
A single Gaussian blur doesn’t accurately model the diffusion of any material:
heavy blurring with a sharp specular creates a very waxy look.
80
It is much better to use several blurs of different radii and blend them,
because skin is a combination of narrow scattering from the upper skin layers and
broad scattering from the lower skin layers. the choice of the RGB channel weights
is somehow an artistic process.
Figure 45. tileable map [43]
Another skin clue are roughs and pores; this kind of meso-scale
irregularities, cannot be baked in single huge resolution bump map, instead we can
extract various tileable bump maps from photos and combine them using a mask for
different zones of the face. This is a simplification of a technique introduced by [54]
were they captured fine scale skin structure of face using silicone mold. Then they
syntetized normal map texture as a stochastic process and applied them on different
zone of the face manually, hiding seams using multi-resolution splines.
We cannot simply add normal maps, because vector addition doesn’t achieve
correct results as in figure.
Bump map 1
+
Bump map 2
=
Figure 46: Here you can see the wrong result of normal vectors addition in red color,
the correct one in black [43].
81
So we derive final normal vector starting from height field map; we combine
global and tileable height map and compute normal vector using equation
N=
Hg − Hr , Hg − Ha,1
(47)
(Hg − Hr )2 + (Hg − Ha )2 + 1
Where Hg is the height of the given textel and Hr is the height of the textel
directly to the right of the given textel, Ha is the height of the textel directly above
the given textel.(for reference map see later).
5.1.3 Specular Component
Specular color should be white, Skin tissue is dielectric.
Only metallic
surfaces have colored specular (because their index of refraction is a complex
number).
In addition to bumps due to pores and other surface irregularities, the
outermost surface of skin has very small scale roughness also called micro-scale.
This averages many perfect Fresnel interactions based on a probability of
hitting a micro-facet The micro-facets deviate from the surface normal based on
some roughness parameter “m” which is similar to “specular exponent” of a Phong
specular term (although it has an inverse relation to exponent). Instead of usual
Phong specular term, a physically based BRDF captures increased specularity at
grazing angles [58].
Oily
Epidermi
Extremely small
Figure 47: microscale representation, made-up by sebum and dead cell [43].
82
A single specular roughness value (or exponent) simulates a very regular
distribution of fine scale roughness on a surface
Skin is an incredibly complex material with many small-scale features mixed
together.
In [53] Statisical data is available in their paper which assigns m values and
specular weights over 10 regions of the face
Figure 48: Example of different oiliness on the face (only 4 zones); the nose
is the most oily [12].
We can store these values in a single low resolution texture map for the face.
It’s a great start for choosing specular roughness values. Noise can be added as
desired, and there’s no reason not to paint in new values, if you get something that
looks even better.
Rim lighing simulates the effect of peach fuzz where the skin is brighter at
glancing angles than when viewed straight on. The Fresnel Factor comes handy. The
Fresnel Factor Fλ (u ) expresses the reflection of light on the well-oriented
microfacets i.e. the facets whose normal vector is H (half way vector H = (V + L ) .
For a non-polarized electromagnetic wave, its formulation is:
83
2
2
1 (a − u ) + b 2 ⎡ (a + u − 1 u )
Fλ (u ) =
⎢
2 (a + u )2 + b 2 ⎣ (a − u + 1 u )2
2
1
a 2 = ⎛⎜ nλ2 − kλ2 + u 2 − 1 + 4nλ2 kλ2
2⎝
2
1
b 2 = ⎛⎜ nλ2 − kλ2 + u 2 − 1 + 4nλ2 kλ2
2⎝
(
)
(
)
⎤
+ b2
+ 1⎥
2
+b
⎦
+ nλ2 − kλ2 ⎞⎟
⎠
− nλ2 + kλ2 ⎞⎟
⎠
(44)
Where nλ is the ratio of the refraction indices above and below the surface
and kλ is the extinction coefficient of the surface. An interesting characteristic of the
Fresnel factor is that Fλ = 1 at grazing incidence ( u = 0 ) whatever the wavelength
λ (see figure).
Schlick [2] proposed a faster approximation with less then 1% error:
(
)
F λ (u ) = f λ + 1 − f λ (1 − u )
5
where f λ is the material value at normal incidence angle.
Vacuum refraction index is the reference for other material indices.
Figure 49. Fresnel reflectance, Fresnel Transmittance
84
(45)
5.1.4 Gamma correction
[55] An important component for the realistic final result often used in offline rendering pipeline is gamma correction.
Gamma correction controls the overall brightness of an image. Images which
are not properly corrected can look either bleached out, or too dark.
Almost every computer monitor (LCD, CRT), from whatever manufacturer,
has one thing in common. They all have a intensity to voltage response curve which
is roughly a 2.2 power function. This just means that if you send your computer
monitor a message that a certain pixel should have intensity equal to x, it will
actually display a pixel which has intensity equal to x ^ 2.2. Because the range of
voltages sent to the monitor is between 0 and 1, this means that the intensity value
displayed will be less than what you wanted it to be. (0.5 ^ 2.2 = 0.217 for example)
Monitors, then, are said to have a gamma of 2.2.
Sample Input to Monitor
Graph of Input
Output from Monitor
Graph of Output L = V ^ 2.2
To correct this annoying little problem, the input signal to the monitor (the
voltage) must be "gamma corrected".
The solution, fortunately, is a simple one. Since we know the relationship
between the voltage sent to the monitor and the intensity which it produces, we can
correct the signal before it gets to the monitor. The signal is adjusted so that it is
essentially the complement of the curve shown above. There are other
considerations as well when one speaks of a "correct" image. These include the
ambient light in a room where the computer is, the brightness and contrast settings
85
on the monitor, and finally personal taste. Doing correct gamma correction for color
maps and final pixel values increases the realism of the image. Ignoring it makes the
dark edges become dark too quickly, giving the surface a strange CG look. We are
used to seeing images like these because we haven’t been correcting for gamma for
years, but it’s wrong. It’s much easier to brighten the gamma corrected image,
because the levels are all linear, but if you try to brighten non-gamma corrected
image the brightest elements clip and become colored strangely because everything
is non-linear. Any image from a digital imaging device or something painting in a
paint program (or otherwise authored on our non-linear displays) will have nonlinear color values. To accomplish the task of gamma correction we need to raise the
color maps input value to the 1/2.2 power before shading with them. This is referred
to as a gamma correction of 2.2. because we are correcting the input for a monitor
whose gamma is 2.2. Don’t correct source maps which encode non-color information
like: pre-computed ambient occlusion maps, normal maps, alpha channels which
encode values for coverage, blending, compositing, etc…This can be pre-computed
or automated as part as a export process to avoid the pow() instruction for each
texture lookup. New hardware supports sRGB texture formats which do accurate
gamma correction automatically before texture filtering happens in the shader. Then
after shading in linear color space we restore correct gamma curve, so the very last
color value we write to the framebuffer is corrected with the inverse so that the
brightness on the screen follows a linear relationship with our original computed
color.
86
Chapter 6
Conclusions and future work
After examining range of techniques for subsurface scattering simulation we
focused on skin rendering, while most of translucent object appearance could be pre
computed and put in static environment, human skin is tight bound to virtual
character animation.
This skin shader is good to reproduce skin in low lit environment such as
closed environment or night day time; it lack of front-scattering: this effect is clearly
seen during a sunny day when the viewer is opposite position to light source looking
at low tickness skintissue like ears. In Matrix paper they used ray-tracing to recreate
this effect; we could implement it using spherical harmonics technique, or
computing object thickness and evaluating light absorption. We could have compute
ambient occlusion and irradiance map dynamically.
The next step would be to evaluate different lighting model as Oren Nayar,
Lafortune, for diffuse and specular component.
Skin shader has rooms of improvements and optimization since we could use
less number of layers composition, use lower resolution texture, render target and
shadow map. This kind of versions would be sufficient to model other skin part of
the body or animal skin, for example. The standard project configuration without
optimization runs at about 30 frame per second which is our target for target
application like virtual character.
87
Figure 50: results of interactive dipole diffusion approximation.
88
Figure 51: full view of head model rendered in real time using skin shading
89
Figure 52: same models from another point of view; note how light reflects strongly
at grazing angles
90
Figure 53. same models, but seen closer; note skin pores and wrinkles
91
References
[1] - DAN BAKER, NATY HOFFMAN, PETER-PIKE SLOAN 2004. Advanced
Real-Time Reflectance, Game Developer Conference 2004, San Francisco, CA,
USA.
[2] - SCHLICK, C. 1994. An inexpensive BRDF model for physically based
rendering. Computer Graphics Forum 13, 3, 233.246.
[3] - RANDIMA FERNANDO AND MARK J. KILGARD nVidia 2003. The Cg
Tutorial: The definitive Guide to Programmable Real-Time Graphics –
Chapter 1. ISBN-10: 0321194969 , ISBN-13: 978-0321194961.
[4] – FLEMING, R. W., JENSEN, H. W., AND BÜLTHOFF, H. H. 2004.
Perceiving translucent materials. In APGV '04: Proceedings of the 1st
Symposium on Applied perception in graphics and visualization, ACM Press, New
York, NY, USA, 127.134.
[5] - YI GONG AND WEI CHEN AND LONG ZHANG AND YUN ZENG AND
QUNSHENG PENG 2006. An Approximate Image-Space Approach for Realtime Rendering of Deformable Translucent Objects. Computer Graphics
International, 124-135.
[6] - RUI WANG, JOHN TRAN, DAVID LUEBKE 2005. All-Frequency
Interactive Relighting of Translucent Objects with Single and Multiple
Scattering. SIGGRAPH '05: ACM SIGGRAPH 2005 Papers, 1202—1207, ACM
Press, New York, NY, USA.
[7] - MATT PHARR. Layered Media for Surface Shader. In Siggraph 2001
course 48, http://www.renderman.org.
[8] - HAO, X., AND VARSHNEY, A. 2004. Real-time rendering of translucent
meshes. ACM Trans. Graph. 23, 2, 120.142.
92
[9] - CSABA KELEMEN AND LASZLO SZIRMAY-KALOS 2001. A Microfacet
Based Coupled Specular-Matte BRDF Model with Importance Sampling.
EUROGRAPHICS 2001.
[10] - DACHSBACHER, C., AND STAMMINGER, M. 2003. Translucent shadow
maps. In EGRW '03: Proceedings of the 14th Eurographics workshop on
Rendering, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland,
197.201.
[11]
-
XIE
YONGMING
2007.
Geometry
http://appsrv.cse.cuhk.edu.hk/~ymxie/Geometry_Shader/
shader
.
Last
tutorial
access:
3/04/2007.
[12] – TAKANORI IGARASHI, KO NISHINO, SHREE K. NAYAR 2005. The
Appearance of Human Skin. Technical Report: CUCS-024-05. New York, NY,
USA.
[13] – WIKIPEDIA: THE FREE ENCICLOPEDIA - http://wikipedia.org/, last
access: 3/04/2007.
[14] - ANDREW HWANG. http://www.andrewhwang.com/testsite/, Last access:
3/04/2007.
[15] - RANDIMA FERNANDO, nVidia. GPU Gems 2: Programming
Techniques for High-Performance Graphics and General-Purpose
Computation (Gpu Gems) 2005. ISBN-10: 0321335597, ISBN-13: 9780321335593.
[16] - EVERITT, C. 2001. Interactive order-independent transparency. In
white paper: http://developer.nvidia.com/object/order independent transparency.
[17] - KALLE KOUTAJOKI 2002. BSSRDF (Bidirectional Surface Scattering
Distribution Function) , HELSINKI UNIVERSITY OF TECHNOLOGY.
93
[18] - NATALYA TATARCHUK, ATI Research. D3D Tutorial: Advances in Real
Time Skin Rendering. Game Developer Conference 2005, San Francisco, CA,
USA.
[19] – DAVID GOSSELIN, ATI Research. D3D Tutorial: Real Time Skin
Rendering. Game Developer Conference 2004, San Francisco, CA, USA.
[20] - LENSCH, H. P. A., GOESELE, M., BEKAERT, P., KAUTZ, J., MAGNOR, M.
A., LANG, J., AND SEIDEL, H.-P. 2002. Interactive rendering of translucent
objects. In PG '02: Proceedings of the 10th Pacific Conference on Computer
Graphics and Applications, IEEE Computer Society, Washington, DC, USA, 214.
[21] - TOM MERTENS AND JAN KAUTZ AND PHILIPPE BEKAERT AND FRANK
VAN REETH AND HANS-PETER SEIDEL 2003. Efficient Rendering of Local
Subsurface Scattering. In PG '03: Proceedings of the 11th Pacific Conference on
Computer Graphics and Applications, IEEE Computer Society, Washington, DC,
USA, 51.
[22] – Eftychios Sifakis, Andrew Selle, Avram Robinson-Mosher, Ronald Fedkiw
2006. Simulating Speech with a Physics-Based Facial Muscle Model. In
Eurographics/ ACM SIGGRAPH Symposium on Computer Animation (2006).
[23] - DORSEY, J., EDELMAN, A., JENSEN, H. W., LEGAKIS, J., AND PEDERSEN,
H. K. 1999. Modelling and rendering of weathered stone. In SIGGRAPH '99:
Proceedings of the 26th annual conference on Computer graphics and interactive
techniques, ACM Press/Addison-Wesley Publishing Co., New York, NY, USA,
225.234.
[24] - FRANCESCO BANTERLE AND ALAN CHALMERS. A Dual Funnel
Approximation for the Appearance of Translucent Material. In SIGGRAPH
'06: ACM SIGGRAPH 2006 Research posters, ACM Press, New York, NY, USA,122.
94
[25] - OLAV MADSEN AND ANDERS KYSTER 2002. Real-time Rendering of
Sub-surface Scattering . University of Copenhagen.
[26] – Christophe Hery- Implementing a Skin BSSRDF (or Several…). In Siggraph
2003 course 9, http://www.renderman.org.
[27] - LIEFENG WANG AND WENLE WANG AND JULIE DORSEY AND XU YANG
AND BAINING GUO AND HEUNG-YEUNG SHUM 2006. Real-time rendering
of plant leaves. In SIGGRAPH '06: ACM SIGGRAPH 2006 Courses, ACM Press,
New York, NY, USA.
[28] - WEYRICH, T., MATUSIK, W., PFISTER, H., BICKEL, B., DONNER, C., TU,
C., MCANDLESS, J., LEE, J., NGAN, A., JENSEN, H. W., AND GROSS, M. 2006.
Analysis of human faces using a measurement-based skin reflectance
model. ACM Transactions on Graphics 25, 3 (July), 1013–1024.
[29] - Z. WEN AND Z. LIU AND T. HUANG. Face relighting with radiance
environment maps. In Proc. of IEEE Conf. on Computer Vision and Pattern
Recognition,2003.
[30] – CLEMENTS BRANDORFF 2006. Rendering Human Skin. TECHNISCHE
UNIVERSITÄT WIEN, Institut für Computergraphik und Algorithmen.
[31] - MICHEAL ASHIKMIN AND PETER SHIRLEY. An Anisotropic Phong
BRDF Model. In Journal of Graphics Tools: JGT, 2000.
[32] - P. KUBELKA, F. MUNK, An article on optics of paint layers, August,
1931.
[33] - PEDRO V. SANDER AND DAVID GOSSELIN AND JASON L. MITCHELL,
Real-time skin rendering on graphics hardware. SIGGRAPH '04: ACM
SIGGRAPH 2004 Sketches. ACM Press, New York, NY, USA, 148.
95
[34] - CHRIS OAT. Rendering gooey materials with multiple layers. In
SIGGRAPH '06: ACM SIGGRAPH 2006 Courses, ACM Press, New York, NY, USA,
65--80.
[35] - CSABA KELEMEN, LASZLO SZIRMAY-KALOS. A Microfacet Based
Coupled Specular-Matte BRDF Model with Importance Sampling.
[36] - FRANZKE, O., AND DEUSSEN, O. 2003. Accurate graphical
representation of plant leaves. In Plant growth modelling and its applications,
Springer-Verlag, B.-G. Hu and M. Jaeger, Eds.
[37] – WILLIAMS, L. 1978. Casting curved shadows on curved surfaces. In
SIGGRAPH '78: Proceedings of the 5th annual conference on Computer graphics
and interactive techniques, ACM Press, New York, NY, USA, 270.274.
[38] - DE BOER, W.H., Real-Time Subsurface Scattering of Arbitrary
Objects,
submitted
to
Journal
of
Graphics
Tools,
AK
Peters
2006.
http://www.whdeboer.com/. Last access 15/04/07
[39] – GREEN, S. 2004. Real-time approximations to subsurface scattering.
In GPU Gems, R. Fernando, Ed. Addison Wesley, Mar., ch. 16, 263–278.
[40] – FRANCESCO BANTERLE AND ALAN CHALMERS A Fast Translucency
Appearance Model for Real-Time Applications. Conference Proceedings,
April 2006. In: To appear in SCCG 2006 - Spring Conference on Computer Graphics
[41] - HANRAHAN, P., AND KRUEGER, W. 1993. Reflection from layered
surfaces due to subsurface scattering. In SIGGRAPH '93: Proceedings of the
20th annual conference on Computer graphics and interactive techniques, ACM
Press, New York, NY, USA, 165.174.
[42]- JENSEN, H. W., MARSCHNER, S. R., LEVOY, M., AND HANRAHAN, P.
2001. A practical model for subsurface light transport. In SIGGRAPH '01:
Proceedings of the 28th annual conference
96
on Computer graphics and interactive techniques, ACM Press, New York, NY, USA,
511.518.
[43] - EUGENE D’EON, nVidia. GDC 2007 Demo Team Secrets: Advanced
Skin Rendering. In Game Developer Conference 2007, San Francisco, CA, USA.
[44] - KRISHNASWAMY AND BARANOSKI .A Biophysically-Based Spectral
Model of Light Interaction with Human Skin (BioSpec).
Eurographics
2004.
[45] - GEORGE BORSHUKOV and J. P. Lewis 2005. Realistic human face
rendering for "The Matrix Reloaded". SIGGRAPH '05: ACM SIGGRAPH 2005
Courses, ACM Press, New York, NY, USA.
[46] - JENSEN, H. W., AND BUHLER, J. 2002. A rapid hierarchical rendering
technique for translucent materials. In SIGGRAPH '02: Proceedings of the
29th annual conference on Computer graphics and interactive techniques, ACM
Press, New York, NY, USA, 576.581.
[47] - GUILLAUME POIRIER 2003. Human Skin Modelling and Rendering.
Master of Mathematics In Computer Science, Technical Report Number CS-200405.
[48] - CARR, N. A., HALL, J. D., AND HART, J. C. 2003. Gpu algorithms for
radiosity and subsurface scattering. In HWWS '03: Proceedings of the ACM
SIGGRAPH/EUROGRAPHICS conference on Graphics hardware, Eurographics
Association, Aire-la-Ville, Switzerland, Switzerland, 51.59.
[49] - MERTENS, T., KAUTZ, J., BEKAERT, P., SEIDELZ, H.-P., AND REETH, F. V.
2003. Interactive rendering of translucent deformable objects. In EGRW
'03: Proceedings of the 14th Eurographics workshop on Rendering, Eurographics
Association, Aire-la-Ville, Switzerland, Switzerland, 130.140.
97
[50] - HENRIK WANN JENSEN 2003. Digital face cloning. SIGGRAPH '03:
ACM SIGGRAPH 2003 Sketches \& Applications, ACM Press, New York, NY, USA.
[51] - DONNER C., AND JENSEN, H.W. 2005. Light diffusion in multi-layered
translucent materials. In Proceedings of SIGGRAPH 2005, 1032-1039.
[52] - CHRISTOPHE HERY, Industrial Light and Magic - Stupid Renderman
Tricks On Shadow Buffers… . Course Siggraph 2002, San Antonio.
[53] - T. WEYRICH, W. MATUSIK, H. PFISTER, J. LEE, A. NGAN, H.W. JENSEN,
M. GROSS 2005. A Measurement-Based Skin Reflectance Model for Face
Rendering and Editing. SIGGRAPH 2005.
[54] - ANTONIO HARO AND IRFAN A. ESSA AND BRIAN K. GUENTER 2001,
Real-time Photo-Realistic Physically Based Rendering of Fine Scale
Human Skin Structure. Proceedings of the 12th Eurographics Workshop on
Rendering Techniques, Springer-Verlag, London, UK.
[55]
–
CGSD
SERVICE-Gamma
Literature.
http://www.cgsd.com/papers/gamma.html - Last access 30/04/07
[56]
–ATI.AMD,
CubeMap
Gen
Tool
http://ati.amd.com/developer/cubemapgen/index.html - Last access 30/04/07
98