\documentclass{article}
\usepackage{fullpage}
\usepackage{graphicx}
\usepackage{minted}
\usepackage{hyperref}
\begin{document}
\title{CS4052 Project - Physically Based Rendering}
\author{Luke Lau 15336810}
\maketitle

\section{Physically Based Rendering}
This project uses a physically based lighting model, which computes per pixel lighting by estimating
how light rays interact with surfaces at a microscopic level, known as the microfacet model. 

\begin{center}
\includegraphics[width=7cm]{microfacets_light_rays}
\includegraphics[width=3.5cm]{surface_reaction}
\end{center}

There are two main properties that determine a material's appearance: roughness and metalness. 
The former determines how much light gets caught in the bumps of the surface and doesn't get
reflected back. The latter is used for approximating how metals absorb refracted light without
scattering.

There are 7 texture maps used when rendering each material. 4 are defined in the model and 3 are
precomputed at the start of the program.
\begin{minted}{glsl}
uniform sampler2D albedoMap;
uniform sampler2D normalMap;
uniform sampler2D metallicRoughnessMap;
uniform sampler2D aoMap;
uniform samplerCube irradianceMap;
uniform samplerCube prefilterMap;
uniform sampler2D brdfMap;
\end{minted}

The Albedo map provides the base colour underneath the diffuse and specular layers, the normal
map is just a standard normal map, and similarly for the ambient occlusion (ao) map.

The Newton's Cradle is loaded from a GlTF2 model, a trendy new model format designed for PBR materials,
and is maintained by the Khronos group (the same group behind OpenGL and Vulkan).
Because it is physically based, the specification expects a metallic-roughness map texture.
The metallic value is stored in the blue channel of the texture whilst the roughness value is stored in the green channel (Inside the source code there is a small python script,
\texttt{metallicRoughness.py} which combines two separate metallic/roughness maps into one suitable
for the GlTF2 format. AssImp recently added partial support for GlTF2 models so I was able to read
in the metallic/roughness texture into a 2 channel texture, and access it from within the shader
like so:

\begin{minted}{glsl}
float metallic = texture(metallicRoughnessMap, texCoords).b;
float roughness = texture(metallicRoughnessMap, texCoords).g;
\end{minted}

The irradiance, prefilter and BRDF maps are all precomputed based off the skybox in order to provide
accurate, image based lighting (IBL) that correctly reflects the contents of the environment. The
skybox itself is loaded in from an equirectangular HDR file, and so needs to be gamma corrected and
converted to a cubemap first. \texttt{skyboxfrag.glsl} handles the gamma
correction and \texttt{equirectangularfrag.glsl} takes care of converting the projection. 

It needs to be in the HDR format since the precomputed maps are based off of physical properties,
and the equations that are involved work in terms of flux. 

\begin{figure}
	\begin{equation}
	L_0(p,\omega_0) =
	\int_\Omega(k_d \frac{c}{\pi} +
				k_s \frac{DFG}{4(\omega_0 \cdot n)
								(\omega_i \cdot n)})
			   L_i(p,\omega_i)n \cdot \omega_i d\omega_i
	\end{equation}
	\label{eq:reflectance}
	\caption{The reflectance equation}
\end{figure}

The irradiance map solves the $k_d \frac{c}{\pi}$ (diffuse) part of figure~\ref{eq:reflectance}, and is
calculated in \texttt{irradiancefrag.glsl}. It approximates the integral by taking a finite number
of samples, and does this for each possible incoming angle of light, storing the result into a
cubemap texture that can easily be looked up later.

The same idea applies for the prefilter and BRDF maps: solve a computationally expensive integral
and shove it into a texture that can be quickly referenced in the main fragment shader. They
precompute the integrals in the $k_s$ (specular) side of the figure~\ref{eq:reflectance}, but unlike
the $k_d$ term, this term is not constant and depends on the viewing angle, $\omega_i$. It would be
crazy to precompute this integral for every single combination of viewing angle and incoming light
angle, so instead this is approximated with the split-sum-approximation\cite{splitsum}. This results
in the equation in figure~\ref{eq:splitsum}, which has two terms: a filter for convoluting
(\texttt{prefilterfrag.glsl}), and the bidirectional reflective distribution function (BRDF) (\texttt{brdffrag.glsl}).

The BRDF is a function that takes in the incoming light direction, the outgoing view direction,
the surface normal and the roughness of the material, and returns how much of the incoming light
is reflected back at that viewing angle.

\begin{figure}
	\begin{equation}
		L_o(p,\omega_o) = \int_\Omega L_i(p, \omega_i) d \omega_i \ast
		\int_\Omega f_r(p, \omega_i, \omega_o)n \cdot \omega_i d \omega_i
	\end{equation}
	\label{eq:splitsum}
	\caption{The split sum approximation}
\end{figure}

The nice thing about generating all these maps is that they can all reuse the same vertex shader,
\texttt{skyboxvert.glsl}, since they all are cubemap shaped!
\texttt{pbrfrag.glsl} then combines all the maps to give the final result.

\begin{figure}
	\centering
	\includegraphics[width=0.3\textwidth]{skyboxLoft}
	\includegraphics[width=0.3\textwidth]{skyboxDesert}
	\includegraphics[width=0.3\textwidth]{skyboxFactory}
	\caption{Different skyboxes having different effects on lighting. Note how this lighting model
	gives reflection for free in the leftmost sphere.}
\end{figure}

\begin{figure}
	\centering
	\includegraphics[width=0.45\textwidth]{gunModel}
	\includegraphics[width=0.45\textwidth]{bulletReflection}
	\caption{A complex, PBR based model. Even the bullets reflect the environment!}
\end{figure}

\begin{figure}
	\centering
	\includegraphics[width=0.45\textwidth]{normalMaps}
	\includegraphics[width=0.45\textwidth]{normalMaps2}
	\caption{Normal mapping and ambient occlusion providing depth to the materials.}
\end{figure}

\begin{figure}
	\centering
	\includegraphics[width=\textwidth]{fresnelEffect}
	\caption{The Fresnel effect, where the edges of the sphere that have a sharper viewing angle
	cause the light to be reflected more strongly.}
\end{figure}

\section{Bone Based Animation}

Since I wanted to take advantage of the GlTF2 model format as much as possible, I
decided to load in the animations via the Assimp library. Many models typically describe their
animations via a skeletal bone system, where each vertex has a weight associated with each bone in
the model, affecting how much it is transformed by when the corresponding bone moves.

The first step was to add bones to the model I had created in blender. I created two bones,
one for each swinging sphere, and then assigned the vertex weights of the two spheres and the bottom
vertices of the ropes to their respective bones.

The bones could then be animated using keyframes. By specifying the starting and apex, I
could use Bezier curves to interpolate between them in such a way that gave the appearance of
momentum. Thankfully, the GlTF2 exporter in Blender bakes the Bezier interpolation into many
subdivided keyframes, so I was able to just linearly interpolate between them in my program.

\begin{figure}
	\centering
	\includegraphics[width=0.45\textwidth]{blenderBone}
	\includegraphics[width=0.45\textwidth]{blenderDopeSheet}
	\caption{Adding bones and animating them in Blender.}
\end{figure}

The program needed to calculate the transformation of each bone at each frame in time of the
animation, and bind the transformations into a uniform array of transformation matrices. The vertex
shader is then able to use these matrices, alongside the weights for each vertex to calculate its
new animated position:

\begin{minted}{glsl}
in ivec4 boneIds;
in vec4 boneWeights;
// 16 possible total bones and 1 identity transformation if not connected
uniform mat4 bones[16 + 1];

void main() {
	// A vertex can be influenced by up to 4 bones
	mat4 boneTrans = bones[boneIds.x] * boneWeights.x;
	boneTrans += bones[boneIds.y] * boneWeights.y;
	boneTrans += bones[boneIds.z] * boneWeights.z;
	boneTrans += bones[boneIds.w] * boneWeights.w;

	mat4 bonedModel = boneTrans * model;
	// ...
}
\end{minted}

\section{Tweaks to AssImp}
Since my models were based in the GlTF2 format, I had to build the Assimp library from source as
there were features such as animation that were only available in the HEAD branch and not the latest
stable release. The support was only partial, so I also found myself adding and fixing some quirks
with the library, mainly relating to animations. My changes and pull requests for them can be found
here on GitHub:

\url{https://github.com/assimp/assimp/pull/2243}
\url{https://github.com/assimp/assimp/pull/2244}

\begin{thebibliography}{9}
	\bibitem{splitsum}
		Brian Karis, Epic Games,
		\textit{Real Shading in Unreal Engine 4}, \\
		\url{https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf}
\end{thebibliography}

\end{document}
