--- /dev/null
+\documentclass{article}
+\usepackage{fullpage}
+\usepackage{graphicx}
+\usepackage{minted}
+\begin{document}
+\title{CS4052 Project - Physically Based Rendering}
+\author{Luke Lau 15336810}
+\maketitle
+
+
+\section{Physically Based Rendering}
+The project uses a physically based lighting model, which computes per pixel lighting by estimating
+how light rays interact with surfaces at a microscopic level, called the microfacet model.
+
+\begin{center}
+\includegraphics[width=7cm]{microfacets_light_rays}
+\includegraphics[width=3.5cm]{surface_reaction}
+\end{center}
+
+There are two main properties that determine a material's appearance: roughness and metalness.
+The former determines how much light gets caught in the bumps of the surface and doesn't get
+reflected back. The latter is used for approximating how metals absorb refracted light without
+scattering.
+
+\begin{center}
+\end{center}
+
+There are 7 texture maps used when rendering each material. 4 are defined in the model and 3 are
+precomputed at the start of the program.
+\begin{minted}{glsl}
+uniform sampler2D albedoMap;
+uniform sampler2D normalMap;
+uniform sampler2D metallicRoughnessMap;
+uniform sampler2D aoMap;
+uniform samplerCube irradianceMap;
+uniform samplerCube prefilterMap;
+uniform sampler2D brdfMap;
+\end{minted}
+
+The Albedo map provides the base colour underneath the diffuse and specular layers, the normal
+map is just a standard normal map, and similarly for the ambient occlusion (ao) map.
+
+The model in the program is loaded from the GlTF2 format, a trendy new model format designed for
+easy transportation of physically based models and materials, maintained by the Khronos group (the
+same group behind OpenGL and Vulkan).
+Because it is physically based, it defines a metallic roughness map texture.
+The metallic value is stored in the blue channel of the texture whilst the roughness value is stored in the green channel (Inside the source code there is a small python script,
+\texttt{metallicRoughness.py} which combines two separate metallic/roughness maps into one suitable
+for the GlTF2 format. AssImp recently added partial support for GlTF2 models so I was able to read
+in the metallic/roughness texture into a 2 channel texture, and access it from within the shader
+like so:
+
+\begin{minted}{glsl}
+float metallic = texture(metallicRoughnessMap, texCoords).b;
+float roughness = texture(metallicRoughnessMap, texCoords).g;
+\end{minted}
+
+The irradiance, prefilter and BRDF maps are all precomputed based off the skybox in order to provide
+accurate, image based lighting (IBL) that correctly reflects the contents of the environment. The
+skybox itself is loaded in from an equirectangular HDR file, and so needs to be gamma corrected and
+converted to a cubemap first. \texttt{skyboxfrag.glsl} handles the gamma
+correction and \texttt{equirectangularfrag.glsl} takes care of converting the projection.
+
+It needs to be in the HDR format since the precomputed maps are based off of physical properties,
+and the equations that are involved work in terms of flux.
+
+\begin{figure}
+ \begin{equation}
+ L_0(p,\omega_0) =
+ \int_\Omega(k_d \frac{c}{\pi} +
+ k_s \frac{DFG}{4(\omega_0 \cdot n)
+ (\omega_i \cdot n)})
+ L_i(p,\omega_i)n \cdot \omega_i d\omega_i
+ \end{equation}
+ \label{eq:reflectance}
+ \caption{The reflectance equation}
+\end{figure}
+
+The irradiance map solves the $k_d \frac{c}{\pi}$ (diffuse) part of figure~\ref{eq:reflectance}, and is
+calculated in \texttt{irradiancefrag.glsl}. It approximates the integral by taking a finite number
+of samples, and does this for each possible incoming angle of light, storing the result into a
+cubemap texture that can easily be looked up later.
+
+The same idea applies for the prefilter and BRDF maps: solve a computationally expensive integral
+and shove it into a texture that can be quickly referenced in the main fragment shader. They
+precompute the integrals in the $k_s$ (specular) side of the figure~\ref{eq:reflectance}, but unlike
+the $k_d$ term, this term is not constant and depends on the viewing angle, $\omega_i$. It would be
+crazy to precompute this integral for every single combination of viewing angle and incoming light
+angle, so instead this is approximated with the split-sum-approximation\cite{splitsum}. This results
+in the equation in figure~\ref{eq:splitsum}, which has two terms: a filter for convoluting
+(\texttt{prefilterfrag.glsl}), and the BRDF equation (\texttt{brdffrag.glsl}).
+
+\begin{figure}
+ \begin{equation}
+ L_o(p,\omega_o) = \int_\Omega L_i(p, \omega_i) d \omega_i \ast
+ \int_\Omega f_r(p, \omega_i, \omega_o)n \cdot \omega_i d \omega_i
+ \end{equation}
+ \label{eq:splitsum}
+ \caption{The split sum approximation}
+\end{figure}
+
+The nice thing about generating all these maps is that they can all reuse the same vertex shader,
+\texttt{skyboxvert.glsl}, since they all are cubemap shaped!
+\texttt{pbrfrag.glsl} then combines all the maps to give the final result.
+
+\begin{figure}
+ \centering
+ \includegraphics[width=0.3\textwidth]{skyboxLoft}
+ \includegraphics[width=0.3\textwidth]{skyboxDesert}
+ \includegraphics[width=0.3\textwidth]{skyboxFactory}
+ \caption{Different skyboxes having different effects on lighting. Note how this lighting model
+ gives reflection for free in the leftmost sphere.}
+\end{figure}
+
+\begin{figure}
+ \centering
+ \includegraphics[width=0.45\textwidth]{gunModel}
+ \includegraphics[width=0.45\textwidth]{bulletReflection}
+ \caption{A complex, PBR based model. Even the bullets reflect the environment!}
+\end{figure}
+
+\begin{figure}
+ \centering
+ \includegraphics[width=0.45\textwidth]{normalMaps}
+ \includegraphics[width=0.45\textwidth]{normalMaps2}
+ \caption{Normal mapping and ambient occlusion providing depth to the materials.}
+\end{figure}
+
+
+\begin{figure}
+ \centering
+ \includegraphics[width=\textwidth]{fresnelEffect}
+ \caption{The Fresnel effect, where the edges of the sphere that have a sharper viewing angle
+ cause the light to be reflected more strongly.}
+\end{figure}
+
+\section{Bone Based Animation}
+
+Since I wanted to take advantage of the GlTF2 model format as much as possible, I
+decided to load in the animations via the Assimp library. Many models typically describe their
+animations via a skeletal bone system, where each vertex has a weight associated with each bone in
+the model, affecting how much it is transformed by when the corresponding bone moves.
+
+The first step of this was to add bones to the model I had created in blender. I created two bones,
+one for each swinging sphere, and then assigned the vertex weights of the two spheres and the bottom
+vertices of the ropes to their respective bones.
+
+The bones could then be animated using keyframes. By specifying their resting position and apex, I
+could then use Bezier curves to interpolate between them in such a way that gave the appearance of
+momentum. Thankfully, the GlTF2 exporter in Blender bakes the Bezier interpolation between the
+keyframes into many subdivided keyframes, so I was able to just linearly interpolate between them in
+my program.
+
+\begin{figure}
+ \centering
+ \includegraphics[width=0.45\textwidth]{blenderBone}
+ \includegraphics[width=0.45\textwidth]{blenderDopeSheet}
+ \caption{Adding bones and animating them in Blender.}
+\end{figure}
+
+The program needed to calculate the transformation of each bone at each frame in time of the
+animation, and bind the transformations into a uniform array of transformation matrices. The vertex
+shader is then able to use these matrices, alongside the weights for each vertex to calculate its
+new animated position:
+
+\begin{minted}{glsl}
+in ivec4 boneIds;
+in vec4 boneWeights;
+// 16 possible total bones and 1 identity transformation if not connected
+uniform mat4 bones[16 + 1];
+
+void main() {
+ // A vertex can be influenced by up to 4 bones
+ mat4 boneTrans = bones[boneIds.x] * boneWeights.x;
+ boneTrans += bones[boneIds.y] * boneWeights.y;
+ boneTrans += bones[boneIds.z] * boneWeights.z;
+ boneTrans += bones[boneIds.w] * boneWeights.w;
+
+ mat4 bonedModel = boneTrans * model;
+ // ...
+}
+\end{minted}
+
+\section{Tweaks to AssImp}
+Since my models were based in the GlTF2 format, I had to build the Assimp library from source as
+there were features such as animation that were only available in the HEAD branch and not the latest
+stable release. The support was only partial, so I also found myself adding and fixing some quirks
+with the library, mainly relating to animations. My changes and pull requests for them can be found
+here on GitHub:
+
+https://github.com/assimp/assimp/pull/2243
+https://github.com/assimp/assimp/pull/2244
+
+\begin{thebibliography}{9}
+ \bibitem{splitsum}
+ Brian Karis, Epic Games,
+ \textit{Real Shading in Unreal Engine 4}, \\
+ https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf
+\end{thebibliography}
+
+\end{document}
projects / opengl.git / blobdiff