X-Git-Url: http://git.lukelau.me/?p=opengl.git;a=blobdiff_plain;f=assignment-5%2Freport.latex;fp=assignment-5%2Freport.latex;h=eea4b473412cc7222ecff68009698991908b23a6;hp=0000000000000000000000000000000000000000;hb=610bb8ddab4ec871cadfed0a0b66695b8fea41a4;hpb=be8759aec179d6d7bed58732134673870c596b4f

diff --git a/assignment-5/report.latex b/assignment-5/report.latex
new file mode 100644
index 0000000..eea4b47
--- /dev/null
+++ b/assignment-5/report.latex
@@ -0,0 +1,201 @@
+\documentclass{article}
+\usepackage{fullpage}
+\usepackage{graphicx}
+\usepackage{minted}
+\begin{document}
+\title{CS4052 Project - Physically Based Rendering}
+\author{Luke Lau 15336810}
+\maketitle
+
+
+\section{Physically Based Rendering}
+The project uses a physically based lighting model, which computes per pixel lighting by estimating
+how light rays interact with surfaces at a microscopic level, called the microfacet model. 
+
+\begin{center}
+\includegraphics[width=7cm]{microfacets_light_rays}
+\includegraphics[width=3.5cm]{surface_reaction}
+\end{center}
+
+There are two main properties that determine a material's appearance: roughness and metalness. 
+The former determines how much light gets caught in the bumps of the surface and doesn't get
+reflected back. The latter is used for approximating how metals absorb refracted light without
+scattering.
+
+\begin{center}
+\end{center}
+
+There are 7 texture maps used when rendering each material. 4 are defined in the model and 3 are
+precomputed at the start of the program.
+\begin{minted}{glsl}
+uniform sampler2D albedoMap;
+uniform sampler2D normalMap;
+uniform sampler2D metallicRoughnessMap;
+uniform sampler2D aoMap;
+uniform samplerCube irradianceMap;
+uniform samplerCube prefilterMap;
+uniform sampler2D brdfMap;
+\end{minted}
+
+The Albedo map provides the base colour underneath the diffuse and specular layers, the normal
+map is just a standard normal map, and similarly for the ambient occlusion (ao) map.
+
+The model in the program is loaded from the GlTF2 format, a trendy new model format designed for
+easy transportation of physically based models and materials, maintained by the Khronos group (the
+same group behind OpenGL and Vulkan).
+Because it is physically based, it defines a metallic roughness map texture.
+The metallic value is stored in the blue channel of the texture whilst the roughness value is stored in the green channel (Inside the source code there is a small python script,
+\texttt{metallicRoughness.py} which combines two separate metallic/roughness maps into one suitable
+for the GlTF2 format. AssImp recently added partial support for GlTF2 models so I was able to read
+in the metallic/roughness texture into a 2 channel texture, and access it from within the shader
+like so:
+
+\begin{minted}{glsl}
+float metallic = texture(metallicRoughnessMap, texCoords).b;
+float roughness = texture(metallicRoughnessMap, texCoords).g;
+\end{minted}
+
+The irradiance, prefilter and BRDF maps are all precomputed based off the skybox in order to provide
+accurate, image based lighting (IBL) that correctly reflects the contents of the environment. The
+skybox itself is loaded in from an equirectangular HDR file, and so needs to be gamma corrected and
+converted to a cubemap first. \texttt{skyboxfrag.glsl} handles the gamma
+correction and \texttt{equirectangularfrag.glsl} takes care of converting the projection. 
+
+It needs to be in the HDR format since the precomputed maps are based off of physical properties,
+and the equations that are involved work in terms of flux. 
+
+\begin{figure}
+	\begin{equation}
+	L_0(p,\omega_0) =
+	\int_\Omega(k_d \frac{c}{\pi} +
+				k_s \frac{DFG}{4(\omega_0 \cdot n)
+								(\omega_i \cdot n)})
+			   L_i(p,\omega_i)n \cdot \omega_i d\omega_i
+	\end{equation}
+	\label{eq:reflectance}
+	\caption{The reflectance equation}
+\end{figure}
+
+The irradiance map solves the $k_d \frac{c}{\pi}$ (diffuse) part of figure~\ref{eq:reflectance}, and is
+calculated in \texttt{irradiancefrag.glsl}. It approximates the integral by taking a finite number
+of samples, and does this for each possible incoming angle of light, storing the result into a
+cubemap texture that can easily be looked up later.
+
+The same idea applies for the prefilter and BRDF maps: solve a computationally expensive integral
+and shove it into a texture that can be quickly referenced in the main fragment shader. They
+precompute the integrals in the $k_s$ (specular) side of the figure~\ref{eq:reflectance}, but unlike
+the $k_d$ term, this term is not constant and depends on the viewing angle, $\omega_i$. It would be
+crazy to precompute this integral for every single combination of viewing angle and incoming light
+angle, so instead this is approximated with the split-sum-approximation\cite{splitsum}. This results
+in the equation in figure~\ref{eq:splitsum}, which has two terms: a filter for convoluting
+(\texttt{prefilterfrag.glsl}), and the BRDF equation (\texttt{brdffrag.glsl}).
+
+\begin{figure}
+	\begin{equation}
+		L_o(p,\omega_o) = \int_\Omega L_i(p, \omega_i) d \omega_i \ast
+		\int_\Omega f_r(p, \omega_i, \omega_o)n \cdot \omega_i d \omega_i
+	\end{equation}
+	\label{eq:splitsum}
+	\caption{The split sum approximation}
+\end{figure}
+
+The nice thing about generating all these maps is that they can all reuse the same vertex shader,
+\texttt{skyboxvert.glsl}, since they all are cubemap shaped!
+\texttt{pbrfrag.glsl} then combines all the maps to give the final result.
+
+\begin{figure}
+	\centering
+	\includegraphics[width=0.3\textwidth]{skyboxLoft}
+	\includegraphics[width=0.3\textwidth]{skyboxDesert}
+	\includegraphics[width=0.3\textwidth]{skyboxFactory}
+	\caption{Different skyboxes having different effects on lighting. Note how this lighting model
+	gives reflection for free in the leftmost sphere.}
+\end{figure}
+
+\begin{figure}
+	\centering
+	\includegraphics[width=0.45\textwidth]{gunModel}
+	\includegraphics[width=0.45\textwidth]{bulletReflection}
+	\caption{A complex, PBR based model. Even the bullets reflect the environment!}
+\end{figure}
+
+\begin{figure}
+	\centering
+	\includegraphics[width=0.45\textwidth]{normalMaps}
+	\includegraphics[width=0.45\textwidth]{normalMaps2}
+	\caption{Normal mapping and ambient occlusion providing depth to the materials.}
+\end{figure}
+
+
+\begin{figure}
+	\centering
+	\includegraphics[width=\textwidth]{fresnelEffect}
+	\caption{The Fresnel effect, where the edges of the sphere that have a sharper viewing angle
+	cause the light to be reflected more strongly.}
+\end{figure}
+
+\section{Bone Based Animation}
+
+Since I wanted to take advantage of the GlTF2 model format as much as possible, I
+decided to load in the animations via the Assimp library. Many models typically describe their
+animations via a skeletal bone system, where each vertex has a weight associated with each bone in
+the model, affecting how much it is transformed by when the corresponding bone moves.
+
+The first step of this was to add bones to the model I had created in blender. I created two bones,
+one for each swinging sphere, and then assigned the vertex weights of the two spheres and the bottom
+vertices of the ropes to their respective bones.
+
+The bones could then be animated using keyframes. By specifying their resting position and apex, I
+could then use Bezier curves to interpolate between them in such a way that gave the appearance of
+momentum. Thankfully, the GlTF2 exporter in Blender bakes the Bezier interpolation between the
+keyframes into many subdivided keyframes, so I was able to just linearly interpolate between them in
+my program.
+
+\begin{figure}
+	\centering
+	\includegraphics[width=0.45\textwidth]{blenderBone}
+	\includegraphics[width=0.45\textwidth]{blenderDopeSheet}
+	\caption{Adding bones and animating them in Blender.}
+\end{figure}
+
+The program needed to calculate the transformation of each bone at each frame in time of the
+animation, and bind the transformations into a uniform array of transformation matrices. The vertex
+shader is then able to use these matrices, alongside the weights for each vertex to calculate its
+new animated position:
+
+\begin{minted}{glsl}
+in ivec4 boneIds;
+in vec4 boneWeights;
+// 16 possible total bones and 1 identity transformation if not connected
+uniform mat4 bones[16 + 1];
+
+void main() {
+	// A vertex can be influenced by up to 4 bones
+	mat4 boneTrans = bones[boneIds.x] * boneWeights.x;
+	boneTrans += bones[boneIds.y] * boneWeights.y;
+	boneTrans += bones[boneIds.z] * boneWeights.z;
+	boneTrans += bones[boneIds.w] * boneWeights.w;
+
+	mat4 bonedModel = boneTrans * model;
+	// ...
+}
+\end{minted}
+
+\section{Tweaks to AssImp}
+Since my models were based in the GlTF2 format, I had to build the Assimp library from source as
+there were features such as animation that were only available in the HEAD branch and not the latest
+stable release. The support was only partial, so I also found myself adding and fixing some quirks
+with the library, mainly relating to animations. My changes and pull requests for them can be found
+here on GitHub:
+
+https://github.com/assimp/assimp/pull/2243
+https://github.com/assimp/assimp/pull/2244
+
+\begin{thebibliography}{9}
+	\bibitem{splitsum}
+		Brian Karis, Epic Games,
+		\textit{Real Shading in Unreal Engine 4}, \\
+		https://cdn2.unrealengine.com/Resources/files/2013SiggraphPresentationsNotes-26915738.pdf
+\end{thebibliography}
+
+\end{document}