Project 5: Kaixiang Miao

INSTRUCTION.md

@@ -303,7 +303,7 @@ UI is accessible anywhere as `cfg.enableEffect0`, etc.
 
 **Pass 3:** Performs post-processing.
 * `quad.vert.glsl`, `post/one.frag.glsl`
-* Takes `pass_deferred.colorTex` as a texture input `u_color`.
+* Takes `pass_BlinnPhong_PointLight.colorTex` as a texture input `u_color`.
 * Renders directly to the screen if there are no additional passes.
 
 More passes may be added for additional effects (e.g. combining bloom with

Performance.txt

@@ -0,0 +1,15 @@
+use 2 gbuffer, calculate world position in shader:
+33
+
+compute normal in copy:
+29
+
+Scissor Test gl.scissor:
+24
+
+Scissor Test NULL:
+16
+
+Without Scissor Test:
+13
+

README.md

@@ -3,27 +3,123 @@ WebGL Deferred Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Kaixiang Miao
+* Tested on: Windows 7, i7-3630QM @ 2.40GHz 8GB, GTX 660M 2GB (Lenovo Y580 laptop, personal computer)
 
 ### Live Online
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+[![](img/index.jpg)](https://immiao.github.io/Project5-WebGL-Deferred-Shading-with-glTF/)
 
 ### Demo Video/GIF
 
-[![](img/video.png)](TODO)
+![](img/72.gif)
 
-### (TODO: Your README)
+### Features
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+* Blinn-Phong shading model
+* Optimized lights rendering by scissor test
+* Optimized by using less G-Buffers (reduce from 4 to 2)
+* Toon shading (ramp shading && edge detection)
+* Bloom effect (Gaussian blur && ping-pong frame buffers)
+* Screen space motion blur
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+### G-Buffers
+<table class="image">
+<tr>
+	<td>depth buffer</td>
+	<td>geometry normal</td>
+	<td>normal (tangent space)</td>
+</tr>
+<tr>
+	<td><img src="img/depth.jpg"/></td>
+	<td><img src="img/geometryNormal.jpg"/></td>
+	<td><img src="img/normalTangentSpace.jpg"/></td>
+</tr>
+</table>
 
+### Effects
+<table class="image">
+<tr>
+	<td>non-toon shading</td>
+	<td>toon shading</td>
+</tr>
+<tr>
+	<td><img src="img/nontoon.jpg"/></td>
+	<td><img src="img/toon.jpg"/></td>
+</tr>
+</table>
 
+<table class="image">
+<tr>
+	<td>no bloom</td>
+	<td>bloom</td>
+</tr>
+<tr>
+	<td><img src="img/nonbloom.jpg"/></td>
+	<td><img src="img/bloom.jpg"/></td>
+</tr>
+</table>
+
+<table class="image">
+<tr>
+	<td>motion blur</td>
+</tr>
+<tr>
+	<td><img src="img/71.gif"/></td>
+</tr>
+</table>
+
+#### Scissor Test
+<table class="image">
+<tr>
+	<td>Scissor Area</td>
+</tr>
+<tr>
+	<td><img src="img/scissor.jpg"/></td>
+</tr>
+</table>
+
+### Analysis
+
+* Using less G-Buffers
+	* calculate the world-space normal in the copy stage
+	* convert the screen-space point to world-space point using the depth value in the depth buffer and the camera transformation matrix
+
+FPS is increased:
+	![table1](img/table1.jpg)
+
+* Scissor Test
+
+While rendering the point-light effect, it's unnecessary to render all screen since point lights have their own radius. The part which is outside of the circle of the point lights can be cut.
+
+Scissor Test increases the FPS significantly.
+
+Without scissor test, the average FPS is **13**.
+
+If only discarding the lights whose scissor is off the screen, the average FPS is increased to **16**.
+
+If we perform scissor test to cut the parts which are out of the rectangular, the average FPS is **24**.
+
+* Toon shading
+
+Implemented with ramp shading and edge detection.
+
+Reference: [Toon shading](http://www.lighthouse3d.com/tutorials/glsl-12-tutorial/toon-shading-version-i/)
+
+* Bloom effect
+
+Bloom gives noticeable visual cues about the brightness of objects as bloom tends to give the illusion objects are really bright. When done in a subtle fashion (which some games drastically fail to do) bloom significantly boosts the lighting of the scene and allows for a large range of dramatic effects.
+
+The idea is to extract the bright color and do a 2-pass Gaussian blur first, then merge the blurred image to the scene, which means we should use framebuffer to temporarily store the color.
+
+Reference: [Advanced Lighting: Bloom](http://learnopengl.com/#!Advanced-Lighting/Bloom)
+
+* Screen space motion blur
+
+store the previous camera transformation matrix
+
+Reference: [Motion Blur as a Post-Processing Effect](http://http.developer.nvidia.com/GPUGems3/gpugems3_ch27.html)
+
 ### Credits
 
 * [Three.js](https://github.com/mrdoob/three.js) by [@mrdoob](https://github.com/mrdoob) and contributors

glsl/copy.frag.glsl

@@ -10,11 +10,26 @@ varying vec3 v_position;
 varying vec3 v_normal;
 varying vec2 v_uv;
 
+vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
+    normap = normap * 2.0 - 1.0;
+    vec3 up = normalize(vec3(0.001, 1, 0.001));
+    vec3 surftan = normalize(cross(geomnor, up));
+    vec3 surfbinor = cross(geomnor, surftan);
+    return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor;
+}
+
 void main() {
     // TODO: copy values into gl_FragData[0], [1], etc.
     // You can use the GLSL texture2D function to access the textures using
     // the UV in v_uv.
 
+    vec3 normap = texture2D(u_normap, v_uv).xyz;
+    vec3 nor = applyNormalMap(v_normal, normap); 
     // this gives you the idea
-    // gl_FragData[0] = vec4( v_position, 1.0 );
+    //gl_FragData[2] = vec4( v_position, 1.0 );
+    gl_FragData[0] = vec4( nor, 1.0);
+    gl_FragData[1] = texture2D(u_colmap, v_uv);
+
+    //gl_FragData[3] = texture2D(u_normap, v_uv);
+    //gl_FragData[4] = 
 }

glsl/deferred/ambient.frag.glsl

@@ -3,7 +3,7 @@
 precision highp float;
 precision highp int;
 
-#define NUM_GBUFFERS 4
+#define NUM_GBUFFERS 2
 
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
 uniform sampler2D u_depth;
@@ -13,8 +13,7 @@ varying vec2 v_uv;
 void main() {
     vec4 gb0 = texture2D(u_gbufs[0], v_uv);
     vec4 gb1 = texture2D(u_gbufs[1], v_uv);
-    vec4 gb2 = texture2D(u_gbufs[2], v_uv);
-    vec4 gb3 = texture2D(u_gbufs[3], v_uv);
+    //vec4 gb2 = texture2D(u_gbufs[2], v_uv);
     float depth = texture2D(u_depth, v_uv).x;
     // TODO: Extract needed properties from the g-buffers into local variables
 
@@ -23,5 +22,8 @@ void main() {
         return;
     }
 
-    gl_FragColor = vec4(0.1, 0.1, 0.1, 1);  // TODO: replace this
+    float ambient = 0.35;
+    //gl_FragColor = vec4(0.1, 0.1, 0.1, 1);  // TODO: replace this
+    //gl_FragColor = vec4(1, 1, 1, 1);
+    gl_FragColor = vec4(gb1.rgb * ambient, 1.0);
 }

glsl/deferred/blinnphong-pointlight.frag.glsl

@@ -2,16 +2,22 @@
 precision highp float;
 precision highp int;
 
-#define NUM_GBUFFERS 4
+#define NUM_GBUFFERS 2
 
 uniform vec3 u_lightCol;
 uniform vec3 u_lightPos;
 uniform float u_lightRad;
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
 uniform sampler2D u_depth;
 
+uniform int u_isToon;
+
 varying vec2 v_uv;
 
+uniform vec3 u_cameraPos;
+uniform mat4 u_invCrtCameraMat;
+//uniform sampler2D u_tex;
+
 vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
     normap = normap * 2.0 - 1.0;
     vec3 up = normalize(vec3(0.001, 1, 0.001));
@@ -21,19 +27,82 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
 }
 
 void main() {
+    vec3 result = vec3(0, 0, 0);
+
+    float zOverW = texture2D(u_depth, v_uv).x * 2.0 - 1.0;
+    vec4 HH = vec4(v_uv.x * 2.0 - 1.0, v_uv.y * 2.0 - 1.0, zOverW, 1.0);
+    vec4 DD = u_invCrtCameraMat * HH;
+    vec4 worldPos = DD / DD.w;
+
     vec4 gb0 = texture2D(u_gbufs[0], v_uv);
     vec4 gb1 = texture2D(u_gbufs[1], v_uv);
-    vec4 gb2 = texture2D(u_gbufs[2], v_uv);
-    vec4 gb3 = texture2D(u_gbufs[3], v_uv);
+    //vec4 gb2 = texture2D(u_gbufs[2], v_uv);
+    //vec4 gb3 = texture2D(u_gbufs[3], v_uv);
     float depth = texture2D(u_depth, v_uv).x;
     // TODO: Extract needed properties from the g-buffers into local variables
 
+    //vec3 pos = gb0.xyz;
+    vec3 pos = worldPos.xyz;
+    //vec3 pos = vec3(0, 0, 0);
+    //vec3 geomnor = gb1.xyz;
+    vec3 colmap = gb1.rgb;
+    //vec3 normap = gb3.xyz;
+    //vec3 nor = applyNormalMap(geomnor, normap);
+    vec3 nor = gb0.xyz;
+
+
+    vec3 L = normalize(u_lightPos - pos);
+    vec3 V = normalize(u_cameraPos - pos);
+    vec3 H = normalize(V + L);
+    float distance = length(u_lightPos - pos);
+    float attenuation = max(0.0, 1.0 - distance / u_lightRad);
+    float intensity = dot(nor, L);
+    vec3 diffuseColor;
+    vec3 specularColor = u_lightCol * pow(max(0.0, dot(nor, H)), 200.0);
+    if (u_isToon == 1)
+    {
+        int edgeDetection = dot(V, nor) > 0.1 ? 1 : 0;
+        if (intensity > 0.9)
+        {
+            diffuseColor = colmap * 0.9 * u_lightCol * 0.9;
+        }
+        else if (intensity > 0.7)
+            diffuseColor = colmap * 0.7 * u_lightCol * 0.7;
+        else if (intensity > 0.4)
+        {
+            diffuseColor = colmap * 0.4 * u_lightCol * 0.4;
+        }
+        else
+        {
+            diffuseColor = colmap * 0.1 * u_lightCol * 0.1;
+        }
+
+        if (pow(max(0.0, dot(nor, H)), 200.0) < 0.4)
+            specularColor = vec3(0, 0, 0);
+        if (edgeDetection == 0)
+        {
+            diffuseColor = vec3(0, 0, 0);
+            specularColor = vec3(0, 0, 0);
+        }
+    }
+    else
+    {
+        diffuseColor = u_lightCol * clamp(intensity, 0.0, 1.0);
+        //specularColor = u_lightCol * pow(max(0.0, dot(nor, H)), 200.0);
+    }
+
     // If nothing was rendered to this pixel, set alpha to 0 so that the
     // postprocessing step can render the sky color.
     if (depth == 1.0) {
         gl_FragColor = vec4(0, 0, 0, 0);
         return;
     }
-
-    gl_FragColor = vec4(0, 0, 1, 1);  // TODO: perform lighting calculations
+    result += (diffuseColor + specularColor) * attenuation;
+    gl_FragColor = vec4(result, 1.0);
+    //gl_FragColor = vec4(gb0.xyz, 1.0);
+    //gl_FragColor = worldPos;
+    //gl_FragColor = vec4(specularColor, 1.0);
+    //gl_FragColor = vec4(u_cameraPos, 1.0);
+    //gl_FragColor = vec4(attenuation, 0, 0, 1.0);
+    //gl_FragColor = vec4(worldPos.y, 0, 0, 1.0);
 }

glsl/deferred/debug.frag.glsl

@@ -33,20 +33,23 @@ void main() {
     vec3 colmap = gb2.rgb;  // The color map - unlit "albedo" (surface color)
     vec3 normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
     vec3 nor = applyNormalMap (geomnor, normap);     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
-
+    if (gb2.a == 0.0) {
+        gl_FragColor = SKY_COLOR;
+        return;
+    }
     // TODO: uncomment
     if (u_debug == 0) {
         gl_FragColor = vec4(vec3(depth), 1.0);
     } else if (u_debug == 1) {
-        // gl_FragColor = vec4(abs(pos) * 0.1, 1.0);
+        gl_FragColor = vec4(pos, 1.0);
     } else if (u_debug == 2) {
-        // gl_FragColor = vec4(abs(geomnor), 1.0);
+        gl_FragColor = vec4(abs(geomnor), 1.0);
     } else if (u_debug == 3) {
-        // gl_FragColor = vec4(colmap, 1.0);
+        gl_FragColor = vec4(colmap, 1.0);
     } else if (u_debug == 4) {
-        // gl_FragColor = vec4(normap, 1.0);
+        gl_FragColor = vec4(normap, 1.0);
     } else if (u_debug == 5) {
-        // gl_FragColor = vec4(abs(nor), 1.0);
+        gl_FragColor = vec4(abs(nor), 1.0);
     } else {
         gl_FragColor = vec4(1, 0, 1, 1);
     }