Skip to content

Latest commit

 

History

History
353 lines (265 loc) · 15.4 KB

INSTRUCTION.md

File metadata and controls

353 lines (265 loc) · 15.4 KB

Instructions

This is due at midnight on the evening of Sun, Nov 6 2016.

Summary: In this project, you'll be introduced to the basics of deferred shading and WebGL. You'll use GLSL and WebGL to implement a forward plus shading pipeline and various lighting and visual effects.

Recommendations: Take screenshots as you go. Use them to document your progress in your README!

Read (or at least skim) the full Instructions before you begin, so that you know what to expect and what to prepare for.

Running the code

You need to launch a http server to view the content in your browser. If you have python, run the following command in your project folder.

python server.py

Then open http://localhost:10565/ in your browser.

This project requires a WebGL-capable web browser with support some extensions. You can use Chrome. Unfortunately Firefox doesn't work with depth texture. And make sure you have updated your browser and video drivers.

Requirements

Ask on the mailing list for any clarifications.

In this project, you are given code for the WebGL Forward Plus Renderer, which basically takes care of all setups like glTF model loading, shaders loading and compiling, buffer binding, etc. Your work is focusing on writing shaders in GLSL 1.00 that runs on GPU.

Required Tasks

  • Write the light debug view

    • glsl/lightDebug.vert.glsl
    • glsl/lightDebug.frag.glsl
    • lightDebug.js (Simply uncomment code)
    • draw lights as points to show their position for debug purpose

  • Write a naive forward shader

    • glsl/forward.vert.glsl
    • glsl/forward.frag.glsl
    • this is used for warmup you to get familiar to webgl and glsl.

  • Write a naive forward plus shader with

    • glsl/lightCulling.vert.glsl: view frustum light Culling on gpu
    • glsl/lightCulling.frag.glsl: view frustum light Culling on gpu
      • Find which tile this pixel belongs to
      • Each pixel represents a light. Light idx = pixel idx in its tile.
      • calculate the view frustum box of this tile. [3]
      • write the gl_FragColor a color depending on if the light hits the frustum box
    • glsl/lightAccumulation.vert.glsl: light Accumulation (forward shading)
    • glsl/lightAccumulation.frag.glsl: light Accumulation (forward shading)

  • Using blinn-phong shading (diffuse + specular) for point lights

    • With normal mapping (code provided)

Extra Tasks

You must do at least 4 points worth of extra features (effects or optimizations/analysis).

Effects:

  • (3pts) Bloom using post-process blur (box or Gaussian) [1]

    • This needs to add one more pass at the end as post processing, and render the lightAccumulation to a texture by binding a framebuffer object.
    • Recommended approach:
      • add shaders bloom.vert.glsl, bloom.frag.glsl in glsl/
      • add a post-process pass js file in js/forwardPlusRenderer/pass/ to handle shader loading, buffer and uniform binding issues. Use other existing passes as examples
      • call the post-process pass with renderFullQuad in forwardPlusPipeline. Also bind a frame buffer to the lightAccumulation pass.
    • You can add other post processing effect following this manner

  • (2pts) Toon shading (with ramp shading + simple depth-edge detection for outlines)

  • (3pts) Screen-space motion blur (blur along velocity direction) [2]

  • (1pts) Add a transparent object draw pass

    • This can use the lightAccumulation program. What you need to do is to enable blend and add one more draw pass at the end.
    • You can draw some different model in transparent pass.

Optimizations/Analysis:

  • (1pts) write tileLightDebug.frag.glsl to get a debug view of light heat map, showing how many lights are there in each tile.

  • (1pts) early z termination with depth prepass

    • Depth prepass is already implemented for you. Simply integrate it into your pipeline.

  • (1pts) Performance analysis on different tile size

    • keep in mind that maximum number of lights in the scene in our implementation is dependent on tile size.

  • (2pts) Fully utilize rgba channel of tileLightsTexture

    • so that we can allow tile size x4 lights in the scene

  • (3pts) frustum box with tile min max depth

    • We only cull light using left, right, top, bottom faces of a view frustum
    • In the original forward-plus paper, min and max depth of a tile is also used for light culling
    • You can let every thread loop through each pixel in its tile of the depth texture, or do a tile min max depth prepass and output a texture first.
    • Be sure to do a performance analysis

  • (3pts) Two-pass Gaussian blur using separable convolution (using a second postprocess render pass) to improve bloom or other 2D blur performance

  • (4pts) More efficient way of view frustum test detection.

    • accurate light culling is expensive. But false positive will greatly reduce the performance.
    • show how many false positive does our naive frustum hittest have. show how many after optimization.
    • see [3] and [5]

  • (3pts) Compare performance to equivalently-lit forward shading and deferred-shading:

    • You can pair with a classmate choosing to do deferred-renderer.

This extra feature list is not comprehensive. If you have a particular idea that you would like to implement, please contact us first (preferably on the mailing list).

Where possible, all features should be switchable using the GUI panel in ui.js.

Performance & Analysis

Optimize your JavaScript and/or GLSL code. Chrome profiling tools (see Resources section) will be useful for this. For each change that improves performance, show the before and after render times.

For each new effect feature (required or extra), please provide the following analysis:

  • Concise overview write-up of the feature.
  • Performance change due to adding the feature.
    • If applicable, how do parameters (such as number of lights, etc.) affect performance? Show data with simple graphs.
      • Show timing in milliseconds, not FPS.
  • If you did something to accelerate the feature, what did you do and why?
  • How might this feature be optimized beyond your current implementation?

For each performance feature (required or extra), please provide:

  • Concise overview write-up of the feature.
  • Detailed performance improvement analysis of adding the feature
    • What is the best case scenario for your performance improvement? What is the worst? Explain briefly.
    • Are there tradeoffs to this performance feature? Explain briefly.
    • How do parameters (such as number of lights, tile size, etc.) affect performance? Show data with graphs.
      • Show timing in milliseconds, not FPS.
    • Show debug views when possible.
      • If the debug view correlates with performance, explain how.

Starter Code Tour

  • js/: JavaScript source code for the renderer
    • main.js: Entrance: Handles initialization of other parts of the program.
    • ui.js: Defines the UI using dat.GUI.
      • The FPR.cfg can be accessed anywhere in the code to read configuration values.
    • forwardPlusRenderer/
      • scene.js: Loads and organize the scene structure.
      • shaders.js: Loads shader files
      • lights.js: manage light and light position, color radius texture
      • forwardPlusRenderer.js: the main renderer manager, class object named FPR or ForwardPlusRenderer, handles the all pipelines, aggregate everything
      • pass/: different passes in pipeline, they corresponds to FPR.pass.*
        • depthPrepass: draw the scene, store the depth in a texture. First step of Forward-Plus
        • depthDebug(No such file, but there's a depthDebug pass): debug view: draw the depth texture
        • forward: naive forward shading pipeline. Isolated from other pass
        • lightCulling: test light and view frustum box(tile) overlapping. Second step of Forward-Plus
          • To get a correct rendering result, it is okay to have false positive (judging every light is overlapping with this tile)
          • To get a better performance, we need to try to avoid false positive
        • lightAccumulation: shade current pixel with light overlapping with current cell. Final step of Forward-Plus
          • It's just naive forward with knowing which light is visible
        • tileLightDebug: debug view: draw a full quad, showing how many lights are there in each tile with color scales
        • lightDebug: debug view: draw light in the scene as point to show their position.
  • glsl/: GLSL Shader source code
    • passName.vert.glsl and passName.frag.glsl forms the shader program for FPR.pass.passName
  • lib/: JavaScript libraries.
  • models/: glTF models for testing. Sponza is the default.
  • index.html: Main HTML page.
  • server.bat (Windows) or server.py (OS X/Linux): Runs a web server at localhost:10565.

For editing JavaScript, you can use a simple editor with syntax highlighting such as Atom, VS-Code, Sublime, Vim, Emacs, etc., or the editor built into Chrome.

Pipelines

You can always switch between pipelines using the gui on the top right corner.

The Forward Shading Pipeline

FPR.pipeline.forwardPipeline:

Renders the scene geometry and perform lighting

  • forward.vert.glsl
  • forward.frag.glsl
    • loop through each light, get their info stored in the u_lightPositionTexture, u_lightColorRadiusTexture and perform blinn phong.

The Forward Plus Shading Pipeline

FPR.pipeline.forwardPlusPipeline:

Pass 1: (Optional) Depth Prepass

  • depthPrepass.vert.glsl, depthPrepass.frag.glsl
  • FPR.pass.depthPrepass
  • The framebuffer object FPR.pass.depthPrepass.framebuffer must be bound during this pass.
  • Renders into FPR.pass.depthPrepass.depthTexture

Pass 2: Light Culling

  • lightCulling.vert.glsl, lightCulling.frag.glsl
  • FPR.pass.lightCulling
  • The paper uses compute shader to do light culling, which is the idea case for desktop API like D3D and OpenGL. Unfortunately WebGL has no compute shader and no support for uniform buffer. So we use texture as a work around, which brings some limitation and performance drop.
  • Actually we are not rendering any thing. We are writing value to buffer (texture in this project).
  • For each pixel, we find the tile it belongs to. Next we compute the view frustum box of this tile. We then test if this box overlaps with the AABB of the light that this pixel is representing (lightIdx = id of pixel in this tile)
  • Input: FPR.light.positionTexture, FPR.light.colorRadiusTexture, FPR.pass.depthPrepass.depthTexture (Optional)
  • Output: FPR.pass.lightCulling.tileLighitTexture
  • FPR.pass.lightCulling.tileLightsFB must be bound.

Pass 3: Light Accumulation

  • lightAccumulation.vert.glsl, lightAccumulation.frag.glsl
  • FPR.pass.lightAccumulation
  • Input: FPR.light.positionTexture, FPR.light.colorRadiusTexture,FPR.pass.lightCulling.tileLighitTexture, FPR.pass.depthPrepass.depthTexture (Optional)
  • Output: Renders directly to the screen as a normal fragment shader does
  • The different from a naive forward fragment shader is that we know which lights are visible for current tile
  • loop through each light, if it is visible, get their info stored in the u_lightPositionTexture, u_lightColorRadiusTexture and perform blinn phong.

There are also some debug passes and pipelines to help you.

  • FPR.pass.lightDebug debug view: draw light in the scene as point to show their position. It can be appended at the end of every pipeline.
  • FPR.pass.tileLightDebug debug view: draw a full quad, showing how many lights are there in each tile with color scales.

Debugging

If there is a WebGL error, it will be displayed on the developer console and the renderer will be aborted. To find out where the error came from, look at the backtrace of the error (you may need to click the triangle to expand the message). The line right below wrapper @ webgl-debug.js will point to the WebGL call that failed.

When working in the early pipeline (before you have a lit render), Test step by step. Make sure you read the light position / color radius texture correctly. lightDebug pass and tileLightDebug can help you confirm you do read them correctly. Also you can try unbind the Frame buffer for lightCulling stage to render on the screen to things how's it going.

Resources

Also see: The articles linked in the course schedule.

Profiling and debugging tools

Built into Chrome:

  • JavaScript debugger and profiler

Plug-ins:

  • Web Tracing Framework Does not currently work with multiple render targets, which are used in the starter code.
  • (Chrome) Shader Editor

Libraries:

  • Stats.js (already included)

README

Replace the contents of this README.md in a clear manner with the following:

  • A brief description of the project and the specific features you implemented.
  • At least one screenshot of your project running.
  • A 30+ second video (or gifs) of your project running showing all features. (Even though your demo can be seen online, using multiple render targets means it won't run on many computers. A video will work everywhere.)
  • A performance analysis

Performance Analysis

See above.

GitHub Pages

Since this assignment is in WebGL, you can make your project easily viewable by taking advantage of GitHub's project pages feature.

Once you are done with the assignment, create a new branch:

git branch gh-pages

Push the branch to GitHub:

git push origin gh-pages

Now, you can go to <user_name>.github.io/<project_name> to see your renderer online from anywhere. Add this link to your README.

Submit

Beware of any issues discussed on the Google Group.

Open a GitHub pull request so that we can see that you have finished. The title should be "Project 5A: YOUR NAME". The template of the comment section of your pull request is attached below, you can do some copy and paste:

  • Repo Link
  • Your PENNKEY
  • (Briefly) Mentions features that you've completed. Especially those bells and whistles you want to highlight
    • Feature 0
    • Feature 1
    • ...
  • Feedback on the project itself, if any.

Third-Party Code Policy

  • Use of any third-party code must be approved by asking on our mailing list.
  • If it is approved, all students are welcome to use it. Generally, we approve use of third-party code that is not a core part of the project. For example, for the path tracer, we would approve using a third-party library for loading models, but would not approve copying and pasting a CUDA function for doing refraction.
  • Third-party code MUST be credited in README.md.
  • Using third-party code without its approval, including using another student's code, is an academic integrity violation, and will, at minimum, result in you receiving an F for the semester.