Final Post

Final Report can be found here

Final Post

Overview of Application:

Features highlights:
- load different obj models
- camera position control
- light position control
- different pencil textures
- paper textures
- Color overlay
- texture shaking (see "torture bunny" section in video)

During preprocessing:
Generate paper normal using noise function, pencil textures, paper textures, load obj model, calculate minimum & maximum curvature of model.

During runtime process:
There are a total of 3 passes:

In the first pass, the model's normal & depth is computed and stored in a 2D texture.

In the second pass, normal/depth texture and a min/max curvature texture is used to calculate the appropriate rotation angles to transform texture coordinates for 3-way blending. We also adjust brightness to emphasize contrast between the pencil strokes.

In the third pass, using normal & depth information from first pass, apply Sobel filter to do edge detection in order to achieve contour of model. Blend with interior texture from 2nd pass. Further blend this result with the paper texture to achieve the effect of a pencil drawing drawn on textured paper.

Frame Rate:
The frame rate is real time (20-30+) for most models but sub-real time (10-20) for very large models (~40,000 polygons). Our overall frame rate may be slowed down by the large 2D textures we are using for the pencil textures (instead of 3D textures, which are much more complicated to built). We also have some if-statements & while loops that could be slowing down our frame rate.

New progress

Result of trial and error. By accident, we generate some cool images.

pop-artBoxy
Paint-Effect
Charcoal
after 3-way blending, with only one intensity level

Current Problem:

1. Cannot set up geometry shader, missing support for extensions, "GL_NV_gpu_program4" and "GL_ARB_texture_float", we instead use the multi-texture method suggested by the author. 3-way blending looks fine, the interior lines look kind of continuous.

2. Instead of using 3d-texture (we don't know how to set up in openGL and CG), we are using very large 2d-texture (with different intensity textures all laid out on one 2d-texture). The problem is we couldn't pass a texture with size larger than 1000 x 1000 into the program.

Future work:

Incorporate light intensity difference into the texture mapping. Give the model more 3-dimensional look. Create more pencil textures and move the preprocessing part out. Give user mouse control.

Finally heard back from the author with explanations on 3-way blending:

"I rendered a triangle mesh face-by-face (as usual in OpenGL, but DirectX normally requires a different approach) and I set three min-curvature directions (because each face has 3 vertices) for all vertices for each face. Here I attach some part of my source code:

glBegin(GL_TRIANGLES);

// set curvature values as texture coordinates.
glMultiTexCoord3fv(GL_TEXTURE0, face.getVertex(0).getTexCoord

(0).v); // min-curvature directions
glMultiTexCoord3fv(GL_TEXTURE1, face.getVertex(1).getTexCoord(0).v);
glMultiTexCoord3fv(GL_TEXTURE2, face.getVertex(2).getTexCoord(0).v);

glMultiTexCoord3fv(GL_TEXTURE3, face.getVertex(0).getTexCoord(1).v); // max-curvature directions
glMultiTexCoord3fv(GL_TEXTURE4, face.getVertex(1).getTexCoord(1).v);
glMultiTexCoord3fv(GL_TEXTURE5, face.getVertex(2).getTexCoord(1).v);

for(int k = 0; k < 3; k++)
glVertex3fv(face.getVertex(k).getPosition().v);

...

glEnd();

In this code, face.getVertex(i) retrieves the vertex index of the face and getTexCoord(j) retrieves the texture coordinate of the vertex. And I used multi-texturing to set three min-curvature directions for each face.
Therefore, all vertices in a face have three texture coordinates whose values are min-curvature directions.
(Also have three max-curvature directions, total 6.)"

Current Progress

Current Progress and Problems

Progress :

We finished the whole program set-up and figured out how to do multi-pass on GPU using CG. We based our project on the third assignment, modify the program structure to incorporate the following functionalities.

1. Half-edge Mesh data structure : load obj model, compute vertex normal, compute principle curvature direction, display normal and principle curvature direction for debugging. (Cynthia)
2. Created Small set of pencil textures, prepared to do texture mapping (Connie)
3. Incorporated image loader to load pencil/paper textures into the program and passed them to GPU (Cynthia)
4. Implemented sign function distortion method currently distort the whole texture for the screen which will be used for contour shaking(Connie)

Problem:

1. Do not quite understand the principle direction concept. Not sure whether it is computed correctly.
2. Unsure about how to do 3-way blending of pencil texture to the interior. Confused about the wording on the paper.
To be specific, in the pixel shader they propose to use three different angles to rotate pencil textures and then blend them together, but how can we get the three angles? If you consider passing the computed angle from vertex shader, the problem is at a single instance in time, the vertex shader is only computing one angle for a single vertex, when it passes the angle to the pixel shader, the pixel shader will only get one angle value (either interpolated or not interpolated, not sure)

plan for the near future:

Continue to figure out the 3-way blending. Start to implement run-time processes:

1. generate normal and depth map
2. contour detection using the normal and depth map
3. contour shaking / multiple line drawing

fresh start

Last week, we have been busy reading the paper over and over and researching in multipass programming. We have completed a so called "multipass" assignment, however it is not actual multipass on GPU (one pass on GPU and another one on CPU). The lack of good debugging tool for CG will be our major problem, it already shows before our program get larger. We have set up the basic program structure, next step will be the actual algorithm algorithm implementation.

Discussion about final project topic

We both have some sort of artistic training and highly interested in non-photorealistic rendering. We did a lot of research on ACM portal about GPU-based rendering techniques. We found several topics that interested us most. We narrowed down our selection to "Real-Time Pencil Rendering"

We both read the paper very carefully and began to discuss about more details. If time permits, we may incorporate some components of another paper "Real-Time Hatching". We decided use CG as it is up to now common to use and require low hardware specification.

We have now almost finished our final project proposal document although it is due on March 17, since we wanna enjoy our spring break. : ). Later in the semester we may need to live in HMS or Moore, who knows, hope it won't happen.

Possible papers for implementation

1. Real-Time Pencil Rendering
2. Multi-Fragment Effects on the GPU using the k-Buffer
3. GPU-based 3D Oriental Color-Ink Rendering
4. A GPU-based Method for Real-time Simulation of Eastern Painting