Realtime morphing

Shades of Divinity

2005-12-14T09:16:00.000-08:00

Well, the project has been finished for a long time now.. Kinda just forgot to blog that we're finished... =)

We dubbed it "Shades of Divinity", and the final product + more info can be found at http://oskarholmstrand.com/sod/

It's all coming together

2005-03-10T06:49:00.000-08:00

One day left on the project and one week until the final presentation :)
Overall It feels pretty good, I feel like I have reached my goals with this project.
There is always more stuff to do but a "final" version of the code that will be used for the presentation is done. Later i might pack the textures and HLSL files into the .exe for easer distribution
(if someone on this planet wanna download it for some reason:).

Short summary on what's seen in the demo.

Alot of the lightning/shading data is sampled from textures to give the artist as much control of the shading as possible.

Textures
- Diffuse color
- Specular color
- Shininess
- Normal
- Reflectivity
- Height
- Light
- Attenuation

The Ground
- Normalmapped per pixel diffuse and specular lightning (Phong model)
- Parallax mapping for greater 3D feel.
- Receives soft shadows.

The Angel(s)
- Normalmapped per pixel diffuse and specular lightning (Phong model)
- Environment reflections
- Renormalised tangent space basis each pixel for accurate lightning
- Morphs between all textures (diffuse, specular, normal, shininess, reflectivity)
- Receives and generates soft shadows
- Smooth skinning
- Kochanek Bartels Hermite spline interpolation for translation track
- Spherical Linear Interpolation using Quaternions for rotation track

The spotligh
- Projected texture instead of the original spotlight model
- Attenuation map to control the fallof.

Ambient lightning
- Ambient lightning ala "half life 2" so detail from the normalmap is visible even when shadowed,
traditional ambient lightning just makes everything completely flat.
- Works by specifying an ambient cube representing the color of the light flowing from that direction in space
- Eaxh pixel rendered samples and blends between 3 of the ambient cube sided depending on It's worldspace normal

The sky
- Ordinary skybox

Shadows
- 1024x1024 shadow map
- 2x2 PCF sampled 5 times to give "soft" shadow edges.

Volumetric light/fog
- Lit depending on the lights projected texture
- Shadows from blocking objects
- Fog is generated using 3 octaves of perlin noise from a 16x16x16 texture
- Rendered at a small resolution and via "smart" filtering rescaled to fit the screen while minimizing aliasing artefacts

Morphing
- Transitionless slider based morph affecting the whole model
- Plane morph dividing the model in 2
- Lightbased morph where the morphvalue is sampled from the projected lights alpha channel.

The demo engine is coded using C++ and D3D9 in VC.2003.
All shaders is written is HLSL and some makes use of vs/ps 3.0 because of their length and branching requirements.

Maya exporter plugin
- Exports one mesh with the bound skeleton if any
- Triangulates polygons
- Splits vertices depending on hard/soft edges and texture seams
- Calculates tangents and binormals taking into account artist modified vertex normals
- Complete skeleton hierarchy with rotation and translation track exported including bindpose
- Unlimited amount of skinweights per vertex

The exporter is coded in C++ in VC.2003 using the Maya C++ API.

Getting wrapped up

2005-03-09T06:15:00.000-08:00

Well, animation is almost done. I have successfully completed the animation, but it needs some minor fixing until it feels more professional. This workflow - sketching up the whole animation first, and then finalize and perfect it - is the best. Only drawback is that people in the classroom looks over my shoulder and thinks "uh, that's not good", and when some people does this, it's kind of irritation.

Anyway. Right now the follow things needs some attention - then we're ready to go gold.

- animation
The animation needs more attention. This is actually a never-ending task, so I will most likely finish up the other things first, then work the animation until no time is left.

- model
Needs to be tesselated in some areas for better deformations. This must be done in consideration of the normalmaps, which are already generated. Perhaps I could generate new normal-maps, as my workflow allows for it... .Hmm... I placed all manual changes in layers in a .psd file, so I could change the original by just changing the background layers.... (Must be thought over.)

- skin weights
If I add more polygons, I must also paint how they are weighted.

- gui
Fix the gui so it's more personalized

- icon
I'm not happy with the icon I made. I have very fuzzy ideas about new ones, so until I can think of something more concrete, the old one will have to do.

Animation Started

2005-02-14T08:54:00.000-08:00

Have started to animate the character now. So far, about 6 seconds or so is completed. I've noticed quite some problems with my rigging and skin-painting, but I'm correcting them along the way. Hopefully, the number of seconds/day will increase as my rig gets more final, skinpainting gets done and most important - I get used to animating this specific rig.

Perlin noise

2005-02-14T08:32:00.000-08:00

Switched from creating the fogvolumes from 2 scrolling textures to a 3d perlin noise. This gives much more "realistically" looking fogvolumes and it's easier to control/predict the result.
The fogvolume in itself is made by a 16x16x16 texture (only 4k memory) stretched over an area of 64x64x64 and using 3 octaves of noise to give enough detail.
Also wrote the shaders för lightbased morphing and plane morphing in ps 3.0.
Optimized the volumetric lightning a little futher (because it's so fillrate heavy every cycle counts :) and now It's running pretty smooth.

Textures done

2005-02-11T07:44:00.000-08:00

Well, I'm pretty sure I posted a post that got lost somewhere along the way... So sorry for the slow update on the graphics part.

Anyway, this is current status:
Textures are almost done! .. Everything except the reflection map and some tweaking. The problem is that I can't see the reflection map on my current card / pixel shader implementation. In order to see it, either Fredrik would have to write a specific pixel shader in 2.0 just for me to test the result - which would not only add more work but also a pitfall if the implementations would differ. The other solution - which also solves a future problem with not being able to test animation - is to aquire a gfx card with ps3.0, i.e. a gf6800.

After some serious problems with getting our school to even respond, we finally got rejected. And then another teacher ordered the card anyway. So, textures will be 100% done pretty soon after the gfx card has been installed.

Half life 2 shading implemented :)

2005-02-09T02:07:00.000-08:00

Just added support for a skybox and it really lifts the overall feeling compared to a black background :)
Also implement "indirect lightning" ala Half Life 2 so i get rid of the ugliness that comes with normal ambient lightning.
The reason for this is when using normal ambient lightning the details from the normalmap won't show and the model just looks flat, i tried adding very faint pointligts that only affected the diffuse component but the indirect lightning implemented in half life 2 gives equally good result and it's cheaper.
It works by setting up an "ambient cube" which is 6 colors that measures the color of the light "flowing" from that direction, then depending on the surface worldspace normal i sample from 3 of the ambient cube sides and blend between them depending on the normals direction.
This gives pretty good results and the surface detail from the normalmap is visible even when shadowed.

What's new

2005-01-26T13:42:00.000-08:00

Been busy the last week because I finally got a new computer to put the 6800 in.
The card is fast, VERY fast and with SM 3.0, Ultrashadow, floating point blending etc I've been able to increase the visual quality and get a nice speedbost.
Also implemented the volumetric lightning a little different and now its very fast.
Got rid of aliasing problems regarding the volumetric light and right now I'm in the process of adding volumetric fog.
Overall it's starting to look really good when everything starts coming together :)

Back on track again,

2005-01-14T09:16:00.000-08:00

Since my last update I haven't had so much time to devote into the project because of another course we read parallel (me and a classmate made a quake3 map viewer in java for a graphics assignment).
But now I'm back on track just in time for the second project demonstration.
Today me and Oskar met and decided what to show during the demonstration.
I added some configurability to the viewer, we can now disable /enable environment mapping, volumetric lightning and plane morphing. We can also switch textures for the projected light and control some animation of it. Everything to make it easier to demonstrate for our professors.

I also updated my timeschedule for the third phase of the project. Since I've been very much ahead of my current schedule it was almost completely rewritten since the original.

To sum it up, my goals with the project is almost achieved so I will spend the remaining phase
- rewriting the engine to fit the new nVidia card (still haven't tested it since the PSU hasn't arrived)
- increase the visual quality
- optimize everything
- write some kind of demo which will be used in the final presentation of the project.
- Fix bugs that might appear, probably regarding animation since that part isn't thoroughly tested yet.

I'll probably wont be able to keep myself away from testing new effects and shader techniques but nothing "new" is planned for the project.

textures starting...

2005-01-07T06:53:00.000-08:00

So, the modelling is 100%, and the rigging is pretty much finished (haven't painted bone weights yet) ... Anyway - I have now started to seriously work on the textures. This is a VERY important part. Ugly textures = ugly overall look. Nice textures = nice overall look.

For the rigging I decided to do many things manually. This is because my experience says that when I try to make things work automatically, they tend to backstab me by not working at all and be very difficult to adjust. By having them manually operated, I will have to do some more work when animating, but I will most likely avoid some very irritating errors. I have tested the rig by baking it and it seems to work fine. At first it didn't work because I had an animated group above the root joint, which ofcourse wasn't read by the exporter. I solved this simply by creating an expression in the rootjoint to copy the things from my animated group. These expressions are baked when baking the simulation, so it works good.

Small update.

2004-12-13T10:34:00.000-08:00

Haven't done so much the past week because I needed to put some time into the other courses I read pararell with the project. (I'm doing a Quake3 map viewer in java :)
But I implemented parallaxmapping for the ground today and it looks pretty cool.
Also built a new viewer for Oskar and explained how the lightning is calculated so he can start planning his textures.

The GFX card is here but I need a better PSU before I dare to try it, this computer only has a 250w PSU right now and I'll probably blow it if I try using the card now.
But a 305w PSU is ordered and it'll probably be here soon.

Good model done...

2004-12-06T14:59:00.000-08:00

The highpoly good mesh has now been completed and exported as a normal map... Feels kind of wierd to work on something for a long time then just accomplish a 2048x2048 map from it... But - it looks pretty cool... Well I haven't really been able to check if all details are good, because I can't pan in the current viewer - but I think there only will be minor errors which can be fixed in photoshop...

Also slowly started rigging. I really hate rigging. But it's coming along ok. It's actually probably almost finished but I don't wanna call it finished yet, because I'm not sure if it's good enough - have to test it a bit more. The irritating thing is that in order to finally test it, I have to paint the skin weights. If it turns out to be not ok - I will have to paint them all again. (unless the import/export skin weights can magically help me) .. Anyway - therefore, I'm kind of avoiding finishing the rigging right now =)

The high-poly evil mesh will be started today... I really should be going home to sleep (soo tired), but I kind of feel like atleast starting the evil mesh first =)

2004-12-03T05:17:00.000-08:00

Short summary on whats happened since the last post.

- Implemented volumetric lightshaft rendering with shadows after an idea from ATI:s presentation at GDC, this is really fillrate heavy but looks so cool so i'll probably have to include it somehow in the final demo :).

- Continued working on the engine and it now has full skinning support according to my exported data from Maya. Also found out that if you paint skinweights in maya it can generate more skinweights then the max you set when the skin is bound.

- Added some GUI elements to control the morphing and a timeline to control the animation.

- The shader that will be used for the angel is almost complete, there are some things that can be optimised away like the spotlight calculations if i use a projected texture for that to.

- Tried a nice thing where a plane divides the model and affects the morphing. Ex, the top half of the angel can be "good" and the lower half "fallen". This might be used for some nice effect later on. Im also thinking of doing the morph gradually where the divider cuts the angel to get a more soft transition and add some noise function so the transition doesnt look so gradual.

The GFX card hasn't arrived yet wich is bad because im really keen to try the performance of it and see how big the performance win is for shadowbuffers, but it'll hopefully be here soon.

1 week behind

2004-12-01T05:17:00.000-08:00

Well... I went to the SACO education fair, and then to the elmia dreamhack, representing the school... and it took some time I would have used on the project so I'm about 1 week behind now... But I'm compensating by working harder and more.

Currently I'm doing the high-poly mesh for the good angel - to be used in the generation of the normal maps. It's proceeding along nicely, but sure takes some time. I've also adjusted the low-poly mesh in some areas to better fit the requirements of the high-poly. .. The evil angel will be created in Zbrush, which gives room for more smooth things and such...

over and outzors.

Oh and I really need the gfx-card soon, because this card is having some problems handling 2048 textures...

Small progress update

2004-11-17T04:11:00.000-08:00

Projected texture with shadows

Implemented projective texturing for the spotlight and it looks pretty cool.
The above image looks a "little" gray because the diffuse texture is gray both for the floor and the angel. All the color is coming from the projected texture, instead of using a constant diffuse / speclar color for the light i sample them from the projected texture.

Also the shadows is much smoother and has fewer aliasing problems than in the last post. I switched from a second depth shadow buffer and uses a first depth with a constant bias instead.
The shadow edges are filtered using a 3x3 PCF filter to make them look smoother (yes, it's a percentage closes filter but not bilinearly interpolated as some claim a PCF filter would be, the original paper about PCF from sigraph '87 does NOT use bilinear interpolation).

I really hope the 6800 GT gfxcard will be here soon, with that i'll probably be able to boost the framerate quite a bit because of the hardware accelerated shadow rendering wich for some reason isn't implemented on ATI cards.

Progress update

2004-11-15T18:55:00.000-08:00

Shadow

Since the last post i've been busy coding shadows, glow etc (im still in the vs/ps research/design phase of the project).
The shadowing is done using an ordinary shadowmap and i managed to get rid of most aliasing artifacts by using a second depth shadowbuffer.
It works by rendering the backfaces from the lights point of view instead of biasing the z-values for the frontfaces.
By rendering the backfaces i move the z-fighting problem to the "shadowed" areas of the object, this however introduced some problems when using normalmapping (the texel normal on a backface seen from the light can be tilted so much that it's actually visible) but this was solved by computing a geometric selfshadowing term using the interpolated normal from the vertices instead of the fetched one from the normalmap.
There is still some aliasing problems when a face is close to parallell to the light and at the moment i dont know if this can be helped at all.
I've also tried some different filtering techniques to soften/antialias the shadows, 2x2 PCF, 3x3 blur etc and each of them is good except in special cases so i havent decided yet with one to use.

Im now upp to in 3 passes to render a shadowed scene (1 for the shadowmap rendered from the lights perspective, 1 to compute the shadowing term and store that in alpha and 1 for the ordinary lightning etc wich gets alphablended with the previous pass).
The last 2 passes could be combined when i get hold of a card that supports ps 3.0 but with ps 2.0 im running out of space because the added calculations for filtering the shadow makes me exceed the limit on number of instructions.

I've also coded a pretty cool gloweffect but it remains to be seen if it's gonna be used in the final demo.

The screenshot above shows the shadowing/selfshadowing running with a 3x3 blur filter and a 1024 x 1024 shadow map generated from a spotlight.
There is only a flat normalmap and a green diffuse texture applied to the angel so it doesnt look to good but it's mainly to show the shadowing in progress.

If the fps seems low it's just because the "real" pixel / vertexshaders are in use but there is not any fancy texturedata to really show them off.

In the coming weeks i'll probably focus more on improving the shadow quality and fixing the engine design.

Engine progress

2004-11-03T10:00:00.000-08:00

The engine is starting to shape up now.
There are some design issues left but it's getting usable at least (it's hard to keep from "overdesigning" instead of just getting the job done :)
There is a simple transform oriented scenegraph implemented with methods to collect renderable objects, lights, shadowcasters etc. so the rendering logic is controlled from the app layer (this makes it must simpler/faster to test effects etc).
So far skinned and nonskinned meshes work both with/without normalmapping.

There is only rotational skinning support right now but i'll implement the translation part in the near future.
The rotation is interpolated using SLERP (Spherical Linear Interpolation) and seems to give close enough results when compared to maya.
For the translation i'll probably use a hermite spline.

I also skin the normals/tangents using only the rotational part of the skinmatrix, this isn't entirelly mathematically correct but gives reasonable results (accurately recomputing the TBN space per vertex per frame is out of the question).

Concept sketch 001

2004-11-01T05:24:00.000-08:00

Well this isn't exactly how it's going to turn out, but they look pretty cool anyway so I'll post 'em for your pleasure...

Well, the armor is from the regular angel actually, and have been a bit changed since this sketch. Colours are a little off, and wings shouldn't be that way at all.

Demo controls, v.00.01

2004-11-01T05:20:00.000-08:00

Checkbuttons:
Normalmap
Specularity (toggles both specular map and specular power map)
Diffuse color (if off, use a default colour)
Wireframe (grays out other controls)
Shadows

Sliders:
Zoom amount
Morph amount [ angel -------- 50/50 -------- fallen ]

Time Controls: < | >
Play (starts animation, regular speed)
Freeze (pauses the animation)
Backplay (starts animation backwards, same speed)

Other buttons:
Reset (resets all controls to default values)
Demo! (starts the automatic controls)

Other controls:
Scrollwheel = zooms in/out, forward = in, backward = out
Left buttonpress + mousemove = camera pan
Right buttonpress + mousemove = Camera rotate

perhaps an invert mouse option?

Exporter updated

2004-10-29T03:57:00.000-07:00

Updated the G3D Maya exporter so I account for the artist generated smoothinggroups (hard / soft edges controlling vertex normals) when calculating the tangent and binormal.
So now I calculate the tangent and binormal after the vertexsplit (a vertex can be split because of different texturecoords or normal depending on which face it belongs to) and align them to the artist generated normal. Hopefully this will solve some problems with sharp edges.
Found a neat little function called "Gram-Schmidt orthogonalize" to align the tangent and binormal against the normal :)

Normal maps are evil

2004-10-27T11:53:00.000-07:00

Darnit, my original post got lost. Not only are normal maps evil, blogger seems to be evil too!

Anyway, what I had to say was: Normal maps may seem very versatile, but they are actually very specialized. This puts lots of pressure on the gfx artist (me). Anyway, my current problem consist of staying low-poly while still getting more detail without having normal-map errors. I think most of the errors come from hard/soft edges, but I have faith in my newest attemt. Here I will use no hard edges and bevel all edges that are supposed to be hard then turn them soft. Except the blade, which I will keep hard so it gets real hard (hopefully).

New GFX card :)

2004-10-27T04:13:00.000-07:00

Just found out that i'll have access to a nVidia GeForce 6800 ultra card for this project.
Using this card i'll be able to run the demo in much higher resolutions with antialiasing.
I also got 32 bit FP so there will probably be fewer aliasing problems when using the shadowbuffer.
It will probably take a week or two before i get the card so are gonna focus on designing the engine until then.

Screenshot

2004-10-27T03:23:00.000-07:00

Posting a screenshot of the basic G3D Viewer.
It's not completed yet but hopefully it will be enough to use for Oskar in the coming days. He will mainly use it to test his generated normalmaps.
There is no lightcontrol implemented yet but there is a white light at a fixed position of 20 units from the objects center along the z-axis which should be enough to test the normalmap.

The viewer is written in D3D and uses HLSL for shaders.
The textures that contribute to the rendered image are
- Diffuse color
- Specular color
- Normal map
- Shininess

The bumpmapping used is worldspace bumpmapping because in a later stage i'll use it on skinned meshes and it's convenient to do the lightning calulations in worldspace because i will have skinned my tangents/normals in worldspace.
The shading is done using blinn and a correct half angle vector is calculated per pixel
(not interpolated from the vertices because it's not a linear transform).
The object is built of 2 triangles and its the normalmap that gives the apperance of a wavy / bumpy surface.

Viewer

2004-10-26T09:28:00.000-07:00

After the project meeting today i started on a G3D viewer.
Using the viewer Oskar will be able to test hist creations on his own and i can focus on design / implementation of the small engine that will drive the final demonstration.

G3D viewer Features
- Choose G3D Mesh to display
- Choose wich diffuse color, normal, specular color, shininess textures to apply
- Rotation and zooming of the mesh
- Rotation and translation of a light

Hopefully i'll be able to complete it until next week (have some exams this week so not so much time for the project).

First post

2004-10-26T03:39:00.000-07:00

Hi. Im the programmer on this project.
Hopefully with an online diary i'll be writing more about the progress of the project :)

Now when period 1 of the project is finished everything feels pretty good and im ready to start period 2 and really dig myself into the graphical part of the programming.

A short resume of whats been done in period 1.

I programmed an exporter for Maya 6.0 wich exports
- Geometry data (vertices, triangles, uvs etc)
- Bound skin (all the joints affecting the geometry)
- Animations for the skin (translational and rotational)
- Calculated tangent and binormals for the vertices

Small testprograms for skinning and animation so i could verify that the exported data is correct.

Played a little with HLSL and implemented normalmapping and the morph we will use.

Blog created, old diary perhaps gone?

2004-10-26T03:13:00.000-07:00

Had my old project diary at home. But my computer at home is currently broken so our good friend Simon suggest a blog instead. So here we are.

Currently I'm working on the low-poly model.

Me and Fredrik have done a lot of tests to ensure the smoothness of the project. Problem areas are almost all about normal maps and include: Soft/hard edges, UV-mapping, and ofcourse generation.

Soft/hard edges is mostly a problem for me. It affects how the normal maps are generated.
UV-mapping is limited to be very non-streched and to have as few seams as possible. The seams should have as much the same orientation as possible.
During the generation of normalmaps, I will have to be very considerate and make sure that the correct data is sampled.

About my sketches:
I forgot to do a few sketches, but I remember them now. They are not the most important but I aim to do them in the coming weeks. The most important sketch(fron/back/side views) is finished and I have also used it for my low-poly model.