Normal Mapping in 2D

Featured

An extremely common method of having pixel-accurate lighting information in real-time 3D (and obviously sometimes 2D) renderers is the usage of normal maps. I implemented support for them a while back but it wasn’t until recently that I smoothed out all the bumps (no pun intended) involving them.

The normal of a surface is the direction in which the surface is “facing”, hence a normal map can be used to define, per-pixel, the direction of a surface.

All objects with normal maps are drawn to an off-screen buffer, called the normal buffer which defines the direction of each pixel on the screen. Objects without normal maps are simply drawn as having “flat” normals.

 

Here’s what a normal map looks like:

00_Normals_Explained

The individual channels describe a direction per-axis:

  • The red channel describes the x-component where black = facing left and white = facing right.
  • The green channel describes the y-component where black = facing down and white = facing up.
  • The blue channel describes the z-component which is only defined as pointing “towards” the screen (i.e. always positive). This channel can be recalculated in a shader as long as the normal map was normalized before it was removed so I use it to store an additional mask instead.

Note that sometimes these channels are flipped depending on implementation specifics.

 

Here’s what a lit sprite in the engine looks like without a normal map:

01_Flat
Left to right: Lit scene, lighting preview, normal buffer

The lighting is not exactly interesting, and in particular the metal reflection is completely flat and boring.

Note that even though the normals are completely flat the sprite still has a slight sense of volume to it. This is because of some silhouette calculations which is part of the light shader. It is not based on anything other than the alpha channel of the sprite so its use is very limited.

 

Here’s the same sprite with the normal map applied:

02_Naive

Looks great! The lighting is well defined around the sprite and the metal/light reflections are spot-on. All done, right?

Well, not quite.

What happens if the user decides to rotate or flip the object? The normals (defined by the pixel colors) which define the direction of the surface would not rotate or flip to counteract the changes to the sprite, resulting in incorrect lighting!

 

03b_Rotated_Incorrect
Incorrect normals due to rotation.

03_Flipped_Incorrect
Incorrect normals due to negative scaling.

We want to figure out a way to take these transformations into account when drawing our normal map to the normal buffer.

What we are interested in is: What direction is the transformed pixel now “pointing” in, or more accurately:
What is the orientation of each vertex’s texture coordinates on the screen?

Enter: Tangents and bitangents.

Basically, by utilizing the difference between position and texture coordinates of the triangle vertices, we can calculate two vectors, called the tangent and bitangent, which correspond to the transformed orientation of the texture coordinates.

 

// v0,  v1,  v3  = Triangle vertex positions
// uv0, uv1, uv2 = Triangle vertex texture coordinates
computeTangentBasis(vec2 v0, vec2 v1, vec2 v2, vec2 uv0, vec2 uv1, vec2 uv2)
{
  // Position delta
  vec2 deltaPos1 = v1 - v0;
  vec2 deltaPos2 = v2 - v0;
  // UV delta
  vec2 deltaUV1 = uv1 - uv0;
  vec2 deltaUV2 = uv2 - uv0;

  // Compute tangent and bitangent
  float r = 1.0f / (deltaUV1.x * deltaUV2.y - deltaUV1.y * deltaUV2.x);
  vec2 tangent =   normalize((deltaPos1 * deltaUV2.y - deltaPos2 * deltaUV1.y) * r);
  vec2 bitangent = normalize((deltaPos2 * deltaUV1.x - deltaPos1 * deltaUV2.x) * r);
}

Pseudo-code for calculating tangent and bitangent (in 2D). Note that some 3D implementations do not normalize the vectors as done above, in order to have small triangles leave less of an effect on the normal transformation.

These two vectors are stored for every vertex and a matrix can then be constructed in the vertex shader which when multiplied with a normal vector transforms it from tangent/texture to view space:

// Tangent/bitangent matrix
mat2 TB = mat2( t.x, b.x,
                t.y, b.y );

 

The tangent/bitangent (TB) matrix solves rotation of the normal map, but the flip problem still persists. By looking at the image below we can see that there are 4 permutations of flipping the axes:

05_Permutations

Looking at the image we quickly realize that there are actually only two cases: Flipping both axes equates to just rotating the sprite 180 degrees, and flipping one of them equates to flipping the other and rotating by 180 degrees.

If we visualize the tangent/bitangent vectors this becomes even clearer:

06_Tangents_Bitangents
Tangents/bitangents visualized with tangents shown in red and bitangents in green.

Since multiplying with the TB matrix solves the rotation we need to determine when normals also need to be flipped. This can be done by utilizing the cross product of two 3-dimensional vectors:


Handedness of tangent/bitangent vectors.

We extend our tangent and bitangent vectors with z = 0 and calculate the cross product which gives us the “normal” vector (should be either (0, 0, 1) or (0, 0,-1) if the tangent and bitangent were normalized). If the sign of the z component is negative we know we have to flip the normal.

if (cross(vec3(t, 0), vec3(b, 0)).z < 0.0)
    flip = true;
else
    flip = false;

Checking the handedness of the tangent/bitangent in the vertex shader.

 

How you flip the normal depends on your implementation. I ended up doing this in the fragment shader:

 // Flip
 normal.y = mix(normal.y, -normal.y, flip);

 // Multiply with tangents and binormals
 normal.xy = TB * normal.xy;

 // Flip
 normal.y = mix(normal.y, -normal.y, flip);

 

Finally, we’ve got correct normals for any angle/scaling, yay!

 

04_Flipped_Correct

07_Permutations_Corrected

 

Making sure normal maps are oriented correctly is super important for entities that move a lot such as characters and cloth objects. Luckily this approach works on a triangle-per-triangle basis and thus is very easy to apply to any mesh!

 

08_Character_Tangents
Tangents and bitangents visualized on a skinned mesh character

 

If you are interested in a more thorough explanation on tangent/bitangents, and normal mapping in general, then this tutorial is well worth a look: http://www.opengl-tutorial.org/intermediate-tutorials/tutorial-13-normal-mapping/

Leave a Reply

Your email address will not be published.