Transforming Normals

Transforming Normals

Problem Statement

A normal is defined in object space.
You need the normal in world space (world space = eye space in my language).
You know the object space to world space transformation matrix.
How do you get the transformed normal (the normal in world space)?

You Can't Just Transform It

It should be obvious that merely transforming the normal by the object to world space transformation matrix won't work.
For example, it obviously won't work if the object to world space transformation is a translation.
If you think about it, you'll see it won't work if the transformation is a non-uniform scale; angles are not preserved by non-uniform scale.

Solution

Consider the planar equation for a group of points:

    N = [a b c d]         (remembering that the normal vector is [a b c])

    P = [x y z 1]

    NP^T = 0  (remember the plane equation a*x + b*y + c*z + d = 0)

Assume the transformation matrix is M, then we want the transformed planar equation, (N' this is also the transformed normal vector we are interested in) to satisfy:

    N'MP^T = 0            (where MP^T is a transformed point)

of course,

    
    NM^-1MP^T = 0      (since NP^T = 0)

so we get what we want if we transform the planar equation thusly,

    N' = NM^-1

or by using column vectors and post-multiplying by the transformation matrix:

    N'^T = (NM^-1)^T
        
        = (M^-1)^TN^T

Conclusion

And there you have it. To transform a normal using a homogenous transformation matrix to world space you have to multiply it by the inverse transform matrix.
That is to say, first invert the matrix, then transpose the inverse.

I have created a freeware library called glhlib that contains the functions for doing these. It goes well with projects written in OpenGL.
In Block 12 (SOFTWARE MATRIX OPERATIONS. DOES NOT USE OPENGL), there is a bunch of functions for doing matrix operations in software :

GLint glhInvertMatrixf2(GLfloat *m, GLfloat *result) [a general matrix inverter]
GLint glhInvertMatrixd2(GLdouble *m, GLdouble *result) [a general matrix inverter]
void glhQuickInvertMatrixf2(GLfloat *m, GLfloat *result) [for orthogonal matrix and contains no scale. Uses much less CPU instructions]
void glhQuickInvertMatrixd2(GLdouble *m, GLdouble *result) [for orthogonal matrix and contains no scale. Uses much less CPU instructions]
void glhTransposeMatrixf2(GLfloat *result, GLfloat *m) [obvious]
void glhTransposeMatrixd2(GLdouble *result, GLdouble *m) [obvious]

We would use it like this :

GLfloat WorldMatrix[16], InverseWorldMatrix[16], ITWorldMatrix[16];
glhQuickInvertMatrixf2(WorldMatrix, InverseWorldMatrix);
glhTransposeMatrixf2(ITWorldMatrix, InverseWorldMatrix);
//Now ITWorldMatrix contains our matrix of interest
//Let's transform a normal
GLfloat ObjectSpaceNormal[4], EyeSpaceNormal[4];
//This functions assumes w is 0.0
//The value 4 is submitted because we have 1 normal, and it has 4 elements, so 1*4 = 4!
glhMultiplyMatrixByVector4by4f_3(4, ObjectSpaceNormal, ITWorldMatrix, EyeSpaceNormal);

The SSE or 3DNow version of MultiplyMatrixByVector has not been written as of this writting.