Several people have expressed interest in various forums in understanding the algorithm 3ds Max uses to compute tangents and binormals for normal mapping. We are going to be exposing this algorithm in the next version of the 3ds Max SDK documentation, but since I’ve got a blog I’ve decided to share the information right away.

Together with the normal vector, the tangent and binormal vector are important because they create a basis which defines the tangent space used by a renderer to perform the normal mapping. In the 3ds Max SDK the function for creating tangent and binormal vectors used by the scanline renderer for rendering normal maps is ComputeTangentAndBinormal() and is found in the gutil.h header file.

This function takes two points as input: tv which is a texture vertex point (e.g. a value retrieved using the Mesh::mapVerts() function), and v which is a vertex point (e.g. a value retrieved using the Mesh::verts member variable). It returns the two basis vectors via the third argument, bvec. After executing the function the bvec[0] element will contain the tangent vector, and the bvec[1] element will contain the binormal vector. The binormal vector is the cross product of the surface normal with the tangent.

The algorithm used by 3ds Max is as follows:

void ComputeTangentAndBinormal(const Point3 tv[3], const Point3 v[3], Point3 bvec[2])
{
  float uva,uvb,uvc,uvd,uvk;
  Point3 v1,v2;

  int ix = 0; // 0 corresponds to the U axis
  int iy = 1; // 1 corresponds to the V axis

  uva = tv[1][ix]-tv[0][ix];
  uvb = tv[2][ix]-tv[0][ix];

  uvc = tv[1][iy]-tv[0][iy];
  uvd = tv[2][iy]-tv[0][iy];

  uvk = uvb*uvc - uva*uvd;

  v1 = v[1]-v[0];
  v2 = v[2]-v[0];

  if (uvk!=0) {
    bvec[0] = FNormalize((uvc*v2-uvd*v1)/uvk);
  }
  else {
    if (uva!=0)
      bvec[0] = FNormalize(v1/uva);
    else if (uvb!=0)
      bvec[0] = FNormalize(v2/uvb);
    else
      bvec[0] = Point3(0.0f,0.0f,0.0f);
  }


  Point3 normal = Normalize( (v[1] - v[0]) ^ (v[2] - v[1]) );
  bvec[1] = CrossProd( normal, bvec[0] );
}


Alone this function is not the whole story. It is important to know how it is used in 3ds Max. Sample code found in the NormalBump sample of the 3ds Max SDK (e.g. the function VNormalChannel::InitTangentBasis() in the file maxsdk/samples/materials/NormalBump/VNormal.cpp) demonstrates how this function is used in normal bump mapping, and is the same algorithm as is used in the scanline renderer when generating and displaying the normal map.

void VNormalChannel::InitTangentBasis( Mesh& mesh, Matrix3& tm, int mapChannel ) {
  int numFaces = mesh.getNumFaces();
  Face *geomFaces = mesh.faces;
  TVFace *mapFaces = mesh.mapFaces( mapChannel );
  Point3 *geomVerts = mesh.verts, *mapVerts = mesh.mapVerts( mapChannel );
  Point3 geomTri[3], mapTri[3], basisVec[2];
  Tab<TangentBasis> tangentBasisSet; //temp array

  //Error checking
  if( (mapFaces==NULL) || (mapVerts==NULL) ) {
    tangentBasisSet.SetCount(0);
    validTangentBasis = TRUE;
    return;
  }

  // 1. Allocate the tangent array
  tangentBasisSet.SetCount( numItems );
  if( numItems>0 )
    memset( tangentBasisSet.Addr(0), 0, numItems*sizeof(TangentBasis) );

  // 2. Loop through the faces, calculating the tangent for each
  for( int f=0; f<numFaces; f++ ) {
    Face& geomFace = geomFaces[f];
    geomTri[0] = geomVerts[ geomFace.v[0] ];
    geomTri[1] = geomVerts[ geomFace.v[1] ];
    geomTri[2] = geomVerts[ geomFace.v[2] ];

    TVFace& mapFace = mapFaces[f];
    mapTri[0] = mapVerts[ mapFace.t[0] ];
    mapTri[1] = mapVerts[ mapFace.t[1] ];
    mapTri[2] = mapVerts[ mapFace.t[2] ];

    ComputeTangentAndBinormal( mapTri, geomTri, basisVec );

    Point3 mapNormal = FNormalize( (mapTri[1] - mapTri[0]) ^ (mapTri[2] - mapTri[1]) );
    if( mapNormal.z<0 ) basisVec[1] = -basisVec[1]; //is the UV face flipped? flip the binormal

    // 3a. For smoothed faces,
    // add the tangent to the sums for the three verts, and normalize below
    if( !IsFaceted(f) ) {
      for( int fv=0; fv<3; fv++ ) {
        int v = geomFace.v[fv];
        int index = FindIndex( f, v, fv );
        tangentBasisSet[index].uBasis += basisVec[0];
        tangentBasisSet[index].vBasis += basisVec[1];
      }
    }


    // 3b. For faceted faces, store one face tangent
    else {
      int index = FindIndex( f, 0, 0 );
      tangentBasisSet[index].uBasis = basisVec[0];
      tangentBasisSet[index].vBasis = basisVec[1];
    }
  }

  // 4. Normalize the tangents
  for( int i=0; i<numItems; i++ ) {
    TangentBasis& bv = tangentBasisSet[i];
    bv.uBasis = tm.VectorTransform( bv.uBasis );
    bv.vBasis = tm.VectorTransform( bv.vBasis );
    bv.uBasis.Unify();
    bv.vBasis.Unify();
  }

  // 5. Finalize the operation
  memswp( &(this->tangentBasisSet), &tangentBasisSet, sizeof(tangentBasisSet) );
  validTangentBasis = TRUE;
}