[BUG] Seams in normal-mapped DAE models (again) [w/ fix] - LOGGED

An old bug returned from hell. This time, I'm forced to use the FBX DAE exporter and my models get terribly faceted normals when using normal maps. But this time I decided to go down in the T3D collada loading source code and see what is going wrong, and I have results.

The problem has two causes. I managed to fix one of them, and will be looking into the other one soon.

The first problem is that some DAE exporters generate duplicated normals when there shouldn't be. In my case, I have a DAE with ~3K triangles, ~4K positions but ~9K normals. A quick glance at the DAE shows many duplicated normals. When loaded into vertex buffers the numbers of positions, uvs, normals, etc must be the same, so T3D ends up duplicating entire vertices to fit the duplicated normals. This is also a performance problem because it generates useless vertices which take VRAM space and need to be skinned.

Fortunately, T3D already has a mechanism in place to avoid duplicated vertices in the DAE. The problem is that it only detects vertices which share the same position/normal/uv/etc indices, and does not check if the actual data those indices point to are actually identical. A small change makes it capable of detecting actual duplicated vertices.

In colladaAppMesh.h, replace this:

struct VertTuple
{
   S32 prim, vertex, normal, color, uv, uv2;
   VertTuple(): prim(-1), vertex(-1), normal(-1), color(-1), uv(-1), uv2(-1) {}
   bool operator==(const VertTuple& p) const 
   {
      return   prim == p.prim && 
               vertex == p.vertex && 
               color == p.color &&
               normal == p.normal && 
               uv == p.uv &&
               uv2 == p.uv2;
   }
};

By this:

struct VertTuple
{
   S32 prim, vertex, normal, color, uv, uv2;

   Point3F dataVertex;
   Point3F dataNormal;
   ColorI  dataColor;
   Point2F dataUV;
   Point2F dataUV2;

   VertTuple(): prim(-1), vertex(-1), normal(-1), color(-1), uv(-1), uv2(-1) {}
   bool operator==(const VertTuple& p) const 
   {
      return dataVertex == p.dataVertex && 
             dataColor == p.dataColor &&
             dataNormal == p.dataNormal && 
             dataUV == p.dataUV &&
             dataUV2 == p.dataUV2;
   }
};

Now in colladaAppMesh.cpp, inside ColladaAppMesh::getPrimitives(), change this:

VertTupleMap tupleMap;

...to this:

MeshStreams streams;
VertTupleMap tupleMap;

Change this:

for (U32 v = 0; v < 3; v++) {
   // Collect vert tuples into a single array so we can easily grab
   // vertex data later.

...to this:

streams.reset();
streams.readInputs(meshPrims[iPrim]->getInputs());

for (U32 v = 0; v < 3; v++) {
   // Collect vert tuples into a single array so we can easily grab
   // vertex data later.

And finally change this:

tuple.vertex = offsets[MeshStreams::Points]  >= 0 ? pSrcData[offsets[MeshStreams::Points]] : -1;
tuple.normal = offsets[MeshStreams::Normals] >= 0 ? pSrcData[offsets[MeshStreams::Normals]] : -1;
tuple.color  = offsets[MeshStreams::Colors]  >= 0 ? pSrcData[offsets[MeshStreams::Colors]] : -1;
tuple.uv     = offsets[MeshStreams::UVs]     >= 0 ? pSrcData[offsets[MeshStreams::UVs]] : -1;
tuple.uv2    = offsets[MeshStreams::UV2s]    >= 0 ? pSrcData[offsets[MeshStreams::UV2s]] : -1;

...to this:

tuple.vertex = offsets[MeshStreams::Points]  >= 0 ? pSrcData[offsets[MeshStreams::Points]] : -1;
tuple.normal = offsets[MeshStreams::Normals] >= 0 ? pSrcData[offsets[MeshStreams::Normals]] : -1;
tuple.color  = offsets[MeshStreams::Colors]  >= 0 ? pSrcData[offsets[MeshStreams::Colors]] : -1;
tuple.uv     = offsets[MeshStreams::UVs]     >= 0 ? pSrcData[offsets[MeshStreams::UVs]] : -1;
tuple.uv2    = offsets[MeshStreams::UV2s]    >= 0 ? pSrcData[offsets[MeshStreams::UV2s]] : -1;

tuple.dataVertex = tuple.vertex > -1 ? streams.points.getPoint3FValue(tuple.vertex) : Point3F::Max;
tuple.dataNormal = tuple.normal > -1 ? streams.normals.getPoint3FValue(tuple.normal) : Point3F::Max;
tuple.dataColor  = tuple.color > -1  ? streams.colors.getColorIValue(tuple.color) : ColorI(0,0,0);
tuple.dataUV     = tuple.uv > -1     ? streams.uvs.getPoint2FValue(tuple.uv) : Point2F::Max;
tuple.dataUV2    = tuple.uv2 > -1    ? streams.uv2s.getPoint2FValue(tuple.uv2) : Point2F::Max;

This solves part of the problem. The seams are gone, but I still think there's something off about my normals: with a flat (rgb(127,127,255)) normal map, my lighting looks different than not using normals. It seems as if it's coming from a different direction or something. Also, seams are still present in areas with UV discontinuities, so I believe there's something wrong with the tangent calculation functions.