Patch to handle "Set smoot/set solid

#1 ...

#2 Well spotted! I'll try it at once.

#3 Joseph, on a second glance, it seems to me that your fix is in the right direction, but it's still incomplete. Suppose a vertex belongs to a solid face and a smooth face. If the algorithm encounters the smooth face first, then the fix is correct: the first time the vertex is processed, the smooth normal is computed; the second time, the early-out check fails and the solid normal is computed. However, if the algorithm encounters the solid face first, the solid normal is computed; the second time, the early-out check succeeds and the smooth normal is never computed.
I am working on a more elaborated fix. Stay tuned.

#4 ...

#5 I thought about that solution, but in the end I tried a different approach. It's slower and somehow brute but it can produce less vertices in case you have a large tesselated "solid" surface that is planar. With the "flag" approach you might end up duplicating all vertices for each triangle of the surface, although all of them have the same normal. My approach is to compute the normals before the early-out test and use them in the test. This is the code:

# Compute vert normals
		vert = msh.verts[face.v[faceIndex].index]
		if face.smooth:
			normal = matrix.passVector(Vector(vert.no[0], vert.no[1], vert.no[2]))
		else:
			normal = matrix.passVector(Vector(face.no[0], face.no[1], face.no[2]))
		normal.normalize()

		# See if the vertex/texture/normal combo already exists..
		bvIndex = face.v[faceIndex].index
		for vi in range(0,len(self.vertsIndexMap)):
			if bvIndex == self.vertsIndexMap[vi]:
				# See if the texture coordinates match up.
				tx = self.tverts[vi]
				no = self.normals[vi]
				if hypot(tx[0] - texture[0], tx[1] - texture[1]) < 0.0001 \
					and hypot(no[0] - normal[0], no[1] - normal[1]) < 0.001:
					return vi

		'''
			Add new mesh vert and texture
			Get Vert in world coordinates using object matrix
			Texture needs to be flipped to work in torque
		'''
		nvert = matrix.passPoint(Vector(vert.co[0], vert.co[1], vert.co[2])) * scaleFactor
		vindex = len(self.verts)
		self.verts.append(nvert)
		self.tverts.append(texture)
		self.vertsIndexMap.append(bvIndex)
		self.normals.append(normal)
		self.enormals.append(self.encodeNormal(normal))

You will need to add "from math import hypot" somewhere, for example at the beginning of the file. I used hypot() instead of direct tests to overcome possible rounding errors (also for the texture coordinate for symmetry). Surely this approach is slower. I don't have at hand large models to check how much slower it is, unfortunately.

#6 ...

#7 Correct. Actually you would't need to check the magnitude of the z component, because normals are normalized. So if the xy components are equal, the third component may only differ in sign, its magnitude being sqrt(1-x*x-y*y). However, when both x and y are very near to zero, the check on only two coordinates may be inaccurate, so it makes sense to check all three components. Your approach with the fabs functions looks ok to me. It might even be marginally faster because it doesn't compute any square root.