64bit server, 32bit clients
by Konrad Kiss · in Torque Game Engine Advanced · 12/02/2008 (10:40 am) · 15 replies
I've been doing my best to compile a 64 bit TGEA 1.7.1 on a 64 bit Vista all day today. This is not so easy as I first believed.
I had to add a number of defines, and undef others for 3rd party libraries like opcode, since __asm is not available in VC++ for x64. I had to get a new version of libpng that has 64 bit support. I've been adding defines for a number of files in the source.
I took some time to stop and ask for your opinion whether I'm walking into a dead end here. In some thread I've seen GG mention that TGEA is 64 capable. Well, not right out of the box, I've just learned that. But this is nevertheless a very important feature, that would be put to good use when working with heavy duty server side solutions - I'm sure about this.
So... I want to have my game servers 64 bit - to be able to use more than 3 gigs of OS memory. I'm in the dark here, trying to start a conversation about 64 bit Torque server apps, their pros and cons.
How would the server work with clients that are compiled as Win32? Would this screw the networking code up? How would this backfire on me? What are the chances of me being able to compile TGEA 1.7.1 or 1.8 as x64? Has anybody done this before?
Any thoughts on this would be very much appreciated!
-- Konrad
I had to add a number of defines, and undef others for 3rd party libraries like opcode, since __asm is not available in VC++ for x64. I had to get a new version of libpng that has 64 bit support. I've been adding defines for a number of files in the source.
I took some time to stop and ask for your opinion whether I'm walking into a dead end here. In some thread I've seen GG mention that TGEA is 64 capable. Well, not right out of the box, I've just learned that. But this is nevertheless a very important feature, that would be put to good use when working with heavy duty server side solutions - I'm sure about this.
So... I want to have my game servers 64 bit - to be able to use more than 3 gigs of OS memory. I'm in the dark here, trying to start a conversation about 64 bit Torque server apps, their pros and cons.
How would the server work with clients that are compiled as Win32? Would this screw the networking code up? How would this backfire on me? What are the chances of me being able to compile TGEA 1.7.1 or 1.8 as x64? Has anybody done this before?
Any thoughts on this would be very much appreciated!
-- Konrad
About the author
http://about.me/konrad.kiss
#2
12/04/2008 (9:30 am)
I compile TGEA 1.7.1 on 64 bit Vista quite often using Visual C++ Express 2008 and the included solution. However, it is very possible that it compiles to 32 bit, I've never really looked into it. 
#3
We work with 64 bit servers and 32 bit clients all day long here at the office (not in torque, though). I've done a couple of conversions, and they weren't too bad. The main problem will be (as you've seen), compiling the code and getting libraries that are 64 bit compatible. There should be no effect on the networking layer, as the only inherent data type that changes size is the pointer (size_t, time_t also change to 64, but I'm not sure if those are used anywhere). Both ints and longs remain 32 bits.
I haven't compiled an x64 version of TGEA, but I'm interested in your results.
Jaimi
12/04/2008 (11:59 am)
Hi Konrad -We work with 64 bit servers and 32 bit clients all day long here at the office (not in torque, though). I've done a couple of conversions, and they weren't too bad. The main problem will be (as you've seen), compiling the code and getting libraries that are 64 bit compatible. There should be no effect on the networking layer, as the only inherent data type that changes size is the pointer (size_t, time_t also change to 64, but I'm not sure if those are used anywhere). Both ints and longs remain 32 bits.
I haven't compiled an x64 version of TGEA, but I'm interested in your results.
Jaimi
#4
@Jaimi: Thanks for the info! This is very promising, it just gave me my will back to go on with this. Hitting a wall at every turn can be quite demoralizing. This is just what I needed. I will try to write a how-to once everything works fine. I'm glad about the transparency of the networking layer, I was mostly afraid of that, I wouldn't like to poke that at all. Size_t has been causing a number of warnings, but I guess it would be more of a problem if I downgraded from 64 bits to 32, so for now I'll let it be. I'll definitely let you know how it goes.
Thanks once again!
12/04/2008 (12:21 pm)
@Patrick: So there's hope! :) Thanks!@Jaimi: Thanks for the info! This is very promising, it just gave me my will back to go on with this. Hitting a wall at every turn can be quite demoralizing. This is just what I needed. I will try to write a how-to once everything works fine. I'm glad about the transparency of the networking layer, I was mostly afraid of that, I wouldn't like to poke that at all. Size_t has been causing a number of warnings, but I guess it would be more of a problem if I downgraded from 64 bits to 32, so for now I'll let it be. I'll definitely let you know how it goes.
Thanks once again!
#5
12/07/2008 (1:58 am)
Finally, after about a gazillion compiler and linker errors, compiling new 64 bit zlib, ogg and vorbis libs, TGEA 1.7.1 x64 compiles for me. But right when I start debugging, it breaks in U32toSTE returning an invalid pointer. I was wondering if anybody had this, and thought I would ask before trying to trace this one down. 
#6
You can probably see that this is a big job... lots of places assume that a TorqueScript bytecode is U32. The only way to fix this is to change the TorqueScript byte code to use U64... or better yet....
Oh... and even after you fix that there will be more problems like this in Torque.
The bottom line is while you could get Torque to compile to 64bit it will not be simple to do... but i believe its a very worth while cause.
If you get it working i'd really like to move your changes into the official Torque codebase.
12/07/2008 (2:22 am)
This is a classic problem found when trying to compile older code bases over to 64bit. From Microsoft's Common Visual C++ 64-bit Migration Issues article...Quote:An int and a long are 32-bit values on 64-bit Windows operating systems. For programs that you plan to compile for 64-bit platforms, you should be careful not to assign pointers to 32-bit variables.U32toSTE converts a U32 to a StringTableEntry. The U32 is still 32 bits, but StringTableEntry is a const char*.... so it will be 64bits. So when this conversion happens you've already lost the upper 32bits of the pointer, so its no longer valid.
You can probably see that this is a big job... lots of places assume that a TorqueScript bytecode is U32. The only way to fix this is to change the TorqueScript byte code to use U64... or better yet....
typedef U64 BYTECODE;... so that its easier to change it in the future.... 128bit computing... yay!
Oh... and even after you fix that there will be more problems like this in Torque.
The bottom line is while you could get Torque to compile to 64bit it will not be simple to do... but i believe its a very worth while cause.
If you get it working i'd really like to move your changes into the official Torque codebase.
#7
12/07/2008 (2:29 am)
@Tom: Thank you! I've been marking all my changes so it will be easy to apply them to stock TGEA once I'm done. Thanks again for the great help!
#8
I've taken care of times, streams, vectors, strings... I'm only hoping that the worst is already behind me, but I have a feeling that this will just get uglier.
Anyway, I've hit a wall and I wanted to ask for some advice.
I crash in tsShape.cpp in init() here (line count += mesh ? mesh->getNumPolys() : 0;) - bold is not visible for some reason:
mesh does not seem to be valid. Vectors are already 64 bit good, but this here is a bit confusing for me.
Through IMPLEMENT_ALLOC tsShapeAlloc.cpp defines a number of functions with suffixes 32, 16 and 8.
If I only understood what those are for, I could have a better idea about how to modify tsShapeAlloc, which - I have a feeling - could eventually lead to fixing the crash above.
If someone could enlighten me about what's going on here, I'd be very thankful.
12/09/2008 (7:31 am)
I've been through hell. I'm not marking my code anymore, will do a diff once it all works. I can now run TGEA but it occasionally runs onto a mine, of course.I've taken care of times, streams, vectors, strings... I'm only hoping that the worst is already behind me, but I have a feeling that this will just get uglier.
Anyway, I've hit a wall and I wanted to ask for some advice.
I crash in tsShape.cpp in init() here (line count += mesh ? mesh->getNumPolys() : 0;) - bold is not visible for some reason:
S32 start = subShapeFirstObject[ss];
S32 end = start + subShapeNumObjects[ss];
for (j = start; j<end; j++) {
Object & obj = objects[j];
if (od<obj.numMeshes) {
TSMesh * mesh = meshes[obj.startMeshIndex + od];
[b]count += mesh ? mesh->getNumPolys() : 0;[/b]
}
}mesh does not seem to be valid. Vectors are already 64 bit good, but this here is a bit confusing for me.
Through IMPLEMENT_ALLOC tsShapeAlloc.cpp defines a number of functions with suffixes 32, 16 and 8.
If I only understood what those are for, I could have a better idea about how to modify tsShapeAlloc, which - I have a feeling - could eventually lead to fixing the crash above.
If someone could enlighten me about what's going on here, I'd be very thankful.
#9
In short it can be a nightmare to work out the file format.
If you look in TSShape::assembleShape() you'll see the following...
... looking in the loop below that you'll see it uses ptr32 as an array to hold the TSMesh*s and at the end it assigns it to the 'meshes' FixedVector. The most gross part...
The problem here is clear... its trying to store TSMesh pointers in an array of 32bit values when pointers are not 32bit anymore. Game over.
The tricky part is that i *think* that allocating a different size for allocShape32() might break the loading process. You'll have to give it a try.
12/09/2008 (11:49 am)
TSShape uses this odd method for loading/save DTS files where three seperate arrays of U8, U16, and U32 have the data split up amongst them. These arrays are not freed after the file is loaded either... they are kept around as the FixedVector class allocates no memory... it just holds a pointer given to it.In short it can be a nightmare to work out the file format.
If you look in TSShape::assembleShape() you'll see the following...
// read in the meshes (sans skins)...straightforward read one at a time ptr32 = tsalloc.allocShape32(numMeshes + numSkins*numDetails); // leave room for skins on old shapes
... looking in the loop below that you'll see it uses ptr32 as an array to hold the TSMesh*s and at the end it assigns it to the 'meshes' FixedVector. The most gross part...
TSMesh *mesh = TSMesh::assembleMesh(meshType,skip);
if (ptr32)
ptr32[i] = skip ? 0 : (S32)mesh;The problem here is clear... its trying to store TSMesh pointers in an array of 32bit values when pointers are not 32bit anymore. Game over.
The tricky part is that i *think* that allocating a different size for allocShape32() might break the loading process. You'll have to give it a try.
#10
12/09/2008 (12:24 pm)
@Tom: I was afraid you'd say something like this. I'll let you know how it went. Thanks for the response.
#11
12/21/2008 (9:29 pm)
You make any more progress on this?
#12
12/22/2008 (8:12 am)
I've stopped for now after bumping into an endless number of pointer issues. The DTS loader is a minefield, and it reaches everywhere. I've decided to take a break, and get back to it fresh in January. I'm not giving up. I'll start again with 1.8 instead of 1.7.1. 
#14
03/04/2011 (12:48 am)
Quote:The DTS loader is a minefieldwhile messing with DTS loader in order to get it "thread-safe"... definitely a minefield!
#15
To me this seems pretty straightforward:
should become (its a little non-canon because of my stl-ification of base torque containers)
The question is if during TSShape::assembleShape, does tsalloc.allocshape serve any higher purpose than keeping allocations continuous?
11/30/2013 (11:15 pm)
So I've stumbled into this minefield as part of the process of trying to add 3d capabilities to my 64-bit t2d fork. To me this seems pretty straightforward:
// read in the meshes (sans skins)...straightforward read one at a time
ptr32 = tsalloc.allocShape32(numMeshes + numSkins*numDetails); // leave room for skins on old shapes
...should become (its a little non-canon because of my stl-ification of base torque containers)
typedef ptrdiff_t PTR; // in platform/types.h
// read in the meshes (sans skins)...straightforward read one at a time
ptr32 = tsalloc.allocShape32(numMeshes + numSkins * numDetails * sizeof(PTR)/4); // leave room for skins on old shapes
PTR* mesh_ptrs = (PTR*)ptr32;
S32 curObject = 0; // for tracking skipped meshes
for (i=0; i<numMeshes; i++)
{
bool skip = checkSkip(i,curObject,skipDL); // skip this mesh?
S32 meshType = tsalloc.get32();
if (meshType == TSMesh::DecalMeshType)
// decal mesh deprecated
skip = true;
TSMesh * mesh = TSMesh::assembleMesh(meshType,skip);
if (mesh_ptrs)
mesh_ptrs[i] = skip ? NULL : (PTR)mesh;
// fill in location of verts, tverts, and normals for detail levels
if (mesh && meshType!=TSMesh::DecalMeshType)
{
TSMesh::smVertsList[i] = mesh->verts.address();
TSMesh::smTVertsList[i] = mesh->tverts.address();
TSMesh::smNormsList[i] = mesh->norms.address();
TSMesh::smEncodedNormsList[i] = mesh->encodedNorms.address();
TSMesh::smDataCopied[i] = !skip; // as long as we didn't skip this mesh, the data should be in shape now
if (meshType==TSMesh::SkinMeshType)
{
TSSkinMesh * skin = (TSSkinMesh*)mesh;
TSMesh::smVertsList[i] = skin->batchData.initialVerts.address();
TSMesh::smNormsList[i] = skin->batchData.initialNorms.address();
TSSkinMesh::smInitTransformList[i] = skin->batchData.initialTransforms.address();
TSSkinMesh::smVertexIndexList[i] = skin->vertexIndex.address();
TSSkinMesh::smBoneIndexList[i] = skin->boneIndex.address();
TSSkinMesh::smWeightList[i] = skin->weight.address();
TSSkinMesh::smNodeIndexList[i] = skin->batchData.nodeIndex.address();
}
}
}
meshes.set((TSMesh * const *)mesh_ptrs,numMeshes);The question is if during TSShape::assembleShape, does tsalloc.allocshape serve any higher purpose than keeping allocations continuous?
Associate Konrad Kiss
Bitgap Games
Thank you, and sorry for the bump.