Connection to dedicated server crash
by Stephen Zepp · in Torque Game Engine · 07/04/2004 (7:04 am) · 26 replies
Has anyone taken HEAD and made a Mac client executable that can connect to a dedicated server across the internet?
Our base code hasn't deviated much at all from HEAD in regards to any networking, but our Mac compiled client hard crashes right after the two way connection handshake with a dedicated server. We see progress lines on the console file all the way down to "Connection accept.", but then the client itself crashes hard. The server continues to send datablock updates and such until the timeout period is met, and then resets.
Our last successful execution block appears to be:
GameConnection::readConnectAccept, called from NetInterface::handleConnectAccept. That seems to leave:
removePendingConnection(conn); // remove from the pending connection list
conn->setNetworkConnection(true);
conn->onConnectionEstablished(true); // notify the connection that it has been established
conn->setEstablished(); // installs the connection in the connection table, and causes pings/timeouts to happen
conn->setConnectSequence(connectSequence);
as our only probable errors, but I'm pretty confident that we haven't touched any of this code, so we can't figure out why it would work on both Linux and Windows builds, but not on the Mac build.
Has anyone run into anything similar, see any possible "out of the box" problems, or have any suggestions?
Thanks!
Our base code hasn't deviated much at all from HEAD in regards to any networking, but our Mac compiled client hard crashes right after the two way connection handshake with a dedicated server. We see progress lines on the console file all the way down to "Connection accept.", but then the client itself crashes hard. The server continues to send datablock updates and such until the timeout period is met, and then resets.
Our last successful execution block appears to be:
GameConnection::readConnectAccept, called from NetInterface::handleConnectAccept. That seems to leave:
removePendingConnection(conn); // remove from the pending connection list
conn->setNetworkConnection(true);
conn->onConnectionEstablished(true); // notify the connection that it has been established
conn->setEstablished(); // installs the connection in the connection table, and causes pings/timeouts to happen
conn->setConnectSequence(connectSequence);
as our only probable errors, but I'm pretty confident that we haven't touched any of this code, so we can't figure out why it would work on both Linux and Windows builds, but not on the Mac build.
Has anyone run into anything similar, see any possible "out of the box" problems, or have any suggestions?
Thanks!
#2
I was hoping at this stage to see if anyone out there was "oh yah, we had that too--try this".
If there isn't anyone like that, we'll work on it a few more days and come back with exact info :)
07/04/2004 (7:43 am)
Unfortunately, the crash appears to destroy the stack, so we haven't been able to nail it down precisely. That, and I'm not particularly familiar with Mac debugging--best info we were able to get was some mangled method names, and each and every code line interface reference was to "crt.c line 300"...Obviously not very useful.I was hoping at this stage to see if anyone out there was "oh yah, we had that too--try this".
If there isn't anyone like that, we'll work on it a few more days and come back with exact info :)
#3
07/05/2004 (11:33 am)
Hmm yeah let us know if you get more info Stephen.
#4
07/06/2004 (12:31 pm)
We are having a problem where Mac clients can connect to Mac servers and Win32 clients to Win32 server, but the clients crash if you try and mix them(ie Mac client to Win32 server). Might be the same problem your having. We're not sure if it's our code or something in HEAD that's causing the problem. Please post if you find a solution or the source of your problem.
#5
As mentioned in the initial post, we get basically all the way to just before "Connection Established", but give up after the (debug line in HEAD): Connection accept. Is your progress similar, or do you crash elsewhere during the connection handshake?
07/07/2004 (5:55 am)
@Owen: Hmmm...well, currently we have both linux and windows clients (same codebase) able to connect to a linux server, but I wonder if your problem and ours is related. Where exactly do you crash, logfile wise? As mentioned in the initial post, we get basically all the way to just before "Connection Established", but give up after the (debug line in HEAD): Connection accept. Is your progress similar, or do you crash elsewhere during the connection handshake?
#6
08/09/2004 (10:16 am)
Any progress on this tread?
#7
08/09/2004 (10:37 am)
As far as our problem went: I reconstructed our codebase from a clean HEAD of Torque and am able to connect fine now. Not exactly sure what was causing our problem, I'm leaning towards some discrepancies between the project setups[diff files included, diff opitions] on the different platforms.
#8
08/13/2004 (9:51 am)
Stephen, did you ever get this figured out?
#9
My code base is what was on the head as of Aug 27, 2004. Apparently Owen's problem was fixed after updating to the head sometime around/before Aug 9.
What I have is an "Out of range read" assertion during the datablock transfers in the function
inline bool BitStream::readFlag(), line 159, [core\bitStream.h].
What's really weird is that the server (Apple) is sending infromation on "~\data\shapes\octahedron.dts" (some what garbled in the bstream data) but the client side has casted this as "LightningData" defined by consoleobject (by the way, I don't have any LightningData in my game). I'm gathering this information by tracing the functions in the call stack and looking at various variables. Anyway, I can see in the call stack that the bstream is unpacked in the function
void LightningData::unpackData(BitStream* stream), line 302, [fx\lightning.cc] and calls
strikeTextureNames[i] = stream->readSTString();
This function will eventually dwiddle down to calling the bstream->readFlag() function, which results in an assertion.
I appreciate any suggestions as to where I should look on the Mac server side as to why it's sending corrupt data (bstream). Well, actually I'm not sure if it is corrupt but possibly related to TGE port configuration (??).
09/08/2004 (6:42 pm)
I'm seeing similar problem as what Owen has described earlier: client/server connection between Windows platforms are fine but mixing Apples and Windows (oranges) is a problem. My code base is what was on the head as of Aug 27, 2004. Apparently Owen's problem was fixed after updating to the head sometime around/before Aug 9.
What I have is an "Out of range read" assertion during the datablock transfers in the function
inline bool BitStream::readFlag(), line 159, [core\bitStream.h].
What's really weird is that the server (Apple) is sending infromation on "~\data\shapes\octahedron.dts" (some what garbled in the bstream data) but the client side has casted this as "LightningData" defined by consoleobject (by the way, I don't have any LightningData in my game). I'm gathering this information by tracing the functions in the call stack and looking at various variables. Anyway, I can see in the call stack that the bstream is unpacked in the function
void LightningData::unpackData(BitStream* stream), line 302, [fx\lightning.cc] and calls
strikeTextureNames[i] = stream->readSTString();
This function will eventually dwiddle down to calling the bstream->readFlag() function, which results in an assertion.
I appreciate any suggestions as to where I should look on the Mac server side as to why it's sending corrupt data (bstream). Well, actually I'm not sure if it is corrupt but possibly related to TGE port configuration (??).
#10
09/09/2004 (6:52 am)
I had these kinds of problems before. I found that is was usually related to pack() and unpack() not syncing up. Make sure that whenever you modify the networking stuff, it matches up exactly. There is a flag you can set during build that can help. I believe it is NETWORK_DEBUG or DEBUG_NETWORK.
#11
The new assertion now lies in
void NetConnection::eventReadPacket(BitStream *bstream), line 240, [sim\netEvent.cc]
AssertISV(sum == DebugChecksum, "Invalid checksum.");
Now I need to figure out why a mismatch in checksum.
09/09/2004 (7:32 am)
Thanks for the info Kurtis. I was able to trap a checksum assertion by turning on the DEBUG_NET.The new assertion now lies in
void NetConnection::eventReadPacket(BitStream *bstream), line 240, [sim\netEvent.cc]
AssertISV(sum == DebugChecksum, "Invalid checksum.");
Now I need to figure out why a mismatch in checksum.
#12
09/09/2004 (7:40 am)
The checksum assumption is premature because the comparison evaluates the checksum sent by the server, which I have not rebuilt yet. I'm rebuilding the code on Mac to retest.
#13
I'm beginning to suspect that the console declaraction has something to do with this because of the mismatching ClassId in a call to
SimObject* ptr = (SimObject *) ConsoleObject::create(cptr->getNetClassGroup(), NetClassTypeDataBlock, classId);, line 120, [game\gameConnectionEvents.cc].
I'll need to investigate this further.
09/09/2004 (7:58 am)
Results are same after rebuilding the OSX in Mac: Mac (server) sends MissionMarkerData which is cast as LightningData on the client side.I'm beginning to suspect that the console declaraction has something to do with this because of the mismatching ClassId in a call to
SimObject* ptr = (SimObject *) ConsoleObject::create(cptr->getNetClassGroup(), NetClassTypeDataBlock, classId);, line 120, [game\gameConnectionEvents.cc].
I'll need to investigate this further.
#14
09/09/2004 (12:20 pm)
This is now resolved. Turns out that I had a source file missing in the Mac project, DOH.
#15
09/12/2004 (6:30 am)
Hehe. Ah well, it was interesting read anyway Akio! :)
#16
Does that make sense? Probably not. Ok, little background. All classId are assigned to an object as they're loaded into a game. This order is dependent on what you have in your .mis file and every dynamic objects that are in the game. The console methods calls and implements console datablocks which in turn assigns the classId. It's assigned in a first-come-first-served basis. And if you have a missing file in your in your build project, that object is skipped, hence the classIds becomes offset by one.
Here is a code frag which you can call on the client side and the server side to compare if your classId matches up correctly:
This is in [gameConnection.cc] file and should only be called after you load a mission. What it does is it dumps all the objects and its classIds in the loaded mission. Take a dump on both the client and the server side and visually compare where the offset is.
09/12/2004 (8:17 am)
Yeah, I felt stupid after I discovere a file missing in the project, but I did find out what the symptoms are with this problem. Because of the missing file, all object's classId were off by one on the client side starting with that object's missing file. Does that make sense? Probably not. Ok, little background. All classId are assigned to an object as they're loaded into a game. This order is dependent on what you have in your .mis file and every dynamic objects that are in the game. The console methods calls and implements console datablocks which in turn assigns the classId. It's assigned in a first-come-first-served basis. And if you have a missing file in your in your build project, that object is skipped, hence the classIds becomes offset by one.
Here is a code frag which you can call on the client side and the server side to compare if your classId matches up correctly:
ConsoleMethod( GameConnection, listClassObjects, void, 2, 2, "")
{
SimDataBlockGroup *g = Sim::getDataBlockGroup();
U32 i, groupCount = g->size();
SimDataBlock* obj;
S32 classId;
Con::printf("--------------- class object listing (%d) ----------------", groupCount);
for (i = 0; i < groupCount; i++)
{
if (Sim::findObject(i,obj))
{
classId = obj->getClassId(0);
Con::printf("index=%d, class id=%d, name=%s", i, classId, obj->getClassName());
}
}
}This is in [gameConnection.cc] file and should only be called after you load a mission. What it does is it dumps all the objects and its classIds in the loaded mission. Take a dump on both the client and the server side and visually compare where the offset is.
#17
09/12/2004 (8:46 am)
Nice write-up Akio. If you want to expand on it, it'd make a great resource submission (good info like this sometimes gets buried in the forums).
#18
I have this same problem now, and I'm not sure how to solve it. Windows <-> Windows and Linux <-> Linux works fine, but mixing them does not work. I've run your little snippet, and sure enough, class IDs start to differ although the number of objects seems to be the same:
And so on, it continues to differ from there on. This however doesn't tell me much... maybe it isn't supposed to?
I assume I have some kind of source difference somewhere, but I'm not sure how to find it. The exes are compiled from the same source in subversion, and I don't *think* there are any platform dependant changes - however, the Linux makefiles may not have the same files listed as the VS project, because I'm only now trying to sync it up. Is there any easy way to see what files differ between those? Or do I have to manually compare file for file if both are being compiled?
Any ideas or other pointers on how to find the offending code would be extremely appreciated, as I need to get a dedicated server up asap.
Thanks!
05/10/2005 (9:08 am)
Hello,I have this same problem now, and I'm not sure how to solve it. Windows <-> Windows and Linux <-> Linux works fine, but mixing them does not work. I've run your little snippet, and sure enough, class IDs start to differ although the number of objects seems to be the same:
--------------- class object listing (124) ---------------- index=3, class id=1, name=AudioDescription index=4, class id=1, name=AudioDescription index=5, class id=1, name=AudioDescription index=6, class id=1, name=AudioDescription index=7, class id=1, name=AudioDescription index=8, class id=1, name=AudioDescription index=9, class id=1, name=AudioDescription index=10, class id=1, name=AudioDescription index=11, class id=1, name=AudioDescription index=12, class id=1, name=AudioDescription index=13, class id=1, name=AudioDescription index=14, class id=1, name=AudioDescription index=15, class id=5, name=CameraData index=16, class id=14, name=MissionMarkerData index=17, class id=14, name=MissionMarkerData ...
--------------- class object listing (124) ---------------- index=3, class id=1, name=AudioDescription index=4, class id=1, name=AudioDescription index=5, class id=1, name=AudioDescription index=6, class id=1, name=AudioDescription index=7, class id=1, name=AudioDescription index=8, class id=1, name=AudioDescription index=9, class id=1, name=AudioDescription index=10, class id=1, name=AudioDescription index=11, class id=1, name=AudioDescription index=12, class id=1, name=AudioDescription index=13, class id=1, name=AudioDescription index=14, class id=1, name=AudioDescription index=15, class id=5, name=CameraData index=16, class id=15, name=MissionMarkerData index=17, class id=15, name=MissionMarkerData ...
And so on, it continues to differ from there on. This however doesn't tell me much... maybe it isn't supposed to?
I assume I have some kind of source difference somewhere, but I'm not sure how to find it. The exes are compiled from the same source in subversion, and I don't *think* there are any platform dependant changes - however, the Linux makefiles may not have the same files listed as the VS project, because I'm only now trying to sync it up. Is there any easy way to see what files differ between those? Or do I have to manually compare file for file if both are being compiled?
Any ideas or other pointers on how to find the offending code would be extremely appreciated, as I need to get a dedicated server up asap.
Thanks!
#19
05/10/2005 (1:26 pm)
Dump your class ID tables, and compare? (See console directory for the code that builds all that info.)
#20
I had understood it as sourcefiles (.cc) most likely to be different, but as far as I can judge, it seems that they are the same (not 100% confirmed, but all added files since project start in subversion are included in makefile - maybe there is some too many though?). Really wish there was some easy way to compare this, stupid Visual Studio doing everything backwards. ;-)
Could it be other reasons? Script files, mission files, well the whole tree should and seems to be in sync, but apparently some kind of dynamic creation is differing, if I understand this correctly.
Some more friendly tips and pointers in the right direction would be very appreciated, before I dive headfirst into something I don't know where to find. =)
05/11/2005 (2:58 am)
Thanks for your quick reply. However, I'm not sure I understand what I should do and what to look for. Can I walk the object tree somehow, looking for specific class IDs and get what sort of info from that? Where it is created, some kind of name or something? I'm sorry, not trying to make you do my work, but I'm not really sure how all this fits together, or what I should look for to solve this. And our "star programmer" that might is in Greece right now on vacation... =)I had understood it as sourcefiles (.cc) most likely to be different, but as far as I can judge, it seems that they are the same (not 100% confirmed, but all added files since project start in subversion are included in makefile - maybe there is some too many though?). Really wish there was some easy way to compare this, stupid Visual Studio doing everything backwards. ;-)
Could it be other reasons? Script files, mission files, well the whole tree should and seems to be in sync, but apparently some kind of dynamic creation is differing, if I understand this correctly.
Some more friendly tips and pointers in the right direction would be very appreciated, before I dive headfirst into something I don't know where to find. =)
Associate Kyle Carter