[Solved]Crashing when using disconnect() against a dedicated server

#1 Just thinking out loud here - how did you call the disconnect? Did you schedule it or call it directly?

#2 Either leads to the same thing. I've been calling it as disconnect(); straight for a while, and after your comment, I tested with schedule(0, 0, "disconnect"); and get the same thing.

I don't think sending a player to a GUI that covers the screen is out of the ordinary, nor do I think it's out of the ordinary to have players disconnect with schedules running, though I've tried to clean all those up before the player gets to that screen.

#3 So, I did a commandToServer and had the serverCmd call %client.delete(); and got a break at the following line inside NetConnection::eventReadPacket in netEvent.cpp:

while(mWaitSeqEvents && mWaitSeqEvents->mSeqCount == mNextRecvEventSeq)

Callstack:

>	Dodgeball_OPTIMIZEDDEBUG.dll!NetConnection::eventReadPacket(BitStream * bstream=0x0012c58c)  Line 352 + 0x25 bytes	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!NetConnection::readPacket(BitStream * bstream=0x0012c58c)  Line 826	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!GameConnection::readPacket(BitStream * bstream=0x3f800000)  Line 1066	C++
 	Dodgeball_OPTIMIZEDDEBUG.dll!NetConnection::handlePacket(BitStream * bstream=0x0012c58c)  Line 687	C++
 	Dodgeball_OPTIMIZEDDEBUG.dll!ConnectionProtocol::processRawPacket(BitStream * pstream=0x00000374)  Line 236 + 0xa bytes	C++
 	Dodgeball_OPTIMIZEDDEBUG.dll!NetConnection::processRawPacket(BitStream * bstream=0x0012c58c)  Line 654 + 0x8 bytes	C++
 	Dodgeball_OPTIMIZEDDEBUG.dll!NetInterface::processPacketReceiveEvent(NetAddress srcAddress={...}, RawData packetData={...})  Line 86	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!fastdelegate::FastDelegate2<NetAddress,RawData,void>::operator()(NetAddress p1={...}, RawData p2={...})  Line 1075 + 0x49 bytes	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!Signal<void __cdecl(NetAddress,RawData)>::trigger(NetAddress a={...}, RawData b={...})  Line 479	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!Journal::Call<Signal<void __cdecl(NetAddress,RawData)>,NetAddress,RawData>(Signal<void __cdecl(NetAddress,RawData)> * obj=0x10dc3340, void (NetAddress, RawData)* method=0x1046d330, NetAddress a={...}, RawData b={...})  Line 540 + 0x102 bytes	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!JournaledSignal<void __cdecl(NetAddress,RawData)>::trigger(NetAddress a={...}, RawData b={...})  Line 87 + 0x4d bytes	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!Net::process()  Line 586 + 0x38 bytes	C++
 Dodgeball_OPTIMIZEDDEBUG.dll!Process::processEvents()  Line 76 + 0x1e bytes	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!StandardMainLoop::doMainLoop()  Line 577 + 0x5 bytes	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!torque_enginetick()  Line 104	C++

 Dodgeball_OPTIMIZEDDEBUG.dll!torque_winmain(HINSTANCE__ * hInstance=0x00400000, HINSTANCE__ * __formal=0x00000000, char * lpszCmdLine=0x002f28c6, HINSTANCE__ * __formal=0x00000000)  Line 445 + 0x32 bytes	C++

 Dodgeball_OPTIMIZEDDEBUG.exe!WinMain(HINSTANCE__ * hInstance=0x00400000, HINSTANCE__ * hPrevInstance=0x00000000, char * lpszCmdLine=0x002f28c6, int nCommandShow=1)  Line 47 + 0x23 bytes	C++

 Dodgeball_OPTIMIZEDDEBUG.exe!__tmainCRTStartup()  Line 324 + 0x35 bytes	C

 Dodgeball_OPTIMIZEDDEBUG.exe!WinMainCRTStartup()  Line 196	C

So if I disconnect from the client, it blows up while ticking, and if I disconnect from the server, it blows up reading an event packet.

#4 Just wondering if anyone else has ideas on this kind of issue?

#5 There is a ticket for a crash when disconnecting from a dedicated server. It was never reported in the forums, something we found ourselves before the preview.

According resolution date on the ticket the fix should be in the Preview, but that particular ticket hasn't made it through verification yet in QA. The issue stemmed from particle emitter nodes and an order of operations error in their cleanup.

Do you have it happening in a stock install and is your project based on the Full or Empty template?

#6 It's based on Full, and I didn't report it as a bug because I've got some networking and engine mods done, as well as changes to the connection flow based partly on the RTS pack (to allow for game lobbies, so there's a two-part onConnect being done, though disconnecting after a player is out is still supposed to be handled via a disconnect() call from the client, mainly because I want to hang onto client objects on the server for other operations after the game is done but before the server shuts down or recycles- except for the way I've structured connections, it's not that funky).

I also have particle emitters in play right now, so I'll see about taking them out tomorrow and see if that fixes the crashing. Thanks for the heads-up.

#7 No problem, here's Dave's comment from the ticket what the fix was:

Quote:
it ended up being an order of operations issue with mission clean up. These particle emitters were generated by particle emitter nodes and put into the ClientMissionCleanup group. This group was being cleaned up following the server connection's destruction in scripts/client/serverConnection.cs disconnect(). The problem with this is that the ParticleEmitter's Datablock is deleted with the server connection's deletion. So any work with the datablock during the emitter's removal resulted in a crash.

The solution is to place the mission clean up prior to the server connection's deletion. This never showed up before because prior to THREED-1208, the mission clean up did not correctly execute.

Before you remove all your particle emitters I'd check where the missionCleanUp is happening first. This may just be a case of the bug getting ported in by accident with your custom code.

#8 No dice...

I replaced my modified serverConnection.cs with one from a new project generated with a full template so the disconnect(), disconnectedCleanup(), and onConnectionDropped() were all shiny and new. Using a client-side disconnect() with no schedule, it crashes in stdGameProcess.cpp around line 305:

if ( pobj->mIsGameBase && con && con->getControlObject() == pobj )

mIsGameBase is true according to mouse hover in debug, though I don't get anything for the other values, so maybe the crash occurs around the check on the con variable?

Next, I tried a schedule(0, 0, "disconnect") and got a crash at the same line. Using a schedule of (50, 0, "disconnect"), which forces it to wait for the next frame, I believe, results in a crash in processList.cpp around line 251 (inside advanceObjects():

onTickObject(pobj);

So no dice for working on client-side issues...

Next I disabled the particle emitters and explosions by commenting them out, and got the same result for the schedule(50, 0, "disconnect");. I don't think there's much point to test it without the schedule, if giving it time to clean up results in a crash.

So the next test was to see if my server-side script modifications to clientConnection.cs were doing it. I can't replace the whole script because I have other functions in there, but I can replace GameConnection::onDrop(), which I commented out and replaced with a shiny new generated version. The schedule(50, 0, "disconnect"); remains on the client side.

Results? Crash, same place.

And with both the new onDrop and new serverConnection.cs? Same thing.

A few more notes:
- I'm not using the package system as referenced in the THREED-1208 issue, mainly because it's way too confusing to be deployed with any Torque release as a default configuration, so when I work on projects I rip out the packages and use the regular GameConnection:: calls. I don't believe this has any bearing on the issue except to clarify troubleshooting the scripts.
- For GameConnection, I've ripped out damage flash functionality, since my game doesn't use it and it's a very close quarters action game with up to 50 players within an arena about 200 meters across (with most player visible most of the time), so there's been a lot of network optimization- but for GameConnection, that's it.

#9 clearCameraObject()

...sigh...I figured that deleting the camera- or at least holding off on that deletion until the server cycled- would be enough, but that's what was needed.

Problem solved.