Removing client/server communication

#1 It's possible, but not easy by any stretch. I successfully did it as a proof of concept last year, but I cannot share code due to contracts.

Anyway, the basic idea was to hack Torque as so the server-side simulation was rendered instead of the ghosts. I had to:

- The server objects add themselves to the client process list;
- Make only the client process list call processTick() (server process list does nothing).
- Commented out ghosts and datablock transmission;
- Commented out calls to packUpdate()/unpackUpdate()/writePacketData()/readPacketData();
- Hack isGhost(), isServerObject() and isClientObject() so they always return true (this makes both client-only and server-only code work in the same object).
- Hunt and modify all "if" blocks that used isGhost/isServerObject/isClientObject that had "else" statements so both codepaths are executed (or not, depending on the situation).

To make it work, I still had a localClientConnection which performed the internal connection, otherwise I'd need to change far too much code (specially regarding to input). But this was enough to get rid of the limitations imposed by the internal network pipe: the server-side simulation is the one being rendered. Object changes/creation/deletion don't need to go through "network" to be displayed on screen.

But I strongly recommend you try to push the packet size to the maximum value (1500) and see if it serves your needs. If not, try increasing the packet size (it's a define) and using direct access to server-side objects if needed.

#2 Thank you for the fast and detailed reply! Would you be able to tell me what files most of these functions and processes are stored in? We've found a few of them, but we don't have a ton of experience in modifying source code.

Thanks again!

#3 Have you tried adjusting the network settings? By default they are setup for a 28k modem. Might be an easier solution than trying to hack away at all of the code. I believe AFX also caches datablocks, which might help quite a bit as well.

#4 You can turn off hifi networking in your buildfiles/config/project.conf for a quick fix. Just add this line to the file, regenerate your projects and rebuild:

$TORQUE_HIFI_NET = false;

#5 @Tom: that's the problem. It needs changes all over the place. Pretty much any object class you plan to use needs to be verified (Player, Trigger, TSStatic, etc, etc) and requires some familiarity with where things are rendered and updated.

Thus why you should try increasing packet sizes first before going down that route. The client-server communication doesn't eat "performance". Whatever delays you see is the internal bandwidth being maxed out.

The first thing is going into your client/prefs.cs file and setting this value:

$pref::Net::PacketSize = 1500;

It will increase the bandwidth by a huge lot. You can also increase the rate at which packets are sent from server to client:

$pref::Net::PacketRateToClient = "128";
$pref::Net::PacketRateToServer = "128";

This will make a packet be sent at every 8 milliseconds. This means around 6KB of data per tick, which should allow for quite a number of units running around.

BTW, I just found a bug of sorts. Looking at NetConnection::checkMaxRate(), there is code there which is supposed to increase both the rate and the packet size if the connection is local. But this is called from the NetConnection constructor, and at that point it still doesn't know if it's a local connection, so it uses whatever is in the prefs. If you type this in the console, you can force it to re-check and boost the network for local play:

localClientConnection.checkMaxRate();

#6 Regarding Packetsize of 1500 ... has anyone else noticed in 1.1Beta it doesn't work anymore and is capped at 900? Or is it just me?

edit: capped at 1000 actually, 1500 give 476 in-game ...

double edit:
1. rereads Manoel's last paragraph above

2. setting prefs to 1024 gives 1500 in-game

3. Spawning 50 AIs in one go, 1500 gives an "Invalid Packet Gamebase::Unpack Update" and an exit mission.

4. Having my 50 AIs spawn and then increasing packsize to 1500 doesn't cause a problem. Also gives the chance to watch a big change in AI lag from 200->1500.

#7 I've finished all the necessary steps Manoel put up. But the game won't load anything, it stops at where it loads objects on the load screen. I think that's because the client can't find the ghosts. I'm not sure what else I need to do.

Here's the consoleLog:

*** Sending mission load to client: levels/Empty Room.mis
Mapping string: ServerMessage to index: 0
Mapping string: MsgConnectionError to index: 1
onServerMessage:
Mapping string: MsgLoadInfo to index: 2
onServerMessage:
Mapping string: MsgLoadDescripition to index: 3
onServerMessage:
Mapping string: MsgLoadInfoDone to index: 4
onServerMessage:
Mapping string: MsgClientJoin to index: 5
Mapping string: Welcome to a Torque application %1. to index: 6
Mapping string: Visitor to index: 7
onServerMessage:
+- a: Welcome to a Torque application Visitor.
Mapping string: MissionStartPhase1 to index: 8
*** New Mission: levels/Empty Room.mis
*** Phase 1: Download Datablocks & Targets
Mapping string: MissionStartPhase1Ack to index: 0
Mapping string: MissionStartPhase2 to index: 9
*** Phase 2: Download Ghost Objects
Mapping string: MissionStartPhase2Ack to index: 1

Can anyone help?

#8 Most of the networking is short-circuited automatically when you're hosting the game locally. If you enable hi-fi networking (as someone else suggested earlier), you shouldn't really have any issues unless you've got a huge number of objects in the mission.

Though objects are still probably being ghosted via the short-circuit, I know for a fact that the datablocks aren't being networked if you're hosting locally (I did the code for it ages back to speed up load times on TGE and apparently someone was smart enough to roll it into T3D). Instead, they're dumping into and out of a (pretty big) buffer which'll reduce load times considerably. There is, of course, some validation that's done but it's pretty minimal. As far as loading times are concerned on the datablock side of things, you're talking milliseconds of difference in loading times for any reasonable project.

That being said, if you're using something as datablock-bloaty as AFX, well...

#9 I've commented out setGhostAlwaysObject() and ghostWritePacket()/ghostReadPacket(). However, once I did that, the function GameConnection::writePacket(), mControlObject becomes NULL. Then, the game won't enter into the "PlayeGui" in the script after the function GameConnection::readPacket().

Anyone know why mControlObject is now null?

Here's the console log after the script function "dumpProcessList":

==>dumpProcessList();
client process list:
id 8578, order guid 0, type Camera
id 8577, order guid 0, type Camera
id 8571, order guid 0, type SpawnSphere
id 8569, order guid 0, type Sun
server process list:

#10 @Leibang: Since there aren't any ghosts, the client-side thinks it's not controlling anything so it doesn't have a reference point to render from. Without a control object (or camera object), the game just doesn't render anything.

I don't really remember how I got around that. You can try hacking GameConnection::readPacket() so it grabs the control object directly from the server-side connection. In single-player, the client-side GameConnection has a pointer to it's server-side GameConnection object: mServerConnection. You can try grabbing the mControlObject and mCameraObject directly from it instead of reading them from the packet:

S32 gIndex = bstream->readInt(NetConnection::GhostIdBitSize);
GameConnection* serverConn = dynamic_cast<GameConnection*>(mServerConnection.getObject()); //added
GameBase* obj = serverConn->getControlObject();// dynamic_cast<GameBase*>(resolveGhost(gIndex)); //changed

#11 Manoel, here's how I changed the ghost structure:

NetConnection::activateGhosting()
 //pObject = (NetObject*)ConsoleObject::create(pClient->getNetClassGroup(), NetClassTypeObject, mGhostArray[j]->obj->getClassId(getNetClassGroup()));
pObject = mGhostArray[j]->obj;

NetConnection::ghostReadPacket()
//NetObject *obj = (NetObject *) ConsoleObject::create(getNetClassGroup(), NetClassTypeObject, classId); 
NetObject *obj = mRemoteConnection->resolveObjectFromGhostIndex(index);

Now everything appears in the world editor, except nothing renders expect a giant wall of green. What am I missing?

==>dumpProcessList();
client process list:
id 9725, order guid 0, type Camera
id 9695, order guid 0, type TSStatic
id 9694, order guid 0, type TSStatic
id 9693, order guid 0, type TSStatic
id 9511, order guid 0, type SpawnSphere
id 9509, order guid 0, type Sun
server process list:

#12 Thanks a lot to share Information @Manoel, Leibang

me too trying to remove Torque server and client communication

so if i have a success, i'm report this thread :)