Engine client commands

#1 Argv[] is the parameter passed to all console methods ( and functions ) when you call them from script. It is an array of const char*, which correspond to each parameter you passed ( from script ).

I "think" what you are saying you want ... is to send your data directly from C++ rather than from script via command(s). Is that correct?

I wouldn't try to modify the argv array... I would see what the code is doing below that and replicate it in your own function. I haven't looked at that code recently, but it should be Instaciating some type of NetEvent, and then Posting it to a NetConnection. Whatever data/strings are sent in that event are first put into the NetEvent of course. That is the code you want to replicate ( if you want to send NetEvents from C++ ).

You also may want to subclass NetEvent ... for instance if you want your event's data to be received and processed in a particular way ( that is not the same as how command(s) are ). Look at NetEvent.h for more info.

#2 Call me crazy, but that sounds exactly like the stock Torque "Ghosting" system. If you have some data encapsulated in some engine objects and you need to notify all connecting clients when that data changes, that sounds a lot like a standard Torque NetObject.

Stick what data you need in that object, and when the object(s) get an update to their data you set the mask bits on the object and let Torque Ghosting system take over. The data goes into a BitStream in packUpdate and gets taken out on the client(s) in unpackUpdate.

I believe the intention behind Remote Commands was for them to be used for more "ad hoc" net communication, and not for data that will see frequent updating.

#3 Mark - This is very similar to what i'm asking about in the next thread:
http://www.garagegames.com/mg/forums/result.thread.php?qt=75016

You wrote:

Quote:
when the object(s) get an update to their data you set the mask bits on the object ... The data goes into a BitStream in packUpdate and gets taken out on the client(s) in unpackUpdate.

I just don't "Get-it" yet. And have yet to discover the right page of documentation.

- Can you please expand on that, about setting the Mask-Bits and the Bitstream?
- Anyone know the link for the relevant (detailed) documentation regarding the Torque Ghosting system?

#4 If you have data to send from server to client utilizing a NetObject and pack/unpack is one option. But if you are sending from client to server you have only one options, using a NetEvent.

#5 *think* i'm going server -> client - but i'm really not sure yet.

#6 Well, the whole network model often looks like this:
1. An input event from some client (keypress, mouse move).
2. Input event is transmitted to the Sever via a NetEvent (like a Move structure).

Sidenote: Remember that you should never do any "real" processing on the Client side because in a multiplayer client-server environment the server should be the only "true" copy of the game world. Therefore the server should always handle the "true" updating of objects. Therefore when a player presses a button to move forward it might be tempting to just move the player's object forward (I've seen people doing this in code before) but what is more "proper" is to send that off as some sort of "request for movement" to the Server (like the Move structure), whereupon the Server processing the movement of the Player (taking into account all conditions etc) then sends an update back to the client with the results of their request.

3. Server processes the input event and applies its results to the game world simulation.
4. Any affected objects have their data updated. Their Net Masks are set to tell the Torque Network Layer which information to transmit.
5. Torque network scoping rules take over and the changed information is transmitted to clients based on relevance.

Here's TDN article that talks about Ghosting on a generic level.
Here's another high-level article about Torque networking.

Now, based on your comment above you're looking more for concrete examples than theoretical overviews, so let's see what we can find here.

(More coming...)

#7 Hmm, those two links are some good reads. I'll have to check into it more as well as James and Marks comments. The stuff you guys are mentioning is a little over my head, looks like I need to do some more research and digging around. I guess I should have been a little more clear as well. The server creates an object (not an object in the game world), but just in memory that basically keeps track of the top 3 scores in a multiplayer mode for bots and players. When someone dies, this score get's updated on the server and all clients will recieve the new top 3 players with their kill counts. So all I really want to do is send the clients a few variables such as a char* and a couple of int's er S32's, that will update some GUI information. I figured that if you call this function in C++:
sendRemoteCommand(conn, argc - 2, argv + 2); (which is what is being called from the ConsoleMethod)
then the appropriate script function on the client side should be called while passing in those variables......I'm guessing. So my question is how exactly is this array setup? Is it an array of pointers, an array of objects? What is the -2 and the +2 mean?

In general...for those who eventually need to know. How can you make your own argc argv arrays....or whatever and use them for your own specific needs?

......maybe I'll step through it and try and pick it apart......I'll post when I find out.
@Steven - I'm right with ya, when it comes to updating objects I'm lost as well.
@everyone else please post more comments. This is very interesting and useful....Thank You ;-)

#8 (Caveat: This information is merely a reflection of what I understand right now, if someone sees something obviously wrong, please correct me so I can learn too)

First, first! For Ghosting to work, your object must inherit from NetObject somewhere. When you inherit from NetObject you get access to all sorts of essential goodies relating to networking like packUpdate, unpackUpdate, isGhost, setMaskBits through the wonderful auto-magic of inheritance. So, start there. If you're just dealing with some sort of data object (that never gets rendered) then inherit right from NetObject otherwise you're better grabbing from somewhere at SceneObject or below.

Let's look at the example of a Player object receiving a Movement update, that's something that everyone understands. You can see in default.bind.cs that when the player presses "forward" that sets the $mvForwardAction variable (assuming you haven't modified that for whatever reason). Those $mv* variables are "direct" script hooks into the Move object, which is a special object in Torque that sends updates to the server as fast as possible (so it's a little bit of a bad example in that respect). In your case your data might get updated directly by player actions, or it might get updated as a secondary result of player actions. So, the first you need to decide is how to get your update information to the server. Let's gloss over that part for the moment, and say that "somehow" your server-side object has gotten updated.

Looking at the Player object a movement update "affects" the Player object inside of the ProcessTick function. This function gets passed that Move structure that got sent to the Server mere milliseconds earlier. Down around line 1249 you see where it actually uses that info:

updateState();
[b]updateMove(move);[/b]
updateLookAnimation();
updateDeathOffsets();
updatePos();

Now updateMove is an enormous, complex function so don't puzzle yourself with the details. Suffice to say, it uses the movement information combined with the current state of the object and it generates some data changes. You probably have some analogous function in your code that updates your data objects. A little further down you see a function called updatePos. That, as you can imagine, takes some potential player movement, does some collision checks and other goodness, then does something really important at around line 3079:

setPosition(start,mRot);
[b]setMaskBits(MoveMask);[/b]
updateContainer();

What's going on there? Well the object is setting a "code" to let the Network code what has changed since this object has last been sent. There's nothing really "special" about these masks, they are defined (usually) right in the object header, as can be seen in Player.h:

enum MaskBits {
      ActionMask   = Parent::NextFreeMask << 0,
      [b]MoveMask     = Parent::NextFreeMask << 1,[/b]
      ImpactMask   = Parent::NextFreeMask << 2,
      NextFreeMask = Parent::NextFreeMask << 3
};

These are simply bit masks that get additively OR'd by different levels of the object hierarchy that track different parts of the object's data. These are defined by you and should be broken up into groups of data that are most often updated as a group. If your object(s) don't have a lot of data it's usually okay to just resend everything for every update.

(More coming...)

#9 [edit]This post overlapped the previous one - so I had not read it yet.[/edit]

Thanks Mark.

Those docs look helpful. The ghosting one I definitely hadn't found yet.
The other one I only read in out-of-date form elsewhere.

Where Examples are Concerned:
I'm working on a hacked fxRenderObject, so I have all the netcode in front of me, but I don't understand a word of it.
Particularly that first setMaskBits - what does that mean and how does one just "create your own mask" when one needs?

void fxObject::inspectPostApply() {
   setMaskBits(ChangeMask | GradualMask | UpdateMask);
}


U32 fxObject::packUpdate(NetConnection *conn, U32 mask, BitStream *stream) {
   U32 retMask = Parent::packUpdate(conn, mask, stream);
   SimTime currentTime = Sim::getCurrentTime();

   if (stream->writeFlag(mask & ChangeMask))
   {
      stream->writeString(mTextureName);
   }
   if (stream->writeFlag(mask & GradualMask))
   {
      stream->write(mHeight);
      stream->write(mSpeed);
      stream->write(mTargetAlpha);
      stream->write(mTargetTime);
   }
/*   if (stream->writeFlag( mask & UpdateMask )
   {
      stream->write(mTargetAlpha);
      stream->write(mTargetTime);
   }
*/
   return retMask;
}

void fxObject::unpackUpdate(NetConnection *conn, BitStream *stream)
{
   Parent::unpackUpdate(conn, stream);
   if (stream->readFlag())
   {
      mTextureName = stream->readSTString();
      }
   if (stream->readFlag())
   {
      stream->read(&mHeight);
      stream->read(&mSpeed);
      stream->read(&mTargetAlpha);
      stream->read(&mTargetTime);
   }
}

#10 Now, after that Mask gets set we play a little bit of a waiting game. (I believe) The Network code works asynchronously to the Server "ticking" (updates are not "tied" to the server tick rate), so "sometime" after that mask gets set, the Network layer will call packUpdate on your object, just like the Player's version:

U32 Player::packUpdate(NetConnection *con, U32 mask, BitStream *stream)

As you can see the functions receives a NetConnection, Mask, and BitStream object as parameters. That U32 mask is the one you set earlier. Now, right around line 4071 is where that mask gets used/examined:

if (stream->writeFlag(mask & MoveMask))

That code kinda caught me off guard at first, but writeFlag also returns the bit value that was passed into it as well writing the flag into the bitstream. So, that code is a very "tight" way of saying "write TRUE if the MoveMask has been set to TRUE (the player has moved), and if so do the following...". In the subsequent IF block the function begins to write data that could have changed that is considered to be "part" of the group of data that belongs to the "MoveMask." Again, this is a decision made by you, the Class designer, as to which pieces of data belong to which mask. There's nothing really "magical" about it. So, the code goes through and happily writes data into the BitStream. For maximum efficiency always use the "write" function that matches the data type of the variable holding the data, as that will write the minimum amount of bits into the stream, maximizing your use of bandwidth. Once the BitStream has been filled with data it is shipped off to the appropriate client(s).

(More coming...)

#11 @stephen, "how do you create your own mask"

To quote Mark's quote, here is an example of where a Mask is defined, looks like from a stripped down player class...

enum MaskBits {
      ActionMask   = Parent::NextFreeMask << 0,
      MoveMask     = Parent::NextFreeMask << 1,
      ImpactMask   = Parent::NextFreeMask << 2,
      // NewMask = Parent::NextFreeMask << 3,          // Add a new bit for your use like this!
      // NextFreeMask = Parent::NextFreeMask << 4    // And change this value!
};

Notes:

Each object derived from NetObject has its own Mask, including NetObject itself. Each class may define some bits it will use to send its pack/unpack data in an efficient way, then the next class down in the heirarchy will use a few more bits, etc etc... That is why you see "Parent::NextFreeMask" defined at the bottom of the enum -- if it wasn't there then every enum might be a hard-coded value that would have to changed if you, say, needed to add another bit to a class high in the heirarchy.

Sometimes you do have to goto-declaration up the class heirarchy counting bits, to make sure you don't "overflow" by using more than 32, but that shouldn't be a problem for most.

#12 On the client side, the complimentary function unpackUpdate is called, which accepts a NetConnection and a BitStream, which is the same BitStream that your data was written into. Note that this function does not get a Mask value, which is why it's important to actually write those Mask values into the BitStream, that way the Client knows what data is in there, and what it needs to unpack. This is VERY important. Notice that when unpackUpdate runs it (often) passes just a numeric parameter into those read functions, not the name of some variable? That's because, when unpacking data on the client, the update function works purely off of an expected data order, and bit offsets.

When reading data from the BitStream, the read functions simply go and get the next X bits in the stream, based on the data type of the read function: readFlag gets the next bit (just 1), readInt will read the next X bits (the parameter) value as an int and pass it back to you. Thus, the order and size of data written into the stream must exactly match the order and size read out of the stream. Otherwise you're going to be reading data into some variable that was actually intended for another variable... much badness will ensue.

Now can you see why it was important to write the values of those Masks? When our example's MoveMask update gets to the client via the BitStream, the unpackUpdate function will skip over the read sections that don't pertain to updating data relating to the MoveMask. You can see the functions checking the value of bits at certain points in that function and then (optionally) reading in further data if the bit is TRUE. It can be difficult to tell sometimes which flag is being read since the code just calls stream->readFlag(); (for readability it would be nice if it read the bit into a variable with a nice descriptive name), so what I usually do is look in the packUpdate function for the first variable that is written after the Mask value. In packUpdate we see the first variable written after the Mask (line 4073):

if (stream->writeFlag(mask & MoveMask))
   {
[b]      stream->writeFlag(mFalling);[/b]

So just do a search for mFalling in unpackUpdate and you should find the spot (it's not a pretty/perfect method, but it works) at line 4175:

if (stream->readFlag()) { [i]//<- this is reading the MoveMask TRUE/FALSE bit
      mPredictionCount = sMaxPredictionTicks;
[b]      mFalling = stream->readFlag();[/b]
	  mClimbing = stream->readFlag();

Whew! Now that we're all done that, we have some updated on the client. Now, if your data isn't something that is constantly examined on the client, you should probably have some sort of function call or script callback to let your client know that the data has been changed so you can update your GUI or whatever is required.

Consider that a brief, whirlwind tour of Torque Network Ghosting.

#13 @DALO

It sounds like you have a very limited amount of data that needs to be transmitted, and only at the end of the game ... so .... the simplest thing would probably be to just use commandToServer and commandToClient.

You definitely shouldn't need to implement a NetObject and use pack/unpack update since that is for objects that require somewhat continuous/realtime networking.


sendRemoteCommand is not just for sending any arbitrary data ... its for sending data and passing it to a specified script function. And that functions name is represented by the tagged string which is the first argv parameter. In other words "sendRemoteCommand" is designed specificely for commandToServer/commandToClient.

sendRemoteCommand creates a very specific type of NetEvent... a RemoteCommandEvent. Goto its declaration and you will see its "process" method, which is the code that executes when this event is received.

virtual void process(NetConnection *conn)
   {
      static char idBuf[10];

      // de-tag the command name

      for(S32 i = mArgc - 1; i >= 0; i--)
      {
         char *arg = mArgv[i+1];
         if(*arg == StringTagPrefixByte)
         {
            // it's a tag:
            U32 localTag = dAtoi(arg + 1);
            NetStringHandle tag = conn->translateRemoteStringId(localTag);
            NetStringTable::expandString( tag,
                                          mBuf,
                                          sizeof(mBuf),
                                          (mArgc - 1) - i,
                                          (const char**)(mArgv + i + 2) );
            dFree(mArgv[i+1]);
            mArgv[i+1] = dStrdup(mBuf);
         }
      }
      const char *rmtCommandName = dStrchr(mArgv[1], ' ') + 1;
      if(conn->isConnectionToServer())
      {
         dStrcpy(mBuf, "clientCmd");
         dStrcat(mBuf, rmtCommandName);

         char *temp = mArgv[1];
         mArgv[1] = mBuf;

         Con::execute(mArgc, (const char **) mArgv+1);
         mArgv[1] = temp;
      }
      else
      {
         dStrcpy(mBuf, "serverCmd");
         dStrcat(mBuf, rmtCommandName);
         char *temp = mArgv[1];

         dSprintf(idBuf, sizeof(idBuf), "%d", conn->getId());
         mArgv[0] = mBuf;
         mArgv[1] = idBuf;

         [b]Con::execute(mArgc+1, (const char **) mArgv);[/b]
         mArgv[1] = temp;
      }
   }

That code right there is how commandToClient/Server executes a script function on the other side and how it passes the parameters to it.

So if you don't want YOUR net event to do that when it is received, you want it to say ... find the simObject called "highscoreManager" and submit a players score ... then you must make your own NetEvent class that does that in its Process method.

Alternatively just use commandToServer/commandToClient, and be done with it.

#14 Echoing what James said here, because it would have been done days ago ;) Nevertheless, Mark (being one of TorqueSchool's finest instructors) has laid it out pretty well as has James and this has turned into an educational thread others will enjoy for years to come.

My extra cents on the maskbits: the maskbits/netmasks are just an extremely efficent way of packing a bunch of on/offs into a small number. So you can write that number once and it means many different things basically for the server and client to run various bits of code in the pack and unpack functions. A flag is a boolean (true/false.. on/off) value which says hey, this person moved, and here follows all the move information. And the order in which they are written must exactly match the order in which they are read.

If you ran out of net masks somewhere, it's just as fine to make and use and set/clear your own boolean and use that as a mask in the exact same way. IMO people get way too bent out of shape trying to use every last netmask and having objects and classes battle it out to see which one is more important. It should be reserved for the most frequently changing things, as it gets transmitted as part of every packet for that object.

#15 I wanted to thank all of you guys for this fantastic explanation! Mark you really did lay it out clearly, this is exactly what I'd been looking for. Everyone else you comments were very helpful in filling in the gaps.

I've spent the better part of the last two days studying this, and then applying it back to my code. I think I can say I really understand Torque Ghosting/Networking now - and my code is working correctly as a result. :-)

I hope others find this thread as useful as I did.
Steven

#16 I'm glad it helped Steven. I hope other people that come across this thread will be able to gain something from it as well. I will add a clarification on usage here.

If the only feature that is needed is a "simple" list of player names and high scores, that changes not terribly frequently, I admit that going the whole "new NetObject" route is a bit heavy-handed. However, if one wants to emulate some of the (now standard) features of FPS scoreboards like: kills, deaths, shot accuracy, and ping time, I would definitely recommend creating a new NetObject that gets Ghosted to keep track of those things.

#17 As someone who has struggled over the years to understand the network code, I have to say this is a great thread for getting the hang of it. I'm glad to see that most of my understanding of it was confirmed.