How to transmit data structures between server and client
by Ingo Seidel · in Torque Game Engine · 06/27/2008 (5:47 am) · 4 replies
Hello!
I am working on a project where the Torque server is connected to another system and exchanges information with that system. The client should be able to receive information from this system. The process is roughly as follows: a user presses a button in the 3D world - message to Torque server - message to external system - message back to Torque server - message to Torque client - display information to user.
The problem that I am having now: is how can I transmit complex data structures to the client? Conceptually it works like this: The C++ code of the Torque server receives the information, calls a server side script function and sends a message to the client side script code (note that client and server are two different executables). This works well for primitive data like strings that can be easily exchanged via the "commandToClient" methods. However, in my case I have extended the Player class in the C++ code base, created a custom data structure and have a pointer to this data structure in the extended class. This is how it looks like:
The server receives the information creates a new data structure and sets it on the SERVER side player object with MyPlayer::setInformation. The server then calls a server side script method and sends the notification that the response has arrived, back to the client. The client script should now be able to call the
getCertainInformation() method on the player object. However, when I am doing this the data field has not been set as the Torque server did only set the information on the SERVER side player object.
Now my actual question: how can the information be propagated to the client? Is this what I am trying do here possible/reasonable at all in Torque? What about the setMaskBits function? I have read that this can be used to notify the server to transmit changed data to the client. But how can this be used and where do I define which kind of information needs to be retransmitted on which bit mask?
thanks
I am working on a project where the Torque server is connected to another system and exchanges information with that system. The client should be able to receive information from this system. The process is roughly as follows: a user presses a button in the 3D world - message to Torque server - message to external system - message back to Torque server - message to Torque client - display information to user.
The problem that I am having now: is how can I transmit complex data structures to the client? Conceptually it works like this: The C++ code of the Torque server receives the information, calls a server side script function and sends a message to the client side script code (note that client and server are two different executables). This works well for primitive data like strings that can be easily exchanged via the "commandToClient" methods. However, in my case I have extended the Player class in the C++ code base, created a custom data structure and have a pointer to this data structure in the extended class. This is how it looks like:
public MyPlayer : public Player ...
public MyDataStructure : public SimObject ...
MyPlayer::setInformation(MyDataStructure *data)
{
setMaskBits(...); // don't know if needed/which to set
this->data = data;
}
const char*
MyPlayer::getCertainInformation()
{
return data->getSomething();
}
ConsoleMethod(MyPlayer, getCertainInformation, 2,2,"")
{
int size = dStrlen(getCertainInformation()
char* buf = Con::getReturnBuffer(size);
dSprintf(buf, size, "%s", object->getCertainInformation());
return buf;
}The server receives the information creates a new data structure and sets it on the SERVER side player object with MyPlayer::setInformation. The server then calls a server side script method and sends the notification that the response has arrived, back to the client. The client script should now be able to call the
getCertainInformation() method on the player object. However, when I am doing this the data field has not been set as the Torque server did only set the information on the SERVER side player object.
Now my actual question: how can the information be propagated to the client? Is this what I am trying do here possible/reasonable at all in Torque? What about the setMaskBits function? I have read that this can be used to notify the server to transmit changed data to the client. But how can this be used and where do I define which kind of information needs to be retransmitted on which bit mask?
thanks
#2
06/27/2008 (3:03 pm)
You could use NetEvents for one time shots like this or the RPC interface in TorqueScript. BTW the user should press the button on the server, not their client, this way the server will remain authoritative. Then some hacker can't just send the "press this button" message to the server when they are really no where near it.
#3
If I understand this process correctly, when having a vector of custom data structures, I would have to do the following:
Would this be the way how the data packing process should be handled? What if my custom data structure not only contains primitive types but again containers like std::vector, will I have to unfold the whole information and write it into the bitstream (including meta information to know where certain information starts and stops)?
06/30/2008 (6:26 am)
Great thread, contains a lot of useful information, however, I am still a bit uncertain about how things work. If I want to send my own data structures, I will have to overwrite the packUpdate and unpackUpdate functions, right? In these functions I have to manually walk through my data structure, write the information to the stream and then retrieve it on the client to set the values of the client object to the values in the stream. This is a quite cumbersome process, is there no easier way to achieve this? Is it not possible to just set data on the sever (i.e. a member of a server/client object pointing to a certain data structure), issue an update and have this information on the same object on the client side (i.e. the class member contains the information that has been set on the server side object)?If I understand this process correctly, when having a vector of custom data structures, I would have to do the following:
MyPlayer : public Player
{
std::vector<MyDataStructure*> myData;
....
}
MyPlayer::packUpdate(...)
{
// for all data structures in std::vector myData
// call MyDataStructure::packUpdate(...)
}
MyPlayer::unpackUpdate(...)
{
// create a new myData std::vector
// create a new MyDataStructure for every chunk of information in the bistream
// put the data structure in the myData vector
MyDataStructure::packUpdate(...)
{
// write all fields of the data structure to the bitstream
}
MyDataStructure::unpackUpdate(...)
{
// retrieve information from bitstream
// set fields in the object based on the position of information in the bitstream
}Would this be the way how the data packing process should be handled? What if my custom data structure not only contains primitive types but again containers like std::vector, will I have to unfold the whole information and write it into the bitstream (including meta information to know where certain information starts and stops)?
#4
yes, there is no magical way to transmit an arbitrary data structure down the wire, you need to serialize & deserialize it yourself. this makes a lot of sense if you look at it from a network efficiency point of view. "expert" serialization allows the bitstream to be far more efficient. for example if your datastructure contains an S32 which only ever has one of a handful of values (eg 0 1 2 3), you really only need two bits to send it down the stream, not the full 32 bits a non-expert serialization would use.
also, you probably want to think about When this data structure is sent down.
if it's a whole lot of data, you might want to find a way to parameterize it or avoid sending it at all.
if it's a moderate amount of data which doesn't change once the object has been instantiated, you could consider putting it in the object's datablock, and serializing it in the datablock's packData() and unpackData() methods. if it's a lot of data which changes over time, you may want to look at breaking it up into different sections which tend to change at the same time, so that you don't retransmit the whole set just because one tiny little part changed. there's something of an art to this.
you'll definitely want to learn a bit about the "netmasks" associated with ghosting.
there's some stuff to read in TDN if you search for "ghosting" or "packupdate".
06/30/2008 (9:21 am)
Hubertus -yes, there is no magical way to transmit an arbitrary data structure down the wire, you need to serialize & deserialize it yourself. this makes a lot of sense if you look at it from a network efficiency point of view. "expert" serialization allows the bitstream to be far more efficient. for example if your datastructure contains an S32 which only ever has one of a handful of values (eg 0 1 2 3), you really only need two bits to send it down the stream, not the full 32 bits a non-expert serialization would use.
also, you probably want to think about When this data structure is sent down.
if it's a whole lot of data, you might want to find a way to parameterize it or avoid sending it at all.
if it's a moderate amount of data which doesn't change once the object has been instantiated, you could consider putting it in the object's datablock, and serializing it in the datablock's packData() and unpackData() methods. if it's a lot of data which changes over time, you may want to look at breaking it up into different sections which tend to change at the same time, so that you don't retransmit the whole set just because one tiny little part changed. there's something of an art to this.
you'll definitely want to learn a bit about the "netmasks" associated with ghosting.
there's some stuff to read in TDN if you search for "ghosting" or "packupdate".
Torque 3D Owner Mark Dynna