Ghosting Limits

#201 Fixed the rayCast... :D

#202 Yay!

#203

#204

And we're back!

It's time for another exciting instalment of learning to make things with Torque. Today we'll add dynamic networking of the cube list, so that when the server makes a change to the cubes, the clients will be updated in an efficient manner.

We could, of course, just re-send the list of cubes every time they change (in the same fashion we use to send all the cubes in packUpdate when the object is first ghosted), but this is really inefficient for small changes, like adding or removing one cube at a time. So what we'll do is create a custom NetEvent that will be sent to each client when a cube object changes. This event will contain the ID of the cubelist object that changed, and the index of the cube within that cubelist that was removed.

Today we'll just implement removal. I'll leave adding as an exercise for the reader ;).

The code for the complete Cubes object with networked removal is here. To remove a cube, use the 'removeCube' script method with a cube position (i.e. %obj.removeCube(0, 0, 0); to remove the cube at position 0, 0, 0).

To achieve this, I first had to add the removeCube methods. This means adding them to the header file after the definition of addCube:

void removeCube(F32 x, F32 y, F32 z);
   void removeCube(U32 index);

Note there are two - one removed by position, the other by index. Only position is exposed to scripts, because I'm lazy.

And their implementations look like this - let me know if there's anything you don't understand.

void Cubes::removeCube(F32 x, F32 y, F32 z)
{
   Vector<Cube>::iterator it;
   U32 i;
   for(i = 0, it = cubeList.begin(); it != cubeList.end(); it++)
   {
      if (it->position == Point3F(x, y, z))
      {
         break;
      }
      i++;
   }
 
   if (it != cubeList.end())
   {
      removeCube(i);
   }
}
 
DefineEngineMethod(Cubes, removeCube, void, (F32 x, F32 y, F32 z),,
                   "Removes the cube at this location")
{
   object->removeCube(x, y, z);
}
 
void Cubes::removeCube(U32 index)
{
   if (index < cubeList.size())
   {
      cubeList.erase(index);
 
      if (isServerObject())
      {
         SimGroup* pClientGroup = Sim::getClientGroup();
         for (SimGroup::iterator itr = pClientGroup->begin(); itr != pClientGroup->end(); itr++)
         {
            NetConnection* nc = static_cast<NetConnection*>(*itr);
            CubeRemovedEvent* pEvent = new CubeRemovedEvent(this, index);
            nc->postNetEvent(pEvent);
         }
      }
 
      if (isClientObject())
      {
         createGeometry();
      }
   }
}

That bit with Sim::getClientGroup simply iterates over every client connected to the server, and sends them a CubeRemovedEvent, which we'll define now. This event is a special type of NetEvent that tells a client to remove a cube when it's received.

#205 Back in the header file, let's define our new type of NetEvent. For a detailed overview of how NetEvent works and what you need to do to define a new one, I recommend this page. I'll just present code here:

class CubeRemovedEvent : public NetEvent
{
   typedef NetEvent Parent;
   SimObjectPtr<Cubes> mObj;
   U32 mIndex;
public:
   CubeRemovedEvent(Cubes *obj, U32 index);
   ~CubeRemovedEvent();
 
   virtual void pack   (NetConnection *conn, BitStream *bstream);
   virtual void write  (NetConnection *conn, BitStream *bstream);
   virtual void unpack (NetConnection *conn, BitStream *bstream);
   virtual void process(NetConnection *conn);
 
   DECLARE_CONOBJECT(CubeRemovedEvent);
};
 
IMPLEMENT_CO_CLIENTEVENT_V1(CubeRemovedEvent);

Just stick this at the end of your header file. The first three lines of the definition are the relevant ones: we store a pointer to the Cubes object that will be modified, and the index of the cube we want to remove.

Now let's actually implement the event's methods for transmitting data and being processed once the data is transmitted.

CubeRemovedEvent::CubeRemovedEvent(Cubes* obj = NULL, U32 index = 0)
{
   mObj = obj;
   mIndex = index;
}

CubeRemovedEvent::~CubeRemovedEvent()
{
}

void CubeRemovedEvent::pack(NetConnection* conn, BitStream *stream)
{
   S32 ghostIndex = conn->getGhostIndex(mObj.getPointer());
   if (stream->writeFlag(ghostIndex >= 0))
   {
      stream->writeRangedU32(U32(ghostIndex), 0, NetConnection::MaxGhostCount);
      stream->writeRangedU32(mIndex, 0, 1024);
   }
}

void CubeRemovedEvent::write(NetConnection* conn, BitStream *stream)
{
   pack(conn, stream);
}

// The networking layer takes care of instantiating a new
// CubeRemovedEvent, which saves us a bit of effort.
void CubeRemovedEvent::unpack(NetConnection *conn, BitStream *stream)
{
   if (stream->readFlag())
   {
      U32 ghostIndex = stream->readRangedU32(0, NetConnection::MaxGhostCount);
      mObj = dynamic_cast<Cubes*>(conn->resolveGhost(S32(ghostIndex)));
      mIndex = stream->readRangedU32(0, 1024);
   }
}

void CubeRemovedEvent::process(NetConnection *conn)
{
   if (mObj.isValid())
   {
      mObj->removeCube(mIndex);
   }
}

Basically, the server calls pack to put all the event's data into a stream going to a client, then the client calls unpack when it receives the event, then the client calls process to make the event work. The trickiest part here is sending the ghost ID of the Cubes object over the wire. Let me know if you need some more explanation there.

Sorry this is kind of rushed! Though you've indicated that you learn best from code so I've not put as much information into the explanation as I might have otherwise.

Warning: don't remove the last cube, because currently createGeometry() doesn't like it when you have a cubeList of length 0. Not sure if you've fixed that, but you might have to if you intend to let all cubes be removed.

Also, this does not currently have any method for you to remove a cube using a raycast. Since you have to give a precise position to removeCube, it's a little clunky. I tried to get this out as quickly as possible, but I'm also not sure exactly how you're handling raycasts and stuff any more. I could recommend a couple of options; let me know if you'd like to hear about them.

To support adding cubes, you'll probably want to create a new type of NetEvent that works the same way, and modify addCube to send it from the server, and to call createGeometry on the client. This event will obviously need the same data as addCube - the Cubes object, the position of the new cube, and its material. And then you'll also need a way to come up with the position to add the new cube at.

#206 Wowzers! Hey thanks Danny, this is extremely valuable :) I've been a tad busy working with some art assets but I'm going to give this a solid workthrough tomorrow! Dude you rock!

P.S. As usual, if I hit any snags I'll be sure to let ya know here after I test it out :)

#207 Alright, got a chance to plug this in this morning. Good news and bad news:

The good news is I was able to integrate all the code with the new NetEvent class with no problems. Compiles fine, no crashes etc. I changed the removeCube() function to use a Point3F value instead of the x, y, z coordinates (mainly just due to how the Torque script functions are passing around vectors and not 3 separate points). Small change, no big deal, it didn't appear to have any effect on the following problem if I used 3 position values or a Point3F:

The bad news is I can run the removeCube() function and it will run without errors but no cube is removed. I have each Cell(in your example the class is named Cubes, I use the name Cell for the class) indexed using an ArrayObject in Torque. This way I can call script functions to see that the number of cubes in the Cell's cubeList doesn't change. Additionally(more importantly), the cube visually doesn't update.

I'm a bit perplexed where the issue lies, since it all compiles and runs the functions fine. Here's a quick test script I used to test it out(I know, it's a bad hack, but it was just to try and see some sort of results quickly before hooking all this into the rayCast):

Check out my comments, you can see I use a few simple functions that are just extensions of the Cell class.

function testRemove(%pos)
{
	// Cells is the name of the SimGroup
	foreach (%obj in Cells)
	{
		// These functions work fine, they are
		// extensions of the Cell class I added
		// in the C++ source file
		%size = %obj.getSize();
		%cellPos = %obj.getPosition();
		
		for ( %i = 0; %i < %size; %i++ )
		{
			// getCubePosition(%i) uses the given index
			// to get the position of the Cube in the 
			// cubeList. Another extension of the Cell class
			%cubePos = %obj.getCubePosition(%i);
			%worldPos = VectorAdd(%cellPos, %cubePos);

			// If the positions match up, remove it!
			if (%worldPos == %pos)
			{
				%obj.removeCube(%pos);
			}
		}
	}
}

So firstly, all that function is doing is iterating the Cell objects in the Cells SimSet. Then it's retrieving the size and position of each Cell. Iterating over the size of each Cell, it retrieves the 'cubePosition' of each cube (using the size variable as an index). If the found position matches the position input to the function, we call our new removeCube() function! But alas, this yields no results :/

#208 Have you tried using a breakpoint inside the source code for removeCube? See if it ever sends those NetEvents! Also, check your position maths in that script fragment you posted. Specifically, make sure that you're using %pos and %cubePos correctly.

#209 Thanks for the reply Danny, I did drop a Con:printf into the CubeRemovedEvent() like so:

CubeRemovedEvent::CubeRemovedEvent(Cell* obj = NULL, U32 index = 0)
{
	mObj = obj;
	mIndex = index;
	Con::printf("CubeRemovedEvent");
}

...and it didn't echo when I ran the testRemove() script I posted. Also note above that I changed Cubes* to Cell* for the class. I did this anyplace I found Cubes* (in unpack).

Anyhow, when I was looking over the pack/unpack methods I noticed that:

mObj = dynamic_cast<Cubes*>(conn->resolveGhost(S32(ghostIndex)));

...is in unpack but not in pack. Should this data be being packed prior to the unpack?

Besides this, am I correct in assuming the sample files will remove the cubes for you? I hadn't tried to just plug in your .cpp and .h files yet to test them. I wanted to learn as much as possible, so I went line by line through your posts and worked in all the additions. I did have to add the extra include to the .h file that you didn't mention above, np :) If your sample files do indeed work to remove dynamically for you, perhaps a sample of the script you use with it could help :) I'm sure if they work for you I'm overlooking something and may need to peruse your samples closer.

#210 Right, I suspect that you're passing incorrect positions to removeCube. What is %pos? An absolute world position? Shouldn't you then call removeCube(%cubePos)? Or you could expose the other version of removeCube (which takes an index) and call removeCubeAtIndex(%i).

In pack, we already have a handle to the object, and we don't need to do any casting. In pack we do this:

S32 ghostIndex = conn->getGhostIndex(mObj.getPointer());
if (stream->writeFlag(ghostIndex >= 0))
{
   stream->writeRangedU32(U32(ghostIndex), 0, NetConnection::MaxGhostCount);

Which sends the object's ID over the network. But when we unpack, we can only convert that ghost ID number into a NetObject*. We want to store a Cubes* (or Cell*), so we need to dynamically cast it to a pointer of the right type.

Casting is kind of tricky and messy. But you can think of it as recovering type information we're pretty sure we should have. When you transfer an object ID over the network, you lose that type information. So on the unpacking end, we need to recover that type information by casting the pointer to the type we think the object should have. Since we always send IDs of Cubes objects, we can be pretty certain we'll always receive the ID of a Cubes object.

(dynamic_cast will return a NULL pointer if the object is not in fact of the type we think it will be, so we have a little bit of safety there.)

#211 Well I'll be...I completely overlooked the fact that our removeCube() is using local coordinates(of the Cell) for removal.

Thanks for the heads up Danny, I just changed the script like you recommended:

...
			// If the positions match up, remove it!
			if (%cubePos == %pos)
			{
				%obj.removeCube(%cubePos);
			}
...

...and it works! Of course, my test script is yanking the cubes located at the given coordinates in every Cell object...but that's to be expected given the script lol. It's a great start though, and now that I have confirmed it works I can tie it into the rayCast! Like you said, I probably will end up exposing the cube removal function that uses the id :)

Very cool stuff, glad it was just a mess up on my part with the script! Thanks Danny, this is going to be the center of much study for me around NetEvents. This really changes the playing field, awesome!

#212 Ooh, one thing I forgot. The way you've written your script, it should only remove one cube. I think it's removing many cubes because of this: %cubePos == %pos. You should use %cubePos $= %pos, because vectors are strings in TorqueScript.