Ghosting Limits
by Jesse Allen · in Torque 3D Professional · 09/08/2014 (12:47 am) · 212 replies
Greetings fellow Torque-goers! I hadn't been overly active on the boards lately, but that's usually a good thing since it means I'm busy actually developing stuff :) Finally hit a snag today, and I was hoping someone a bit more knowledgeable about these things could perhaps point me in the right direction.
I've been working with generating 1000's of cubes, which I have been largely successful with. I have scripted algorithms to do exactly what I need as far as generation is concerned. I've even managed to work out some basic culling and so on. The problem I'm having, however, is with the following error:
NetConnection::object In Scope: too many ghosts.
I did a bit of searching and I did stumble on Vince Gee's fantastic resources for Limiting Shapebase ghosting and Improved Limiting ghosting . These do appear to be useful by their own rights and I will definitely explore these options. However, these particular resources deal specifically with ghost limiting as related to view distance.
The problem I am having seems to stem from the actual hard limits set on the maximum number of allowed ghosts. I was curious:
Also, bonus points for a clear explanation on how view distance affects ghosts in the first place. If I generate in a bunch of cubes and use the command ServerConnection.getGhostsActive() I can see the current number of ghosts. Check. But if I then reduce the viewDistance to an absurdly low amount (such as 10) and run far away from the generated cubes...the command will still show the same number of ghosts, regardless of my distance from them.
Cheers guys, as always thanks in advance for any help.
I've been working with generating 1000's of cubes, which I have been largely successful with. I have scripted algorithms to do exactly what I need as far as generation is concerned. I've even managed to work out some basic culling and so on. The problem I'm having, however, is with the following error:
NetConnection::object In Scope: too many ghosts.
I did a bit of searching and I did stumble on Vince Gee's fantastic resources for Limiting Shapebase ghosting and Improved Limiting ghosting . These do appear to be useful by their own rights and I will definitely explore these options. However, these particular resources deal specifically with ghost limiting as related to view distance.
The problem I am having seems to stem from the actual hard limits set on the maximum number of allowed ghosts. I was curious:
- Is it possible to increase the number of ghosted objects?
- If it were possible, what sorts of problems would this increase potentially introduce?
Also, bonus points for a clear explanation on how view distance affects ghosts in the first place. If I generate in a bunch of cubes and use the command ServerConnection.getGhostsActive() I can see the current number of ghosts. Check. But if I then reduce the viewDistance to an absurdly low amount (such as 10) and run far away from the generated cubes...the command will still show the same number of ghosts, regardless of my distance from them.
Cheers guys, as always thanks in advance for any help.
About the author
Skilled Artist and Musician. Intermediate Torque Developer.
#142
EDIT: Yep, compiled fine. Projectiles still pass through the object though, and any raycast queries return nothing(using Danny's function). So close!!!
09/24/2014 (2:17 pm)
Makes sense, Andrew. I had wondered about this specifically in the past when altering engine files. This clarifies other problems I've had in the past as well. I bet if I do it as you've said it will be fine :) I'll report back soon, if you got it to compile that's got to be it!EDIT: Yep, compiled fine. Projectiles still pass through the object though, and any raycast queries return nothing(using Danny's function). So close!!!
#143
09/24/2014 (2:37 pm)
Weird. The raycast works for me, and the grenade launcher from the Full template collides fine with the cubes. However, the projectiles don't bounce off - they sort of get stuck on the cubes and rise slowly, eventually ending up at the top of the stack before exploding. It's bizarre - I've stepped through the maths and everything *appears* to be working fine. The collision is registered, the projectile's velocity is mirrored around the normal... no idea where this behavior comes from.
#144
Every time I shoot a cube or send a projectile through it, it spits out the correct number in the console. No idea why it chooses to just ignore this result o.O
09/24/2014 (2:44 pm)
Using this shows me it's hitting the right cube, but doesn't seem to care:if(t < info->t)
{
// Yup! Fill in the RayInfo's data so whoever's casting the ray has
// access to it.
hit = true;
info->t = t;
info->normal = n;
info->material = mMaterialInst;
info->object = this;
Con::printf("HIT CUBE NUMBER: %d", i);
}Every time I shoot a cube or send a projectile through it, it spits out the correct number in the console. No idea why it chooses to just ignore this result o.O
#145
09/24/2014 (3:16 pm)
Yep, confirmed with the console echo. I'm getting the same results as Andrew. It's registering the hit when shooting, even giving a different number for each cube! Just choosing not to stop the projectile lol.
#146
09/24/2014 (3:36 pm)
Time to put some breakpoints in Projectile::simulate!
#147
Check it out, this is what I have in script to create the Submatrix in the first place:
I was wondering what I should do to actually call 'addCube' in script. I am certain I need to somehow just say Submatrix.addCube but once the object is created I'm overlooking how to actually call on the object outside of the function. Hope that makes sense :P I tried adding a class to the object and calling it there, but the console berates me saying it can't find the object ' ' trying to call addCube.
09/24/2014 (3:45 pm)
On another note, about the DefineEngineMethod stuff: I added the DefineEngineMethod for addCube at the bottom of the .cpp file and it yelled at me about not being accessible. So I visited the declaration in the header file and changed it to be included under 'public:'. Success! So I now have the engine method created, but I'm having a little bit of trouble actually using it in script. I think I'm just stumbling over something simple, so I figured I'd ask here. Check it out, this is what I have in script to create the Submatrix in the first place:
new SimGroup(World);
function World::Submatrix(%this, %x, %y, %z){
%obj = new Submatrix(){
position =%x SPC %y SPC %z;
rotation = "1 0 0 0";
scale = "1 1 1";
};
}I was wondering what I should do to actually call 'addCube' in script. I am certain I need to somehow just say Submatrix.addCube but once the object is created I'm overlooking how to actually call on the object outside of the function. Hope that makes sense :P I tried adding a class to the object and calling it there, but the console berates me saying it can't find the object ' ' trying to call addCube.
#148
In that context, %obj is the instance of your C++ object. So you would do:
09/24/2014 (4:13 pm)
new SimGroup(World);
function World::Submatrix(%this, %x, %y, %z){
%obj = new Submatrix()
{
position =%x SPC %y SPC %z;
rotation = "1 0 0 0";
scale = "1 1 1";
};
}In that context, %obj is the instance of your C++ object. So you would do:
%obj.addCube(...);
#149
I've also just run into another problem. I can place the Submatrix either with the function above or in the levelInfo and I'm getting the same results. What happens is I add a material to the object, but the object is invisible regardless. Up until now, with all of the screenshots I've been taking, I've been following a specific procedure to take the screenshot:
It appears that if the object is created with the material already declared it won't show up. Which I know, is very weird. It's as if I have to update the object manually in some way for the change to show.
09/24/2014 (4:21 pm)
Thanks Andrew, but that's the problem. I'm trying to add the cube after that function has already happened. In other words, I've lost hold of the %obj. Like I build the world with these Submatrix %obj, then the player is going to want to add a cube. How does he get a hold of the %obj again? In the end, I suppose it'll all tie into a proper working raycast. But for now I was just trying to test it out to see if I could add a cube at all once the Submatrix object was placed. I've also just run into another problem. I can place the Submatrix either with the function above or in the levelInfo and I'm getting the same results. What happens is I add a material to the object, but the object is invisible regardless. Up until now, with all of the screenshots I've been taking, I've been following a specific procedure to take the screenshot:
- Call the function above: World.Submatrix(0,0,0);
- Click on the object in the World Editor, and choose the atlas material.
It appears that if the object is created with the material already declared it won't show up. Which I know, is very weird. It's as if I have to update the object manually in some way for the change to show.
#150
By comparing our updated file to the original RenderMeshExample, I found it pretty quickly and just added this back in. Cheers!
09/24/2014 (4:37 pm)
Okay I found the problem with the materials. In the updated RenderMeshExample.cpp, somehow we lost a very important line in the onAdd:// Refresh this object's material (if any) updateMaterial();
By comparing our updated file to the original RenderMeshExample, I found it pretty quickly and just added this back in. Cheers!
#151
So, you'll have to do add the material parameter to initializing your object like so:
09/24/2014 (4:39 pm)
I saved a working one and checked the level file:new RenderMeshExample() {
Material = "cubeMat";
position = "2.07322 -0.737378 -0.000384271";
rotation = "1 0 0 0";
scale = "1 1 1";
canSave = "1";
canSaveDynamicFields = "1";
};So, you'll have to do add the material parameter to initializing your object like so:
new SimGroup(World);
function World::Submatrix(%this, %x, %y, %z){
%obj = new Submatrix()
{
Material = "cubeMat";
position =%x SPC %y SPC %z;
rotation = "1 0 0 0";
scale = "1 1 1";
};
}
#152
EDIT:
That's what I mean, I was declaring the material when initializing it but to no avail. It's just because the updateMaterial() function was missing, no big deal it's all working :) You may not be seeing it if you're not using the same exact file, but I'm certain some of the prior ones linked on gist.github were missing that one line. I had followed suit at the time, thinking you might have had a reason for excluding it. Luckily, since I noticed it before it wasn't hard to go back and replace it.
Still unable to catch ahold of my object after it's been placed though, so I can test an addCube :)
09/24/2014 (4:48 pm)
See post #150 above. I had noticed it was missing before, but I thought there was some reason for it. It works fine just adding that line back to the onAdd function. Take a look at your RenderMeshExample.cpp. You'll find that line missing :)EDIT:
That's what I mean, I was declaring the material when initializing it but to no avail. It's just because the updateMaterial() function was missing, no big deal it's all working :) You may not be seeing it if you're not using the same exact file, but I'm certain some of the prior ones linked on gist.github were missing that one line. I had followed suit at the time, thinking you might have had a reason for excluding it. Luckily, since I noticed it before it wasn't hard to go back and replace it.
Still unable to catch ahold of my object after it's been placed though, so I can test an addCube :)
#153
09/24/2014 (5:07 pm)
I think the best approach to getting the object after it's created is to add it to the World group, or even create a new SimGroup inside World just for your terrain objects. Like so:new SimGroup(Matrix);
function World::Submatrix(%this, %x, %y, %z) {
...
Matrix.add(%obj);
}Now you can iterate over Matrix using Matrix.getCount() and Matrix.getObject(%i).
#154
09/24/2014 (5:14 pm)
Yep, you're right Danny. I was overlooking a proper index of the Submatrix object lol. So much time we've been spending in C++ world; I could have just revisited some of my assembled scripts I was using for the TSStatic cubes...lol. Thanks!
#155
The SimGroups
Now, if I do go ahead and add the object to a simgroup with:
I assume atm, for the most part the problem is out of my hands due to the complexity of the castRay function? If there's anything else I should try, I'm all for it. Thanks guys! I realize this is definitely a little hairier than expected, especially due to the castRay stuff.
09/24/2014 (6:48 pm)
Alright, here's where I'm at so far. I created a serverCmd function to go ahead and perform a raycast to see if I could get a hit at all first. Figured I'd post it up here to save anyone some time if they're trying to perform the raycast. Now, this one's just doing the raycast from the player's eyepoint etc. for now. I'll have to conjure up a different one later on for isometric view:function serverCmdInteractFPS(%client)
{
%player = %client.player; // %client.getControlObject()
%eyepoint = %player.getEyePoint();
%eyeVector = %player.getEyeVector(); // direction we're looking
%range = 5; // Range to search
%mask = $Typemasks::StaticShapeObjectType;
%raycast = containerRayCast
(%eyepoint, VectorAdd(%eyepoint, VectorScale(%eyeVector,%range)), %mask, %player, false);
%object = getWord(%raycast,0); //Object it hit
%positionHit = getWords(%raycast,1,3); //Position the raycast hit
%normalDirection = getWords(%raycast,4,7); //The direction of the normal that it hit
echo ("Object: " @ %object);
echo ("Position: " @ %positionHit);
echo ("NormalDirection: " @ %normalDirection);
}Right now it's just testing for StaticShapeObjects to be sure it's not missing due to the new TypeMask(probably not, I suppose that's the entire castRay() stuff just not working with projectiles etc. right now). Anyways, I thought I'd share that since it might be handy for testing. It does work, just not on our new object just yet :D The SimGroups
Now, if I do go ahead and add the object to a simgroup with:
Matrix.add(%obj);...I start getting errors that the object is not out of the bin when trying to close the program. So instead of indexing it this way, I am now trying:
%obj.index = World.Submatrix[%x, %y, %z];No problems so far, I think it's fine indexing it this way for now. Unless someone explains a reason I shouldn't. It has gotten rid of the bin errors closing the program so I'm happy lol.
I assume atm, for the most part the problem is out of my hands due to the complexity of the castRay function? If there's anything else I should try, I'm all for it. Thanks guys! I realize this is definitely a little hairier than expected, especially due to the castRay stuff.
#156
The castRay problem is weird and we're looking into it. castRay isn't a complex function to implement (once you've got your head around it) and it works fine for plenty of stock objects, so the discrepancies we're seeing are probably some weird Torque thing. We'll let you know if we find anything :(.
09/24/2014 (7:20 pm)
The bin errors are because your object is not being deleted when the mission ends. Is the World group part of the MissionGroup? If so, you might want to put your Matrix group into the World group. That way when MissionGroup is deleted, World is deleted, and Matrix is deleted, and all the Submatrixes are deleted.The castRay problem is weird and we're looking into it. castRay isn't a complex function to implement (once you've got your head around it) and it works fine for plenty of stock objects, so the discrepancies we're seeing are probably some weird Torque thing. We'll let you know if we find anything :(.
#157
Sounds good about the castRay stuff. I figured it was something a little beyond my current coding level. In the meantime, I'm going to put some more time in with my study guide! I'm also going to see if I can start creating some actual level chunks and stuff until the castRay gets sorted. I've got faith, like you've all told me:
09/24/2014 (7:33 pm)
Yep, you're absolutely right Danny. I was just blindly creating the SimGroup paying no heed to MissionCleanup. It's from when I was first trying to create the cubes, I started with a scriptObject and built off that. I never remembered to go back and be sure it'd be cleaned up. That fixed it, I just needed to add my World group to the MissionGroup. Thanks!Sounds good about the castRay stuff. I figured it was something a little beyond my current coding level. In the meantime, I'm going to put some more time in with my study guide! I'm also going to see if I can start creating some actual level chunks and stuff until the castRay gets sorted. I've got faith, like you've all told me:
Quote:Non-trivial results require non-trivial effort.Thanks for all the help. I bet it comes to you when you're trying to get some sleep(that's what always happens to me)!
#158

I wanted to share the fruits of all the labor in the form of a screenshot. This is a first pass on generating terrain! In the screenshot there are 9000 cubes with a decent framerate! It's actually 9 Submatrix objects there, each 10x10x10. I also did a rough draft (in C++!!!) of a cubeList generation so that the cubes have different layers, and also know where the top is(grass)! Much more work to do around generating the stuff, but I figured I'd share since I was pretty excited :D
This is all with no sort of culling taking place at all just yet (mainly because I'm not entirely sure how to approach that). Also there will be a need to optimize the convexes as well, since the performance takes a nosedive as soon as I begin to walk across all the cubes. I guess the collision against so many cubes in such a rapid succession really is a killer on performance lol. Anyways, I thought it may be an inspiration to you guys trying to find a fix for the castRay stuff :)
09/25/2014 (5:11 am)

I wanted to share the fruits of all the labor in the form of a screenshot. This is a first pass on generating terrain! In the screenshot there are 9000 cubes with a decent framerate! It's actually 9 Submatrix objects there, each 10x10x10. I also did a rough draft (in C++!!!) of a cubeList generation so that the cubes have different layers, and also know where the top is(grass)! Much more work to do around generating the stuff, but I figured I'd share since I was pretty excited :D
This is all with no sort of culling taking place at all just yet (mainly because I'm not entirely sure how to approach that). Also there will be a need to optimize the convexes as well, since the performance takes a nosedive as soon as I begin to walk across all the cubes. I guess the collision against so many cubes in such a rapid succession really is a killer on performance lol. Anyways, I thought it may be an inspiration to you guys trying to find a fix for the castRay stuff :)
#159
And, oh - dude, at some point we'll need to implement networking. At the moment, you're still calling addCube from C++, right? I'm sure you'll want to be able to add and remove cubes dynamically in scripts, and for that you'll need to make changes to (un)packUpdate. We'll go over that soon, too :).
09/25/2014 (5:43 am)
Right, so this is where the fun begins. I'm sure the convex collision stuff really does need optimising, though that's a Torque-specific problem as it really isn't set up to deal with this situation. You'll also find that Minecraft probably spend a lot of effort optimising the load of having millions of potential word voxels. I'd bet that any voxel terrain worth its salt is actually only one or two blocks thick. If you excavate, then more blocks are generated and placed, but they're not all active at once. You should really start thinking about doing something like that, it'll save you a lot of headache without having to look into culling algorithms and so on, or doing anything fancy to optimise collision.And, oh - dude, at some point we'll need to implement networking. At the moment, you're still calling addCube from C++, right? I'm sure you'll want to be able to add and remove cubes dynamically in scripts, and for that you'll need to make changes to (un)packUpdate. We'll go over that soon, too :).
#160
Yep just doing all the generation stuff in C++. You're right, the pack/unpackUpdate stuff is still a mystery. No worries or any rush at all. This thing's at a stage where there are so many ways to work on it there's no reason to be in a hurry :) I'll continue to try for better, more optimized generation and added functions to help test it. This is a huge success already with all of the help you guys have given me! There is actually something to build off of here already, which is more than I expected in such a short time. Especially as I'm just opening the door into C++.
EDIT: Also I forgot to mention that I think the most important part is that I'm actually having fun with this stuff again. Sure, at times it all can be a little overbearing but that comes with the territory when developing. The thrill of challenging myself mentally and working hard for solid results is what makes those small successes all the more rewarding. I've changed my entire outlook on this stuff, mainly because I've been given a sense of direction by some great people.
09/25/2014 (6:05 am)
Thanks for the advice Danny. Yep this sounds pretty much spot on. I have some vague ideas around trying to keep things within a 1 or 2 cube thickness, but nothing that's entirely clear yet :/ Sounds like the perfect direction to take everything though for sure. Should improve performance, reduce draw calls, the whole 9 yards. I know that Torque will be capable once I get the layering right! Yep just doing all the generation stuff in C++. You're right, the pack/unpackUpdate stuff is still a mystery. No worries or any rush at all. This thing's at a stage where there are so many ways to work on it there's no reason to be in a hurry :) I'll continue to try for better, more optimized generation and added functions to help test it. This is a huge success already with all of the help you guys have given me! There is actually something to build off of here already, which is more than I expected in such a short time. Especially as I'm just opening the door into C++.
EDIT: Also I forgot to mention that I think the most important part is that I'm actually having fun with this stuff again. Sure, at times it all can be a little overbearing but that comes with the territory when developing. The thrill of challenging myself mentally and working hard for solid results is what makes those small successes all the more rewarding. I've changed my entire outlook on this stuff, mainly because I've been given a sense of direction by some great people.
Andrew Mac
This is where your issue is coming from. I meant to actually modify those source files in place and then compile them. I just tried the modifications in place and it compiled fine. The reason we made a copy of renderMeshExample.cpp/.h to begin with was because you were making a whole new engine object, so it's easier to work off a copy of that existing one. You don't need to copy the files to source with every change you want to make.
I have heard before that you can copy engine files into your source folder and the engine will compile those in place of the original copies. I haven't tried this myself, but I think at the very least you'd have to make sure the paths remain the same. If it's source/T3D/gameFunctions.cpp I think you'd need to make a T3D folder in your source folder and put gameFunctions.cpp in there in order for the engine to compile that one in place of the original gameFunctions.cpp. As I've said I've never used this method so I'm not too sure. I always just make my changes to the original files and mark them with a comment with my name so I can find all my changes when upgrading Torque.
I'm not sure if there's a system in place to cleanly add new type masks for objects without modifying the original source. Maybe someone else can chime in on that one.