Ghosting Limits

#161 It may be another day or two before I find the time to do a proper debug and find out why ray cast isn't working for you or I Jesse. The weirdest part? We got a 4th party to try it last night and it worked for them. It works for half of us, but not the other half. Truly the worst kind of bug.

I've been trying to keep it as simple as possible during the tutorial, but now that the dust has settled and we're just talking theory, something to think about long term: We chose to treat every box as a separate piece and piece it all together, but what would happen if there was code in there to detect nearby boxes and exclude unseen faces? So a string of 4 cubes in a row would not have the unused faces in between the blocks. If you begin to optimize out unseen faces, what happens with the block of 9000 cubes? That massive block of 9000 cubes would suddenly be made up only the visible surface cubes, and there would be no concern of what's inside it because those have been excluded. Collision fast, draw fast.

I've never tried anything like that before so I don't even know where you'd start. I guess when generating the faces of the cube you'd have to check if there was a cube next to it or not. If there is, don't add the face to the vertex buffer.

#162 No worries Andrew, you've helped so much! Take your time man :)

On the subject of separate faces, it's funny you mention that because I was literally just now exploring options around this. I was kinda messing around with createGeometry() so that instead of just declaring all the points and just going through and drawing a whole cube...I'm trying to draw out each face as a separate piece...and only draw it if certain flags are true/false. It's all in theory atm, but by the time you guys break some ground on the raycast stuff I hope to have some tangible results. I'll post up some pseudocode (or maybe even an example of actual code) once I make a little bit more progress. I think this is definitely the way to go (separating face geometry).

I swear this forum has ESP...lol

#163 Okay, here's kinda what I'm trying to say but I'm just not sure how to actually draw it all out. We just go ahead and make a function to draw a cube, but we declare the variables up front and just update them as we draw each face of the cube. I get lost in the createGeometry() function kinda though, so I'm not sure how to lock/unlock the buffers to actually use all this stuff. I think the biggest disconnect for me is knowing when/how to lock/unlock buffers. Math might not all be correct either, since it's hard to test it when I can't actually draw it lol. Anyways, check it out, I hope this is a step in the right direction:

void Submatrix::drawCube(int x, int y, int z, int size)
{
	// Vertex points
	Point3F ptBL, ptTL, ptBR, ptTR;

	// Normals
	Point3F normal;

	// Texture Coordinates
        // This seemed to be the order the tex coords were
        // being drawn before - BL, TL, BR, TR
	Point2F texBL(     0,  size/2);
	Point2F texTL(     0,       0);
	Point2F texBR( size/2, size/2);
	Point2F texTR( size/2,      0);

	// Front of cube
	normal = Point3F( 0, size, 0);
	ptBL = Point3F(x,        y,        z);
	ptTL = Point3F(x,        y,        z + size);
	ptBR = Point3F(x + size, y,        z);
	ptTR = Point3F(x + size, y,        z);

	addFace(normal, ptBL, texBL, ptTL, texTL, ptBR, texBR, ptTR, texTR);

	// Back of cube
	normal = Point3F( 0, -size, 0);
	ptBL = Point3F( x,        y - size, z);
	ptTL = Point3F( x,        y - size, z + size);
	ptBR = Point3F( x + size, y - size, z);
	ptTR = Point3F( x + size, y - size, z + size);

	addFace(normal, ptBL, texBL, ptTL, texTL, ptBR, texBR, ptTR, texTR);

...
// draw out each face
}

We'd need another function addFace(), as you can see, to actually draw out each face. Also what I was trying to envision was an if statement prior to drawing it to check if it was actually visible or not (i.e. facing the player). I had learned about this type of stuff a while back from some dude's blog but for the life of me I cannot remember who/where I studied that stuff from. I've literally been interested in seeing this come to fruition for like 6 months or more. I was lucky I still had the notes on paper. If nothing else, maybe it can help the 'real' coders to figure out what we need to do. I mean, don't get me wrong, I feel like I can solve the problem with variables and functions but just not too familiar with the buffer usage. Cheers, I'll keep trying and see if I can make more sense out of it.

EDIT: Okay, I think I have the addFace() function setup so all the variables are being passed. I mean, I can get it all to compile but the addFace() function isn't completed just yet. I'm going to attempt to use the buffers, but it's a little different than our previous example because I don't really have the points indexed here. Just calling out to draw each face one at a time. I feel like if I could just pass the data properly it'd work. If nothing else, though, definitely giving me a good exercise and getting me more used to tracing code and trying to re-apply it :)

EDIT2: Bleh, just not versed enough with the syntax to make it all tie together unfortunately. Without hooking directly into the indexes in createGeometry() I'm just having trouble using the buffer. Basically I'm limited to using the createGeometry() function or building within that singular function. When what I'd like to do is create other functions that are dynamically rendering during runtime. Sigh, sucks I had most of it figured out in script. Damned ghosts, lol. Hopefully I can come up with some other approach...

#164 I have a question about setting up objects for a level. It's something that continues to come up, and I've never found a completely clear answer. Say I've got a SimGroup in my level's .MIS file, and I want to add a bunch of objects to it, like so:

//--- OBJECT WRITE BEGIN ---
new SimGroup(MissionGroup) {
   canSave = "1";
   canSaveDynamicFields = "1";
      enabled = "1";
...
   new SimGroup(World) {
      canSave = "1";
      canSaveDynamicFields = "1";

      new SimGroup(Matrix) {
         canSave = "1";
         canSaveDynamicFields = "1";

         new Submatrix() {

           //variables
         };
         new Submatrix() {

           //variables
         };
         new Submatrix() {

           //variables
         };
         	 		 
      };
   };
};

This works fine if you have a small amount of objects. But what if you wanted tons of the objects to be added, by use of a for loop? As far as I can tell, you can't do that in the .MIS file itself.

Now here's why I'm inquiring. I've created the objects in a separate script, no problem. But, if you do it in this way the objects are being created by use of a script and not 'solidified' as part of the level. There is a drastic difference. As a matter of fact, I've finally discovered that this is the root of the TSStatic ghosting problem. Create a static as above, you won't have a single ghost. Try to create them in a script, they all ghost...So, how can you create objects with a for loop to be part of a sim group and still be part of the mission group? This way when the mission loads, those objects don't ghost and they load in as part of the level.

#165 Nope, you can't use loops inside object definitions - the syntax doesn't allow it. The thing is, a .mis file is just TorqueScript, so you could add a for loop outside the MissionGroup definition - after the OBJECT WRITE END comment. It's just that Torque's editors don't know how to read that. If you want to create things programatically, you need to do it in some eternal script.

Quote:the objects are being created by use of a script and not 'solidified' as part of the level.

As long as an object is added to MissionGroup, or is in a group that's in MissionGroup (and so on), then it's the same as any other object that was created in the .mis file.

Quote:This way when the mission loads, those objects don't ghost and they load in as part of the level.

Not sure what you mean here. If you can see them, they've ghosted. To 'load' a level, Torque just executes the .mis file on the server as if it were any other script, which creates a bunch of objects, which then get ghosted to the client so the client can see the level.

#166 Hi Danny, thanks for the help.

Quote:Not sure what you mean here. If you can see them, they've ghosted.

This isn't true for TSStatics. That's what I've been testing, with 2 different results. If you make an object in the .mis file(TSStatic or created geometry), between OBJECT WRITE BEGIN and OBJECT WRITE END that object won't ghost when you load up the level. Try it.

However, if you try to programatically add the object to the SimGroup later on it will ghost.

Quote:As long as an object is added to MissionGroup, or is in a group that's in MissionGroup (and so on), then it's the same as any other object that was created in the .mis file.

See, here's where the problem crops up. I'm not sure when/how to add the object to the group. If I run a script during runtime, the object will ghost. If the object is 'hardcoded' into the .mis file, it won't. I'm trying to figure out where (in what file, in what place) I can 'inject' the function to generate the stuff. Prior to the level actually loading and starting the game. Once you're in the level, anything you create will ghost.

With extensive testing though, I've found that so long as the object is already in place before the mission starts up it won't ghost. This is only true for TSStatics. Oh wait, it works for the Submatrix object as well that Andrew has helped me create too. Like I can create a new Submatrix() in the .mis file and it won't ghost at all. If I do it in the console with the mission open it will ghost.

#167 It looks like you're already placing them in the MissionGroup - and it's still not deleting them properly?

#168 @Richard: It's just a very specific problem I suppose man. It just boils down to solving a problem(or a series of problems lol) to render a world of cubes ultimately.

Torque isn't setup to handle any of this by default, so what Andrew's been doing is stepping me through the process of drawing out the geometry in C++. With the created geometry object, I'm able to create a singular object that can hold many, many cubes.

Taking that one step farther, we want to drop in multiples of this newly created 'submatrix' object into our level. To do so, we have 2 options:

1- Create a SimGroup in the .mis file, and add in all the objects by hand there (very tedious).

or

2- Write out a script with a for loop to generate several of the submatrix objects.

Here's the stuff I've discovered:
If I 'hardcode' the objects into a SimGroup in the .mis file, none of them will ghost.

If I run a script to add the objects, every one of them will ghost.

The problem is, even one submatrix object with 1000 cubes inside of it doesn't get you very far in terms of terrain. You can take like 10 steps and you've reached the end. So I'm looking at potentially 100's (if not 1000's) of submatrix objects - a bit much to be plopping into the .mis file by hand. I'm just trying to figure out where I can run my 'generate' function so that those submatrix objects are generated prior to the level starting up.

#169 Jesse, I think something's going very wrong. With the regular engine, when I add a TSStatic to the level and load it up again, it all works fine. My .mis file contains this:

new TSStatic() {
   shapeName = "art/shapes/actors/Soldier/soldier_rigged.DAE";
   ...
};

And I see this:



Maybe this is something to do with you removing ScopeAlways from TSStatic. Since those objects don't set any mask bits, maybe they don't get ghosted in the normal fashion. But surely they'd all be ghosted initially... it's a mystery.

#170 No, this isn't the problem. If I add a TSStatic in the editor, in the .mis file, or via script they will appear just fine in any case.

What I'm referring to is the actual active ghosts. As monitored by the netgraph.

Try this:
1- Add a TSStatic to your mission, in the editor or in the .mis file.
2- Shut down the level and restart it.
3- When you load the level, you should see your TSStatic.
4- Press N to bring up the netgraph. You shouldn't get any extra ghosts from any TSStatic being reported here as an 'Active Ghost'.

On the other hand:
1- If you don't add the TSStatic in the editor or in the .mis file you can create a script.
2- In that script you can add them to the simGroup. When you load up the level, they will all appear (just like above).
3- The difference is, now all of those TSStatics are reported as Active Ghosts (in the netgraph - hotkey N)

Remember we're monitoring our Active Ghosts with the netgraph (hotkey N). I'm not having any trouble adding the objects at all. I'm just noticing that there is a very distinct difference between ghosting those objects depending on how you add it. Also this is with a default T3D 3.5.1 dev branch build, no changes to the source (besides the added geometry object Andrew has been helping me with).

#171 Ooh, that's very interesting. I think you'll find, though, that even 'inactive' ghosts still have a ghost ID and contribute to the maximum ghost count. I'm not totally sure of that, but that must be the case as far as I understand Torque's networking. I'm not sure how 'active ghosts' is qualified, but I suspect it's not 'ghosts transmitted during the scope always transmission'. There's actually a phase of level loading where ScopeAlways objects are transmitted, before 'regular' ghosting begins. I don't well understand the difference between the two, but I imagine this is it.

So, the reason you're seeing an 'active' ghost when you add a static during gameplay is because it is added to the world after the 'transmit scope always' phase of mission loading.

But I'm not yet sure what the implications of that are.

#172 Thank goodness you understand lol. I do see how it's kinda strange. If I hadn't been test running an old level with the netgraph up I never would have noticed it myself. It's actually kinda good that it behaves in this way (in most cases).

See, in my old level I had a lot of TSStatic pieces pieced together to make an interior. But I also had some other stuff in the level like ShapeBase objects and vehicles. So when I popped the netgraph up I noticed something really cool! Basically I was only getting 'active ghosts' for the ShapeBase objects and vehicles. All of the TSStatics that were building the actual level interior weren't being counted. This is because they all were added in the editor long ago and were part of the .mis file initially.

Quote:So, the reason you're seeing an 'active' ghost when you add a static during gameplay is because it is added to the world after the 'transmit scope always' phase of mission loading.

Exactly. What I'd like to attempt to do here, Danny, is try to run my generation script so that I generate the geometry prior to the 'transmit scope always' phase. Do you have any ideas on where (in what file) I could call my function in order to do so?

#173 This is where the mission file is executed from. But actually, what I recommend doing is this. Add some random object with a datablock to your mission. Anything as long it has a datablock. Then write an onAdd method for its datablock that spawns your terrain.

You could write something in onMissionLoaded instead, I guess, but it would run every time you started any mission. If you use a datablock callback, then only place objects with a specific datablock in one specific mission, then you can write level-specific terrain generation.

For example,

// in level1.mis
new StaticShape() {
   datablock = Level1TerrainData;
};

// in level2.mis
new StaticShape() {
   datablock = Level2TerrainData;
};

// elsewhere
function Level1TerrainData::onAdd(%this, %obj) {
   // spawn matrix for level 1, add to MissionGroup
}

function Level2TerrainData::onAdd(%this, %obj) {
   // spawn matrix for level 2, add to MissionGroup
}

These onAdd functions should run immediately after those objects are created - at the same moment the rest of the level objects are created on the server.

Note that all this setup is just part of the way the default script templates are setup with their mission-oriented structure which is basically straight out of Tribes.

#174 Appreciate it Danny, you've been helpful :)

#175 I finally calculated the UV's just right! I sure have learned to appreciate the small victories :)


Half pixel problem, be gone!

#176 After testing extensively, I'm not being able to use AddCube at all in the game. I've added the DefineEngineMethod for addCube() and for createGeometry() without any problems. Compiles okay, and I'm even able to use the functions in script without any console errors when the functions run. I just simply am getting no results. I think the reason is just the way that we have addCube() as a function of the main submatrix object, like so: Submatrix.addCube().

What this is telling the game to do is add a cube to the Submatrix, so then I call Submatrix.createGeometry() to try to initiate a refresh of the mesh. Nothing works to update the Submatrix object during runtime. It won't even work if I make scripts to add cubes prior to runtime (before the mission loads the submatrix object). No compile errors, no console errors, it's just the design of the addCube() won't work outside of C++. Basically, you have to call addCube() on the object in C++ prior to building it and that's it. Once the Submatrix object is created in the game world, I'm not being able to add any cubes at all. I add them in C++ no problem in the Submatrix::Submatrix part. I'm even able to iterate the SimGroup holding all the Submatrix objects in the game.

I feel I've painted myself into a corner, can anyone shed a little light on what may be going wrong? Pretty much not having any problems generating the world with Submatrix objects (which can even change dynamically in source if that's the intent). Just once in game, no cubes can be added/altered using the Submatrix.addCube() function...despite the function being called in script with no reported errors.

#177 That's to be expected. We haven't gone over it because you weren't at that stage yet, but now is the time to add some networking to your object. I'll write up a tutorial on that tonight.

Basically, when you call addCube, you're doing it on the server. So your server-side object has cubes in it, but the copies ghosted to the client have no idea. What I'll go through with you (unless someone beats me to it) is adding the code that will communicate your changes across the network to make sure all clients are in sync.

EDIT: This is actually gonna be a little more complicated than your usual class. But that's ok, we'll get through it and learn lots together. Also, nice work with the pixel alignment fix ;).

#178 Oh thank goodness. Happy it wasn't something I was messing up. Looking forward to it, thanks Danny :) In the meantime, you guys have given me tons of material to study. Not to mention there are so many things I can do relating to the actual terrain generation. Plenty of stuff I can do to stay busy for sure.

Oh, on a side note I wanted to let you know that I was doing some tests with the Submatrix object generating terrain with 1 cube thickness. It actually performs really well as it is like this, so yes you were right on point about that. Sure, there will be room for optimization down the line but honestly it's going to be a good starting place for now. I feel like the further optimization will be a long road of much study, trying to get cubes to detect neighboring cubes and so on.

Also, I found a little code in the collision/collision.h file that may or may not be relevant to the castRay problem. Particularly the comments:

/// Set the point of intersection according to t and the given ray.
   ///
   /// Several pieces of code will not use ray information but rather rely
   /// on contact points directly, so it is a good thing to always set
   /// this in castRay functions.
   void setContactPoint( const Point3F& start, const Point3F& end )
   {
      Point3F startToEnd = end - start;
      startToEnd *= t;
      point = startToEnd + start;
   }

I wasn't sure if this was an issue or not with what we're doing. I was tracing some of the includes and digging deeper to learn more about the source and stumbled on this. I noticed we don't use setContactPoint() at all, and the comments seem to suggest it's good to include in castRay functions. Food for thought, at least :)

#179 I suspect that comment is outdated. You can see that the the position of the RayInfo is actually set explicitly by SceneContainer::_castRay, so there's no need to set it in objects' castRay functions.

#180 Alright, sounds good. Figured I'd bring it up just in case. Thanks for the info.