Ghosting Limits
by Jesse Allen · in Torque 3D Professional · 09/08/2014 (12:47 am) · 212 replies
Greetings fellow Torque-goers! I hadn't been overly active on the boards lately, but that's usually a good thing since it means I'm busy actually developing stuff :) Finally hit a snag today, and I was hoping someone a bit more knowledgeable about these things could perhaps point me in the right direction.
I've been working with generating 1000's of cubes, which I have been largely successful with. I have scripted algorithms to do exactly what I need as far as generation is concerned. I've even managed to work out some basic culling and so on. The problem I'm having, however, is with the following error:
NetConnection::object In Scope: too many ghosts.
I did a bit of searching and I did stumble on Vince Gee's fantastic resources for Limiting Shapebase ghosting and Improved Limiting ghosting . These do appear to be useful by their own rights and I will definitely explore these options. However, these particular resources deal specifically with ghost limiting as related to view distance.
The problem I am having seems to stem from the actual hard limits set on the maximum number of allowed ghosts. I was curious:
Also, bonus points for a clear explanation on how view distance affects ghosts in the first place. If I generate in a bunch of cubes and use the command ServerConnection.getGhostsActive() I can see the current number of ghosts. Check. But if I then reduce the viewDistance to an absurdly low amount (such as 10) and run far away from the generated cubes...the command will still show the same number of ghosts, regardless of my distance from them.
Cheers guys, as always thanks in advance for any help.
I've been working with generating 1000's of cubes, which I have been largely successful with. I have scripted algorithms to do exactly what I need as far as generation is concerned. I've even managed to work out some basic culling and so on. The problem I'm having, however, is with the following error:
NetConnection::object In Scope: too many ghosts.
I did a bit of searching and I did stumble on Vince Gee's fantastic resources for Limiting Shapebase ghosting and Improved Limiting ghosting . These do appear to be useful by their own rights and I will definitely explore these options. However, these particular resources deal specifically with ghost limiting as related to view distance.
The problem I am having seems to stem from the actual hard limits set on the maximum number of allowed ghosts. I was curious:
- Is it possible to increase the number of ghosted objects?
- If it were possible, what sorts of problems would this increase potentially introduce?
Also, bonus points for a clear explanation on how view distance affects ghosts in the first place. If I generate in a bunch of cubes and use the command ServerConnection.getGhostsActive() I can see the current number of ghosts. Check. But if I then reduce the viewDistance to an absurdly low amount (such as 10) and run far away from the generated cubes...the command will still show the same number of ghosts, regardless of my distance from them.
Cheers guys, as always thanks in advance for any help.
About the author
Skilled Artist and Musician. Intermediate Torque Developer.
#62
So see? There is a bright side!
09/18/2014 (8:11 am)
The deck's not stacked against you - the right hand wasn't dealt, but there are an awful lot of cards already there. At least you didn't have to write a 3D game engine from scratch....So see? There is a bright side!
#63
Gotta remmber, tsstatics are likely to be the single most numerous asset in a given scene, and half the point of ghosts is to prevent re-transmission of data.
09/18/2014 (9:28 am)
Thinking out loud... a minimalist approach may be to add a boolean flag sent along the line that's flipped off similar to how a masterserver and gameserver communicate heartbeats. If the objects aren't in view after say... 10 seconds or so, then remove the ghost. (Would allow for cache preservation while still perhaps adressing the problem without sending a continuous 96 bits per-object-insatnce like was suggested with the position update bit.). 'course, still need to track down where tsstatics are determining they're in view...Gotta remmber, tsstatics are likely to be the single most numerous asset in a given scene, and half the point of ghosts is to prevent re-transmission of data.
#64
Perhaps adding something like Daniel's script engine-side - so configurable update frequency, along with an "only do this while ghosted" thing perhaps.
09/18/2014 (9:42 am)
Knowing Daniel he's hit it on the head - TSStatics don't send status updates like Player, so they don't get dropped from the ghost list when we might like them to.Perhaps adding something like Daniel's script engine-side - so configurable update frequency, along with an "only do this while ghosted" thing perhaps.
#65
Jesse: Xenocell was looking fantastic, and I was also disappointed that it didn't work out. Scope is the hardest thing to manage in any project. I've given up on several big awesome game ideas in favour of working on engine tech and smaller 'game jam' ideas - anything else just means years of effort, frustration, and eventual disappointment (at least for me - that's not necessarily true for everyone. See Minecraft and Dwarf Fortress). YOu've got to know yourself and what's feasible first, but also know ways of motivating yourself to work on stuff, push through, and do the dirty work to end up with a product.
I've found reading /r/gamedev to be really helpful, both for the technical content and the motivational posts and musings about the game development process.
09/18/2014 (6:09 pm)
I would definitely not vote for adding my hack to the engine :P.Jesse: Xenocell was looking fantastic, and I was also disappointed that it didn't work out. Scope is the hardest thing to manage in any project. I've given up on several big awesome game ideas in favour of working on engine tech and smaller 'game jam' ideas - anything else just means years of effort, frustration, and eventual disappointment (at least for me - that's not necessarily true for everyone. See Minecraft and Dwarf Fortress). YOu've got to know yourself and what's feasible first, but also know ways of motivating yourself to work on stuff, push through, and do the dirty work to end up with a product.
I've found reading /r/gamedev to be really helpful, both for the technical content and the motivational posts and musings about the game development process.
#66
I did follow the script pretty easily. It's just iterating through the objects in the SimGroup and updating the position so that any changes to the scope actually occur. But yea it's definitely a hack and way too inefficient to be used in any sort of multiplayer environment. Thanks for the effort and idea though. Honestly, unless these objects are being dropped from scope based on view distance in source...my idea will never see the light of day in Torque. It's unfortunate, since I've come so far with the engine at this point. But now I finally begin to see the the other side of the fence, and realize why it is that so many of the vets don't even use Torque anymore. It's not all bad, it's healthy for my personal development really. Now I'm forced to learn more advanced programming languages to either:
A)Change the source to force the scope update on statics (not likely).
or
B)Move on to another engine which will require me to use C# or C++ instead of the TorqueScript.
Until now, really I've been able to accomplish anything I've wanted with TorqueScript. Failing this, I really don't have a reason to continue holding myself back as a developer with an old scripting language. I thank you Danny for the inspiration, efforts, and time you've contributed to the engine. I still and always will love Torque, but I do feel it's time for me to branch out a bit.
09/19/2014 (2:30 am)
Alright, got a chance to test out the script. No dice. I had to rearrange the way I was generating the cubes a bit to get rid of some 'index out of range' errors but that wasn't too rough. I just setup a new SimGroup and added each %obj with a %obj.parentGroup = command. No big deal. So anyways, upon running the script all it really managed to do was clog up my reported sent/received data without delivering the wanted results: removing the objects from scope. No active ghosts were lost traveling out of range of the cubes. This could still have been something small I overlooked but honestly seeing how badly the script bogged down bandwith it's not worth pursuing.I did follow the script pretty easily. It's just iterating through the objects in the SimGroup and updating the position so that any changes to the scope actually occur. But yea it's definitely a hack and way too inefficient to be used in any sort of multiplayer environment. Thanks for the effort and idea though. Honestly, unless these objects are being dropped from scope based on view distance in source...my idea will never see the light of day in Torque. It's unfortunate, since I've come so far with the engine at this point. But now I finally begin to see the the other side of the fence, and realize why it is that so many of the vets don't even use Torque anymore. It's not all bad, it's healthy for my personal development really. Now I'm forced to learn more advanced programming languages to either:
A)Change the source to force the scope update on statics (not likely).
or
B)Move on to another engine which will require me to use C# or C++ instead of the TorqueScript.
Until now, really I've been able to accomplish anything I've wanted with TorqueScript. Failing this, I really don't have a reason to continue holding myself back as a developer with an old scripting language. I thank you Danny for the inspiration, efforts, and time you've contributed to the engine. I still and always will love Torque, but I do feel it's time for me to branch out a bit.
#67
Doing per-player scope calculations on thousands of cubes is INSANE. I've told you from the beginning how to solve this problem but you've continuously chosen what you perceive to be the easier path and now you've concluded that torque is the problem.
There's another user from IRC who is doing the same thing as you. Making cube worlds in torque. He hit the same problems, and solved them through the way I suggested like 50 posts ago. It's working out great for him, and it's not that hard. We would all gladly help you to learn these methods, but you don't seem willing to learn. You have all the tools in front of you to solve your problem but you're just ignoring them and expecting Torque to do everything for you out of the box.
09/19/2014 (5:23 am)
Unless you switch to an engine specifically designed from the ground up for minecraft style cube worlds, you're going to hit all the same problems. Whats the net ID limit in unreal? 65k. Sound familiar? It's a common limit because you shouldn't have to exceed it. These limits are in place to make you raise the question of "am I doing something I shouldn't?" which is exactly what you're doing.Doing per-player scope calculations on thousands of cubes is INSANE. I've told you from the beginning how to solve this problem but you've continuously chosen what you perceive to be the easier path and now you've concluded that torque is the problem.
There's another user from IRC who is doing the same thing as you. Making cube worlds in torque. He hit the same problems, and solved them through the way I suggested like 50 posts ago. It's working out great for him, and it's not that hard. We would all gladly help you to learn these methods, but you don't seem willing to learn. You have all the tools in front of you to solve your problem but you're just ignoring them and expecting Torque to do everything for you out of the box.
#68
The solution you suggested wasn't 'not that hard'. Drawing out geometry manually per vertex in C++ and balancing this by how many cubes will render without a performance hitch. I've already admitted I'm not very fluent with C++ and I wasn't given anything I could actually use. I was linked examples of source files that I can't do anything with. A good sentiment, but ultimately it doesn't help me to learn. I can see a cube being rendered in one example, although how that is happening isn't completely clear.
Thanks for rubbing it in that there is someone accomplishing my goals who is 'more connected' to the guys in the forum. Goes a long way to help me see what I'm not understanding.
You're suggesting that I'm not giving any effort when the effort required isn't trivial. Let's also completely ignore the fact that I have already written out all the algorithms in script to generate and cull out a cube terrain. No effort at all.
We're talking about drawing verts by hand in C++ - again something that someone using an engine shouldn't have to be doing. I'm sorry Andrew, but your argument isn't very strong. It only serves to strengthen my point. If you offered up some sort of advice besides taking stabs at me maybe but come on man you've started to insult me now.
09/19/2014 (8:17 am)
This is the reason people are turned away from Torque. Someone has a problem and the community reacts with answers that discourage the user from moving forward with the engine. The solution you suggested wasn't 'not that hard'. Drawing out geometry manually per vertex in C++ and balancing this by how many cubes will render without a performance hitch. I've already admitted I'm not very fluent with C++ and I wasn't given anything I could actually use. I was linked examples of source files that I can't do anything with. A good sentiment, but ultimately it doesn't help me to learn. I can see a cube being rendered in one example, although how that is happening isn't completely clear.
Quote:Doing per-player scope calculations on thousands of cubes is INSANE. I've told you from the beginning how to solve this problem but you've continuously chosen what you perceive to be the easier path and now you've concluded that torque is the problem.Well it's been proven already that Torque doesn't drop statics that are out of scope. Yet it somehow manages to drop everything else that is out of view distance. If it weren't for this, none of those calculations would need to take place in the first place. So what if I did draw out all the geometry in C++? Why does that magically exclude that geometry from the ghosting problem? Because there is some broken scoping rule in the networking code that won't allow you to do it otherwise. Oh, and also if the objects weren't being ghosted across the entire scene there wouldn't be 1000s of cubes considered anyway. Just the ones the player sees at the time. But yea, nothing at all wrong with Torque's scoping..
Quote:There's another user from IRC who is doing the same thing as you. Making cube worlds in torque.Screenshots or it didn't happen. Really, if this is such a familiar case that you know someone else being successful with it why hasn't it come up before now?
Thanks for rubbing it in that there is someone accomplishing my goals who is 'more connected' to the guys in the forum. Goes a long way to help me see what I'm not understanding.
Quote:We would all gladly help you to learn these methods, but you don't seem willing to learn.That's why we have a near 5 page thread that I keep returning to asking for advice.
Quote:You have all the tools in front of you to solve your problem but you're just ignoring them and expecting Torque to do everything for you out of the box.Totally. That's why I've already written a TON of script around the problem with no results. Look, I'm not fluent with C++. So that makes me a 3rd class citizen around here apparently.
You're suggesting that I'm not giving any effort when the effort required isn't trivial. Let's also completely ignore the fact that I have already written out all the algorithms in script to generate and cull out a cube terrain. No effort at all.
We're talking about drawing verts by hand in C++ - again something that someone using an engine shouldn't have to be doing. I'm sorry Andrew, but your argument isn't very strong. It only serves to strengthen my point. If you offered up some sort of advice besides taking stabs at me maybe but come on man you've started to insult me now.
#69
TSStatics aren't really intended for this use, but things that aren't flagged ScopeAlways shouldn't be scoped always, you would think. Whether or not it applies to this particular use case hasn't been a question for me - it's more about why something is being ghosted when I don't want it to be.
09/19/2014 (8:20 am)
Oh, come on guys - I said "something like" not "hammer that hack in there." lolTSStatics aren't really intended for this use, but things that aren't flagged ScopeAlways shouldn't be scoped always, you would think. Whether or not it applies to this particular use case hasn't been a question for me - it's more about why something is being ghosted when I don't want it to be.
#70
And we're all willing to help you to understand it, but you have to show a willingness to learn first.
Because if you draw the geometry from within a single networked object you're making 1 netobject (1 TSStatic), which is 1 ghost, that produces a bunch of cubes for only 1 representation in the networking world. So, you'll have 1 static for, lets say, 10k cubes. Now you can have a world with thousands and thousands and thousands of cubes and torques networking doesn't care. When you add or remove a cube its an infrequent event compared to the updates that players will have moving around, etc. Now that single object is responsible for the networking of all of its 10k cube children. When a cube is changed, that object is flagged dirty, and sends an update for that one change.
I can't stress enough that we can all help you to accomplish this. I'm not trying to insult you, I'm trying to help you. You just keep dismissing my advice and perusing the easier path. I was going to let it go and just let you come to the conclusion yourself but then you started blaming Torque.
This whole paragraph just has a bad attitude.
09/19/2014 (9:07 am)
Quote:
I can see a cube being rendered in one example, although how that is happening isn't completely clear.
And we're all willing to help you to understand it, but you have to show a willingness to learn first.
Quote:
So what if I did draw out all the geometry in C++? Why does that magically exclude that geometry from the ghosting problem? Because there is some broken scoping rule in the networking code that won't allow you to do it otherwise.
Because if you draw the geometry from within a single networked object you're making 1 netobject (1 TSStatic), which is 1 ghost, that produces a bunch of cubes for only 1 representation in the networking world. So, you'll have 1 static for, lets say, 10k cubes. Now you can have a world with thousands and thousands and thousands of cubes and torques networking doesn't care. When you add or remove a cube its an infrequent event compared to the updates that players will have moving around, etc. Now that single object is responsible for the networking of all of its 10k cube children. When a cube is changed, that object is flagged dirty, and sends an update for that one change.
I can't stress enough that we can all help you to accomplish this. I'm not trying to insult you, I'm trying to help you. You just keep dismissing my advice and perusing the easier path. I was going to let it go and just let you come to the conclusion yourself but then you started blaming Torque.
Quote:
Thanks for rubbing it in that there is someone accomplishing my goals who is 'more connected' to the guys in the forum. Goes a long way to help me see what I'm not understanding. You're suggesting that I'm not giving any effort when the effort required isn't trivial. We're talking about drawing verts by hand in C++ - again something that someone using an engine shouldn't have to be doing.
This whole paragraph just has a bad attitude.
#71
I feel that after a thread this long, and my past history on the boards it should be clear by now that I am trying diligently to learn. What do I need to do to see this more clearly, and earn the trust of others?
I know that probably any single one of the guys who has offered out advice in this thread could have probably had this up and running on day one. I feel as though I just get looked down on at every turn because I am only fluent with script. I have yet to have a single person offer out resources to help me learn how to draw verts or use a buffer in the first place. Just looking over code written in C++ isn't enough for a non-C++ coder. I do want to learn C++, and I plan on approaching that and being more fluent by next year. I would love some direction or instruction around where to start when drawing geometry in Torque.
09/19/2014 (9:25 am)
Firstly, thanks for returning and helping me to see the 1 ghost for 1000s of cubes example. This escalated quickly and I don't want it to go any further at this level. I respect everyone that frequents these boards equally. They've all gone the distance, made this engine what it is today, and try to share the wealth. Quote:And we're all willing to help you to understand it, but you have to show a willingness to learn first.
I feel that after a thread this long, and my past history on the boards it should be clear by now that I am trying diligently to learn. What do I need to do to see this more clearly, and earn the trust of others?
Quote:I can't stress enough that we can all help you to accomplish this.
I know that probably any single one of the guys who has offered out advice in this thread could have probably had this up and running on day one. I feel as though I just get looked down on at every turn because I am only fluent with script. I have yet to have a single person offer out resources to help me learn how to draw verts or use a buffer in the first place. Just looking over code written in C++ isn't enough for a non-C++ coder. I do want to learn C++, and I plan on approaching that and being more fluent by next year. I would love some direction or instruction around where to start when drawing geometry in Torque.
Quote:This whole paragraph just has a bad attitude.To be fair, I felt the same way about the last few posts you made. I'm a nice guy man, just trying to make a game. A charming one at that. I believe in Torque and I have for quite some time now. But I'm not willing to have someone tell me that drawing geometry by hand in C++ in order to scope a single object over a network is easy - and just believe that. Easy for you, sure. Easy for someone trying to move forward with Torque, not so much.
#72
You're right. I made the suggestion but I never really offered to help. There aren't really torque specific resources to help you accomplish this, but I will help you in anyway I can and I'm sure everyone else here will too.
Where are you at in terms of C++? For instance, have you compiled the engine and ran it? Or do you work off binaries? If you haven't done so yet try setting up a project and compiling and running it.
Torque has a 'source' folder for every project. Once you get it to compile normally, I would suggest taking the RenderMeshExample.cpp/.h files and copying them into your project's source folder, and renaming them to something more suiting.. like CubeArea or something ( I'm not too creative with names ) and then add those files to your project in C++ under the source folder (the one with torqueconfig.h) and change all the RenderMeshExample names inside the files to your new CubeArea. Now compile and make sure there's no errors. If you search in Torsion for RenderMeshExample you'll see the line where its added to the world editor so you can add your own objects. Follow suit and you should be able to add your new CubeArea (or whatever you called it) object to the world from inside the editor.
Let me know if you hit any problems, and if you want me to continue. I'll give you a whole run down on vertex buffers, primitive buffers, and everything you'll need to make headway on the issue so long as you want me to. It may be daunting but I promise you it's not that difficult.
09/19/2014 (10:25 am)
Quote:
I feel as though I just get looked down on at every turn because I am only fluent with script. I have yet to have a single person offer out resources to help me learn how to draw verts or use a buffer in the first place.
You're right. I made the suggestion but I never really offered to help. There aren't really torque specific resources to help you accomplish this, but I will help you in anyway I can and I'm sure everyone else here will too.
Where are you at in terms of C++? For instance, have you compiled the engine and ran it? Or do you work off binaries? If you haven't done so yet try setting up a project and compiling and running it.
Torque has a 'source' folder for every project. Once you get it to compile normally, I would suggest taking the RenderMeshExample.cpp/.h files and copying them into your project's source folder, and renaming them to something more suiting.. like CubeArea or something ( I'm not too creative with names ) and then add those files to your project in C++ under the source folder (the one with torqueconfig.h) and change all the RenderMeshExample names inside the files to your new CubeArea. Now compile and make sure there's no errors. If you search in Torsion for RenderMeshExample you'll see the line where its added to the world editor so you can add your own objects. Follow suit and you should be able to add your new CubeArea (or whatever you called it) object to the world from inside the editor.
Let me know if you hit any problems, and if you want me to continue. I'll give you a whole run down on vertex buffers, primitive buffers, and everything you'll need to make headway on the issue so long as you want me to. It may be daunting but I promise you it's not that difficult.
#73
I have been able to compile the engine for quite some time. Not too long ago I finally figured out how to add files to the source folder and regenerate and recompile the project. I am really lost when it comes to adding new libraries though. Like I have found some libraries that could be useful in the past if only I new how to add them as #includes. As for the syntax, I'm weak with it. I first approached programming with simple html, css, php and so on. Since then, TorqueScript has actually introduced me to the 'real' stuff like functions, classes, iterating, indexing, and so on.
With this in mind, I am able to browse source C++ and half-follow it. I get lost on some of the variables and stuff because the way they are declared isn't as black and white as script. Things like: public or private, U32, float, and so on confuse me. In script I can just drop a % sign in front of a variable and roll with it. I realize this just takes some research I haven't put in yet around the variable declarations.
As an Example for Community
This posting was the most genuine of any I've ever read on the Torque boards. I really feel as though you want to help, and I can assure you I value your efforts. I'm an artist by nature, I play music and I draw. I paint, I work in 3D - I'm permitted to be a little emo at times, it's in my nature as an artistic person. That being said, I can honestly say that yours is the first genuinely helpful posting I've seen since signing on the T3D boards and I am moved by your generosity.
Don't get me wrong, I've had plenty of direction from many a fantastic guy here on the forums. I also understand that if someone just basically 'does it for you' you won't ever learn to get off the ground in the first place on your own. The problem that I've come across time and time again, though, is that while all of this is certainly true the general presentation of advice is usually at a bit lower level than the average guy is going to be able to digest. Believe me when I say that I've spent days, if not weeks, studying samples handed to me in the past just to make sense of them. Given a solid walkthrough up front, though, could have potentially cut the time in half most times.
When I first started with Torque it was with T2D. I had the good fortune of running across a guy who uses the handle 'Doc'. Let me tell you that this guy is one of the kindest people to grace this community with his presence. He got me off the ground in the first place via e-mail and he taught me directly how to handle simple indexing and script to begin with. I was so excited at that time, I even typed out 2 documents as tutorials that are in the T2D wiki to this day(see links below).
I don't know exactly where I'm going with all of this, but I just wanted to say that sometimes just tossing out a 'read the docs' or look at this file doesn't cut it. Some of us (if not most) learn from real life examples and applications. I think everyone here can take a lesson in humbleness from Andrew today. Thank you so much!
Intro to the GUI:Part 1 and Intro to the GUI:Part 2 for anyone interested in T2D GUI :)
These are the results that come from being empowered by community to excel with the engine.
09/19/2014 (10:58 am)
Quote:Where are you at in terms of C++?
I have been able to compile the engine for quite some time. Not too long ago I finally figured out how to add files to the source folder and regenerate and recompile the project. I am really lost when it comes to adding new libraries though. Like I have found some libraries that could be useful in the past if only I new how to add them as #includes. As for the syntax, I'm weak with it. I first approached programming with simple html, css, php and so on. Since then, TorqueScript has actually introduced me to the 'real' stuff like functions, classes, iterating, indexing, and so on.
With this in mind, I am able to browse source C++ and half-follow it. I get lost on some of the variables and stuff because the way they are declared isn't as black and white as script. Things like: public or private, U32, float, and so on confuse me. In script I can just drop a % sign in front of a variable and roll with it. I realize this just takes some research I haven't put in yet around the variable declarations.
Quote:If you search in Torsion for RenderMeshExample you'll see the line where its added to the world editor so you can add your own objects. Follow suit and you should be able to add your new CubeArea (or whatever you called it) object to the world from inside the editor.I will try to make the connection here, this is about where I'm at as far as a starting place. Understanding how to get the C++ into the world editor in the first place. This posting made me aware that this was even an option; I had no idea that once the geometry was created it would be able to be accessed in this way. Just wow, already. A great starting place for certain. I'll let you know if I'm successful. I may be a bit busy this evening but by tonight I should be able to really put in some time on it.
As an Example for Community
This posting was the most genuine of any I've ever read on the Torque boards. I really feel as though you want to help, and I can assure you I value your efforts. I'm an artist by nature, I play music and I draw. I paint, I work in 3D - I'm permitted to be a little emo at times, it's in my nature as an artistic person. That being said, I can honestly say that yours is the first genuinely helpful posting I've seen since signing on the T3D boards and I am moved by your generosity.
Don't get me wrong, I've had plenty of direction from many a fantastic guy here on the forums. I also understand that if someone just basically 'does it for you' you won't ever learn to get off the ground in the first place on your own. The problem that I've come across time and time again, though, is that while all of this is certainly true the general presentation of advice is usually at a bit lower level than the average guy is going to be able to digest. Believe me when I say that I've spent days, if not weeks, studying samples handed to me in the past just to make sense of them. Given a solid walkthrough up front, though, could have potentially cut the time in half most times.
When I first started with Torque it was with T2D. I had the good fortune of running across a guy who uses the handle 'Doc'. Let me tell you that this guy is one of the kindest people to grace this community with his presence. He got me off the ground in the first place via e-mail and he taught me directly how to handle simple indexing and script to begin with. I was so excited at that time, I even typed out 2 documents as tutorials that are in the T2D wiki to this day(see links below).
I don't know exactly where I'm going with all of this, but I just wanted to say that sometimes just tossing out a 'read the docs' or look at this file doesn't cut it. Some of us (if not most) learn from real life examples and applications. I think everyone here can take a lesson in humbleness from Andrew today. Thank you so much!
Intro to the GUI:Part 1 and Intro to the GUI:Part 2 for anyone interested in T2D GUI :)
These are the results that come from being empowered by community to excel with the engine.
#74
09/19/2014 (11:43 am)
Quote:This is intriguing - piqued my interest on the same level as the guy who added networked deformation to the terrain....
Because if you draw the geometry from within a single networked object you're making 1 netobject (1 TSStatic), which is 1 ghost, that produces a bunch of cubes for only 1 representation in the networking world. So, you'll have 1 static for, lets say, 10k cubes. Now you can have a world with thousands and thousands and thousands of cubes and torques networking doesn't care. When you add or remove a cube its an infrequent event compared to the updates that players will have moving around, etc. Now that single object is responsible for the networking of all of its 10k cube children. When a cube is changed, that object is flagged dirty, and sends an update for that one change.
#75
Yeah, so in the engine there are C++ versions of the all the stuff you place in the world with torquescript. The C++ level just gives you more access to the internals of the engine as well as control what you expose to the scripting system. Take a look at this line:
github.com/GarageGames/Torque3D/blob/69838bdc8c9bc055b9b1ae76f42b0f28d2a33909/En...
It's literally saying "Make this a torquescript object called RenderMeshExample". By having that line in there, you expose the object to torquescript. So, now you could do "%blah = new RenderMeshExample()". If you didn't have that line, you wouldn't be able to access it in TorqueScript and it would remain a C++ only class.
A few other key functions to understand in this RenderMeshExample:
initPersistFields - this is where you define things that can be changed via the properties in world editor. So, when you select an object, over in the object inspector where you change position, rotation, and other properties you can expose variables to be changed via that and also via script ( %myObj.WhateverINamedTheField )
onAdd - this is called when the object is added to the scene, like when you double click on it and add it to the world, onAdd is called. Opposite to this is onRemove which happens when the object is removed from the scene. Either when the level changes, game is closed, or you delete the object.
packUpdate/unpackUpdate - this is your networking. packUpdate is called on the server side to package up any updates to send to the client. unpackUpdate is ran on the client side when the update is received.
prepRenderImage - this is when torque goes to draw the object. It's saying to your object "I'm going to draw you soon, give me all your verts, primitives, materials, etc and I'll take care of it.".
I'll pace myself, I don't want to overwhelm you with information. Are you fully versed on the 3D model side of things? Vertices, faces, etc? If not, that's something to touch up on. Let me know how you make out with getting your own copy of RenderMeshExample ( renamed to something more suiting ) into the world and we'll go from there.
09/19/2014 (12:38 pm)
Quote:
I had no idea that once the geometry was created it would be able to be accessed in this way.
Yeah, so in the engine there are C++ versions of the all the stuff you place in the world with torquescript. The C++ level just gives you more access to the internals of the engine as well as control what you expose to the scripting system. Take a look at this line:
github.com/GarageGames/Torque3D/blob/69838bdc8c9bc055b9b1ae76f42b0f28d2a33909/En...
It's literally saying "Make this a torquescript object called RenderMeshExample". By having that line in there, you expose the object to torquescript. So, now you could do "%blah = new RenderMeshExample()". If you didn't have that line, you wouldn't be able to access it in TorqueScript and it would remain a C++ only class.
A few other key functions to understand in this RenderMeshExample:
initPersistFields - this is where you define things that can be changed via the properties in world editor. So, when you select an object, over in the object inspector where you change position, rotation, and other properties you can expose variables to be changed via that and also via script ( %myObj.WhateverINamedTheField )
onAdd - this is called when the object is added to the scene, like when you double click on it and add it to the world, onAdd is called. Opposite to this is onRemove which happens when the object is removed from the scene. Either when the level changes, game is closed, or you delete the object.
packUpdate/unpackUpdate - this is your networking. packUpdate is called on the server side to package up any updates to send to the client. unpackUpdate is ran on the client side when the update is received.
prepRenderImage - this is when torque goes to draw the object. It's saying to your object "I'm going to draw you soon, give me all your verts, primitives, materials, etc and I'll take care of it.".
I'll pace myself, I don't want to overwhelm you with information. Are you fully versed on the 3D model side of things? Vertices, faces, etc? If not, that's something to touch up on. Let me know how you make out with getting your own copy of RenderMeshExample ( renamed to something more suiting ) into the world and we'll go from there.
#76
If I may, I did have a question. Just above the line you linked me about DECLARE_CONOBJECT I noticed this:
Anyways, I'm definitely more interested in the buffers and stuff you've mentioned :)
Man thanks! This is really exciting stuff man!
09/19/2014 (6:39 pm)
Amazing! I've never been so happy to see a cube in all my life Andrew! So now I can generate cubes that I've created with my altered RenderMeshExample! Was straightforward, your descriptions really helped in the above post. So much clarity packed into a single posting. I'm ready to proceed whenever you are man, this is super exciting stuff. If I may, I did have a question. Just above the line you linked me about DECLARE_CONOBJECT I noticed this:
public:
RenderMeshExample();
virtual ~RenderMeshExample();What does public mean? Do these couple lines declare sort of like a creation and a destructor method? Something like that? So many quick little questions I'd love to ask but I'll stay focused for now on the task at hand.Anyways, I'm definitely more interested in the buffers and stuff you've mentioned :)
Quote:Are you fully versed on the 3D model side of things?Yes, I definitely consider myself well versed with this. I can model fairly well from scratch, and also hold a deep understanding of node creation for collision-1, LOD's, adding materials, and so on.
Man thanks! This is really exciting stuff man!
#78
09/20/2014 (7:43 am)
Nice, thanks Danny. Finally making some sense, and now that you've pushed me over the edge...I'm freefalling through all of that site's documentation. Guess I'll begin my journey into C++ a bit sooner than I had planned this year :)
#79
Okay, I'm glad you know the modelling side of things quite well. This next part should be a bit easier because of that. The createGeometry() function is where the magic is happening for this particular cube. Take a look at this bit:
github.com/GarageGames/Torque3D/blob/69838bdc8c9bc055b9b1ae76f42b0f28d2a33909/En...
This is literally defining the verts that make up the cube. There's 8 verts to a cube, and 8 points in that list.
If you've noticed though, the vertex buffer is setup to contain 36 verts, not 8. This is because they're making multiple copies of each vert to build all the triangles. ( 6 sides, 2 triangles on each side, 3 verts per triangle, 6 * 2 * 3 = 36 ).
This is what a vertex buffer is in a nutshell. It's a list of all the vertices. This is what you'll be building. Right now there is only enough verts to make one cube, but it can hold any amount of verts you want to give it. So, when the time comes we'll be filling this up with all the verts for all the cubes that object holds. You'll notice the positions go from -1 to 1 in values. They could be any numbers, this cube is just built with 0,0,0 in its center. 0,0,0 in the model will be the center point when you're moving this model around in the torque world. This is referred to as "model space" vs "world space" which is the position in torque's world. It's the same as in blender or any other modeling program when you build your model in it's own little coordinate system in the modelling program.
They put together their list of vertices as such:
With each of those numbers corresponding to one of the points in the original list. So for instance the first 3 numbers are 3, 0, 1. Take a look at this diagram:

See how 3, 0, 1 makes up one triangle? If you follow through that whole list of 36 you'll see it makes up 12 unique triangles that build your cube. Where did I get these numbers? From this chunk:
github.com/GarageGames/Torque3D/blob/69838bdc8c9bc055b9b1ae76f42b0f28d2a33909/En...
You'll notice the first number in each entry they have corresponds to a number in my list. "{ 3, 0, 3 }, { 0, 0, 0 }, { 1, 0, 1 }," notice the 3, 0, 1? The layout of each entry in there is { Point Number, Normal Number, UV(texCoord) Number }. This is because a vertex in this vertex buffer is made up of 3 things. A point, a normal, and a UV coordinate. You'll recognize these terms from modelling programs. So, first they made an array of all the points, then all the normals, then all the uvs. Then they made an array called cubeFaces which contains, for each vertex, a number referring to an entry in cubePoints, cubeNormals, and cubeTexcoords arrays. This is what's used to fill out the actual entrys in the vertex buffer. They just did this to avoid repeating the same entrys over and over in code since some of the verts are repeating. It's not necessary, it's just cleaner code.
Don't worry too much about the normals or the UVs. Since you're using cubes for everything you'll be able to just leave those alone and use the same ones to build all your cubes. You'll just be filling the array with new vertex entries for each cube.
For now, try changing the positions in the cubePoints array and see how it changes the cube in your world. It should start to give you an idea of how it works. It's exactly like a modeling program only done in code.
Surely this is a lot to take in. We still have to cover the primitive buffer, but it's a lot easier to understand then the chunk I just covered. Let me know if you have any questions and how you get along with changing those point positions. Keep in mind you probably won't be changing much about these cubes, we're just going to eventually expand the vertex buffer to 2 cubes, which will be 72 vertex entries. We'll be able to control the positions of all the verts so we can make the cube next to it, above it, etc. all within this little world of the object.
09/20/2014 (9:27 am)
In C++ classes are made up of two pieces, the header file (.h) and the code file (.cpp). So, in the header file you'll see the layout of the class and in the code file you'll each of those functions defined and filled in. You're right, those are constructor and deconstructor. Public means it can be accessed by things outside of that class, which the constructor and deconstructor have to be obviously. If you make something private you can only use it from within that class.Okay, I'm glad you know the modelling side of things quite well. This next part should be a bit easier because of that. The createGeometry() function is where the magic is happening for this particular cube. Take a look at this bit:
github.com/GarageGames/Torque3D/blob/69838bdc8c9bc055b9b1ae76f42b0f28d2a33909/En...
This is literally defining the verts that make up the cube. There's 8 verts to a cube, and 8 points in that list.
If you've noticed though, the vertex buffer is setup to contain 36 verts, not 8. This is because they're making multiple copies of each vert to build all the triangles. ( 6 sides, 2 triangles on each side, 3 verts per triangle, 6 * 2 * 3 = 36 ).
This is what a vertex buffer is in a nutshell. It's a list of all the vertices. This is what you'll be building. Right now there is only enough verts to make one cube, but it can hold any amount of verts you want to give it. So, when the time comes we'll be filling this up with all the verts for all the cubes that object holds. You'll notice the positions go from -1 to 1 in values. They could be any numbers, this cube is just built with 0,0,0 in its center. 0,0,0 in the model will be the center point when you're moving this model around in the torque world. This is referred to as "model space" vs "world space" which is the position in torque's world. It's the same as in blender or any other modeling program when you build your model in it's own little coordinate system in the modelling program.
They put together their list of vertices as such:
3, 0, 1, 2, 0, 3, 7, 4, 5, 6, 4, 7, 3, 5, 2, 7, 5, 3, 1, 4, 6, 0, 4, 1, 3, 6, 7, 1, 6, 3, 2, 4, 0, 5, 4, 2
With each of those numbers corresponding to one of the points in the original list. So for instance the first 3 numbers are 3, 0, 1. Take a look at this diagram:

See how 3, 0, 1 makes up one triangle? If you follow through that whole list of 36 you'll see it makes up 12 unique triangles that build your cube. Where did I get these numbers? From this chunk:
github.com/GarageGames/Torque3D/blob/69838bdc8c9bc055b9b1ae76f42b0f28d2a33909/En...
You'll notice the first number in each entry they have corresponds to a number in my list. "{ 3, 0, 3 }, { 0, 0, 0 }, { 1, 0, 1 }," notice the 3, 0, 1? The layout of each entry in there is { Point Number, Normal Number, UV(texCoord) Number }. This is because a vertex in this vertex buffer is made up of 3 things. A point, a normal, and a UV coordinate. You'll recognize these terms from modelling programs. So, first they made an array of all the points, then all the normals, then all the uvs. Then they made an array called cubeFaces which contains, for each vertex, a number referring to an entry in cubePoints, cubeNormals, and cubeTexcoords arrays. This is what's used to fill out the actual entrys in the vertex buffer. They just did this to avoid repeating the same entrys over and over in code since some of the verts are repeating. It's not necessary, it's just cleaner code.
Don't worry too much about the normals or the UVs. Since you're using cubes for everything you'll be able to just leave those alone and use the same ones to build all your cubes. You'll just be filling the array with new vertex entries for each cube.
For now, try changing the positions in the cubePoints array and see how it changes the cube in your world. It should start to give you an idea of how it works. It's exactly like a modeling program only done in code.
Surely this is a lot to take in. We still have to cover the primitive buffer, but it's a lot easier to understand then the chunk I just covered. Let me know if you have any questions and how you get along with changing those point positions. Keep in mind you probably won't be changing much about these cubes, we're just going to eventually expand the vertex buffer to 2 cubes, which will be 72 vertex entries. We'll be able to control the positions of all the verts so we can make the cube next to it, above it, etc. all within this little world of the object.
#80
I do understand the diagram and the tris. Where I do get confused is how we come up with the Point Number in the first place. For example: If we look at {3,0,3} and ignore all but the first number we are left with 3 as the Point Number. Using your diagram, it is easy to see that 3 represents that vertex point. However, looking at the code there is no prior mention of the list of points as you defined here:
You are explaining this exactly on the level I am needing you to man, thanks. I can totally understand the coordinates and how they work together to make the geometry for sure. I can see how to add vertex points and then use those as a Point Number in the cubeFaces part. I'm taking it that the cubeFaces part is the actual vertex buffer - that listing of the Point Number, Normal Number, and UV.
EDIT: Oh snap, I think I'm starting to piece together a pattern here...it's the order the original cubePoints were declared perhaps?
EDIT2: Okay yea, this definitely seems to be making a lot of sense all of a sudden...don't stop now man, I'm soaking this up like a sponge!
EDIT3: Nice, I think I've got the first 2 parts of the vertex buffer understood. The Point Number is just the index # of the cubePoints. The Normal Number is just the index # of the cubeNormal. Looks like the cubeNormal is just declaring the 'side' of the cube (x,y,z,-x,-y,-z or top,bottom,left,right,front,back). A bit hazy on how the TexCoords actually work, but I do understand that they are referenced in the cubeFace array there by the index # (0,1,2,3).
EDIT4: Okay, going back over your posting I see that you did outline the Point#, Normal#, and UV# but I must not have grasped it on the first read. Sweet. I'm going to take the initiative here and see if I'm able to create an extended object that consists of 2 cubes while I wait for you to get un-busy enough to reply :D - Really wish I understood the syntax more because I'd just make a function with nested for loops to generate all those points instead of going in by hand...lol
09/20/2014 (10:15 am)
Wow nice. Believe it or not this part was pretty easy for me to take in :) I had already deduced the cubePoints[8] was dealing with the actual verts of the cube! I started to get confused when all of a sudden I was seeing 36 entries, which you've been quick to enlighten me on. Now I can easily see how it is possible to create geometry per vertex, but there is still a little bit of a disconnect for me. I do understand the diagram and the tris. Where I do get confused is how we come up with the Point Number in the first place. For example: If we look at {3,0,3} and ignore all but the first number we are left with 3 as the Point Number. Using your diagram, it is easy to see that 3 represents that vertex point. However, looking at the code there is no prior mention of the list of points as you defined here:
Quote:So if I were to attempt to create other cubeFaces, how could I determine these? Is it just something I should always understand as each octant being predefined as you have outlined in the diagram? In other words, what makes Point 0 the number 0 instead of say, 2? Or any other number?
3, 0, 1, 2, 0, 3, 7, 4, 5, 6, 4, 7, 3, 5, 2, 7, 5, 3, 1, 4, 6, 0, 4, 1, 3, 6, 7, 1, 6, 3, 2, 4, 0, 5, 4, 2
You are explaining this exactly on the level I am needing you to man, thanks. I can totally understand the coordinates and how they work together to make the geometry for sure. I can see how to add vertex points and then use those as a Point Number in the cubeFaces part. I'm taking it that the cubeFaces part is the actual vertex buffer - that listing of the Point Number, Normal Number, and UV.
EDIT: Oh snap, I think I'm starting to piece together a pattern here...it's the order the original cubePoints were declared perhaps?
EDIT2: Okay yea, this definitely seems to be making a lot of sense all of a sudden...don't stop now man, I'm soaking this up like a sponge!
EDIT3: Nice, I think I've got the first 2 parts of the vertex buffer understood. The Point Number is just the index # of the cubePoints. The Normal Number is just the index # of the cubeNormal. Looks like the cubeNormal is just declaring the 'side' of the cube (x,y,z,-x,-y,-z or top,bottom,left,right,front,back). A bit hazy on how the TexCoords actually work, but I do understand that they are referenced in the cubeFace array there by the index # (0,1,2,3).
EDIT4: Okay, going back over your posting I see that you did outline the Point#, Normal#, and UV# but I must not have grasped it on the first read. Sweet. I'm going to take the initiative here and see if I'm able to create an extended object that consists of 2 cubes while I wait for you to get un-busy enough to reply :D - Really wish I understood the syntax more because I'd just make a function with nested for loops to generate all those points instead of going in by hand...lol
Torque Owner Jesse Allen
@Daniel: Totally didn't expect you to pick me up when I was down like that. Respect.
You're right man. I've come this far, no reason to start blaming anyone else for my own shortcomings. You've all been helpful, and for that I am grateful. I really do want to see things like this improved for future users of Torque. That's something I'd be proud to be a part of. When I get so many experienced Torque vets poking their heads in the room without any conclusive results though, it starts to make me feel as though I should just proudly don my dunce cap.
This is all happening during a hard transitional period for me as well. I was full speed ahead developing a really cool game on an alien planet...and then I learned about Xenocell. Totally killed it for me, because I realized those guys made my game give or take a feature or two. Not that it's a bad thing; I was super happy to see it actually! What killed it for me was coming to the realization that those super experienced devs worked 4 years and still didn't get to where they were going. Honestly I had to hold a moment of silence for Konrad and crew, I personally felt the agony of that unsuccessful kickstarter. It deeply affected me, to the point that I decided to change gears altogether on my direction. (If you happen across this Konrad, I want you to know that you are among the greats in my book!)
Anyways that's all more like personal blog material - but since you've gone the extra mile I figured I'd share :P I'll continue to persevere, who knows, maybe I can get something done once I stop returning to the drawing board. I sometimes feel as though I have a snowball's chance in Hell, but for some odd reason having the deck stacked against me only serves to fuel the machine.
edit: I forgot to thank you for the sample script. I'll give it a go when I get a chance to go through it all thoroughly.