Replacement SceneLoader/proxy scene objects?
by Jesse Lowther · in Torque X 2D · 06/22/2010 (6:10 pm) · 11 replies
The situation:
I've noticed that it's often frustrating dealing with creating levels with Torque X... for example, the main character in the game I'm making has a variety of components associated with him, each with custom settings and animations attached.
When creating a new level, I can copy the old level's file and start from there... but when I start changing properties on that main character scene object (for example), I then have to go back and make the exact same change to all my other levels.
My gut tells me that there's a better way to do this... potentially by making some kind of utility to extract this common information into their own files, and then refer to those files in the editor instead of creating the specific scene objects over and over.
However, it looks to me like the default scene loader isn't going to cut it. I imagine that most people who've ever worked with Torque X have probably come to the same point that I'm now coming to. What I'm wondering is... has someone in the community already made a solution for this problem? I'd prefer not to bother reinventing the wheel if I can help it. =)
My current idea for resolving the issue:
If no one has created such a thing and people would find it useful, I can give it a shot! Here's what I'm picturing... (let me know if you think this is a terrible idea/if there would be a better way of doing it/if it's already done and I'm wasting my time. ;) )
* A utility that gives a list of scene objects in a given level, and lets you extract just the XML related to that scene object into its own file. (Possibly also extracting its materials if it is the only object in the level using said materials) For the purposes of this brainstorm, I'll call the files this utility creates "fragment files".
* A modification to SceneLoader to let you inject the contents of these fragment files into levels (including loading their materials and such as needed).
* A "Proxy" component that would let you refer to one of these fragment files in the level editor instead of creating and referring to the whole complex object all over again directly.
* A modification to the SceneLoader to, as it loads a fragment file for a given Proxy component, lets you merge the components already defined in the fragment file with any components put on the Proxy scene object. (For example, both the fragment file and the Proxy scene object could have their own PhysicsComponent attached... say if you had a moon level and wanted to override just the physics of the otherwise untouched object... if the Proxy's scene object defines a component, it always overrides whatever was in the fragment file)
The big questions:
What do you guys think of this idea? Has someone already beaten me to the punch on this concept? ... is there possibly a better way of doing it than the way I'm imagining? Or has something like this not been created yet, and thus it would be useful for the community at large and I should set to work making it happen?
I've noticed that it's often frustrating dealing with creating levels with Torque X... for example, the main character in the game I'm making has a variety of components associated with him, each with custom settings and animations attached.
When creating a new level, I can copy the old level's file and start from there... but when I start changing properties on that main character scene object (for example), I then have to go back and make the exact same change to all my other levels.
My gut tells me that there's a better way to do this... potentially by making some kind of utility to extract this common information into their own files, and then refer to those files in the editor instead of creating the specific scene objects over and over.
However, it looks to me like the default scene loader isn't going to cut it. I imagine that most people who've ever worked with Torque X have probably come to the same point that I'm now coming to. What I'm wondering is... has someone in the community already made a solution for this problem? I'd prefer not to bother reinventing the wheel if I can help it. =)
My current idea for resolving the issue:
If no one has created such a thing and people would find it useful, I can give it a shot! Here's what I'm picturing... (let me know if you think this is a terrible idea/if there would be a better way of doing it/if it's already done and I'm wasting my time. ;) )
* A utility that gives a list of scene objects in a given level, and lets you extract just the XML related to that scene object into its own file. (Possibly also extracting its materials if it is the only object in the level using said materials) For the purposes of this brainstorm, I'll call the files this utility creates "fragment files".
* A modification to SceneLoader to let you inject the contents of these fragment files into levels (including loading their materials and such as needed).
* A "Proxy" component that would let you refer to one of these fragment files in the level editor instead of creating and referring to the whole complex object all over again directly.
* A modification to the SceneLoader to, as it loads a fragment file for a given Proxy component, lets you merge the components already defined in the fragment file with any components put on the Proxy scene object. (For example, both the fragment file and the Proxy scene object could have their own PhysicsComponent attached... say if you had a moon level and wanted to override just the physics of the otherwise untouched object... if the Proxy's scene object defines a component, it always overrides whatever was in the fragment file)
The big questions:
What do you guys think of this idea? Has someone already beaten me to the punch on this concept? ... is there possibly a better way of doing it than the way I'm imagining? Or has something like this not been created yet, and thus it would be useful for the community at large and I should set to work making it happen?
About the author
#2
So after a bit of hacking, prodding, and investigating, the only way I've found to do this so far (that is, to actually merge two scenes together in this manner) has been to explicitly remove the T2DSceneGraph node reference from the actual levels that are subsequently loaded, but this seems painfully manual.
Doing this seems to trick the engine into loading the scene's objects and resources into the original ("common resource") scene's scene graph... which works, but it feels like an unsafe hack to me: both because I have to manually alter the subsequent scene files every time I change them, and because I'm basically tricking the engine. It's working... it's merging the data from the two scene files when I do this... but I get the feeling that I'm committing a big no-no by doing it.
For instance, I don't know whether when I subsequently call "UnloadLastScene" to load yet another level if it's actually freeing the resources from the prior non-common level that loaded.... I can't see those prior objects anymore, and it does successfully load the new level, but I can't tell whether I've created a massive memory leak because it thought that those objects were going to be part of their own scene graph. (Perhaps with sufficient investigating I can figure out for sure, but I am already getting the feeling that there must be a less hacky way to solve this problem)
So... Duncan, how did you do it? Did you find a better way?
After some exhaustive searching of the docs and forums, it looks like in normal Torque2D there's an easy way to merge scene data together in this manner... but I've been unable to determine how to do so other than this manual way in TorqueX 2D... there's no "addToLevel()" method equivalent in TorqueX as far as I can tell, and attempts to manually load the resources by doing (for example) a .AddRange on the old SceneGraph's objects with the new SceneGraph's objects fails as well, and is probably just as terrible of a hack as the one I did above. =)
Anyways, I could definitely use the advice of someone who (it sounds like) has managed to find a safe way to do this already. Thank you. =)
06/29/2010 (10:08 pm)
EDIT: I should mention, I made the original post above... I've been helping my friend Jesse make this game, and I finally decided to get my own account in here and snag my own copy so we could both work on it simultaneously. =) So, continuing the discussion... =)So after a bit of hacking, prodding, and investigating, the only way I've found to do this so far (that is, to actually merge two scenes together in this manner) has been to explicitly remove the T2DSceneGraph node reference from the actual levels that are subsequently loaded, but this seems painfully manual.
Doing this seems to trick the engine into loading the scene's objects and resources into the original ("common resource") scene's scene graph... which works, but it feels like an unsafe hack to me: both because I have to manually alter the subsequent scene files every time I change them, and because I'm basically tricking the engine. It's working... it's merging the data from the two scene files when I do this... but I get the feeling that I'm committing a big no-no by doing it.
For instance, I don't know whether when I subsequently call "UnloadLastScene" to load yet another level if it's actually freeing the resources from the prior non-common level that loaded.... I can't see those prior objects anymore, and it does successfully load the new level, but I can't tell whether I've created a massive memory leak because it thought that those objects were going to be part of their own scene graph. (Perhaps with sufficient investigating I can figure out for sure, but I am already getting the feeling that there must be a less hacky way to solve this problem)
So... Duncan, how did you do it? Did you find a better way?
After some exhaustive searching of the docs and forums, it looks like in normal Torque2D there's an easy way to merge scene data together in this manner... but I've been unable to determine how to do so other than this manual way in TorqueX 2D... there's no "addToLevel()" method equivalent in TorqueX as far as I can tell, and attempts to manually load the resources by doing (for example) a .AddRange on the old SceneGraph's objects with the new SceneGraph's objects fails as well, and is probably just as terrible of a hack as the one I did above. =)
Anyways, I could definitely use the advice of someone who (it sounds like) has managed to find a safe way to do this already. Thank you. =)
#3
Thanks.
07/16/2010 (1:19 am)
Any more anyone has to say on this subject? Any advice would be appreciated.Thanks.
#4
Loading multiple scenes does have a lot of overhead though. I was originally keeping enemies in one file, special effects in another file, etc. However we hit the Xbox 360 memory limit after loading ~4-5 scenes into memory because it seemed like it was allocating ~100MB for each scene that was loaded. We put everything into one scene (and it takes a while to load now), but it uses a lot less RAM.
That's how I'm doing it now, but I'm hoping I'll come across a better way or something will be implemented that lets us link scene files.
07/16/2010 (3:53 am)
I think a lot of us have run into this problem. I felt the same way you do: I didn't want to make changes to all my players/enemies in each different scene file if I wanted to make a global change. The route I took is basically the one that Duncan mentioned. I created a scene file that only holds templates. I load this templates file into memory once, and then all my level files basically just contain placeholders. When the camera detects a placeholder, it looks up what the object is really supposed to be in the templates file and spawns that.Loading multiple scenes does have a lot of overhead though. I was originally keeping enemies in one file, special effects in another file, etc. However we hit the Xbox 360 memory limit after loading ~4-5 scenes into memory because it seemed like it was allocating ~100MB for each scene that was loaded. We put everything into one scene (and it takes a while to load now), but it uses a lot less RAM.
That's how I'm doing it now, but I'm hoping I'll come across a better way or something will be implemented that lets us link scene files.
#5
(1) the editor saves references to ALL the materials in the project in every scene that you save
(2) the tx engine loads new copies of all the materials in a scene when the scene is loaded - it does not use copies that have already been loaded by another scene.
So if you have 100mb (in memory) of textures and stuff, then every time you load a scene it fills up that much memory.
There are a number of options for you here. The two simplest ones are:
(1) grab Matt's material ripper to strip unused material references from your scene files
(2) get Pino's modified engine which includes a fix so that tx does not load new copies of textures; uses the already loaded ones instead.
07/16/2010 (11:23 am)
Loading multiple scenes using the unmodified TX engine will usually take up a lot of memory. This is because of a combination of two things.(1) the editor saves references to ALL the materials in the project in every scene that you save
(2) the tx engine loads new copies of all the materials in a scene when the scene is loaded - it does not use copies that have already been loaded by another scene.
So if you have 100mb (in memory) of textures and stuff, then every time you load a scene it fills up that much memory.
There are a number of options for you here. The two simplest ones are:
(1) grab Matt's material ripper to strip unused material references from your scene files
(2) get Pino's modified engine which includes a fix so that tx does not load new copies of textures; uses the already loaded ones instead.
#6
I did notice a strange thing before however: one of the objects we had in the common file was a template, and I found that when I instantiated an instance of the template in a newly loaded scene it wouldn't actually appear... but I wonder if that was due to other problems I was experiencing. It will probably be worth it for me to try that again. Thanks for the input guys, we deeply appreciate it! =)
07/17/2010 (2:34 am)
Ah, aye: we're using Pino's modifications, so hopefully that should be a good start at least. I think the idea of using templates makes more sense to me now that I've implemented the more "heavy handed" approach I mentioned here and in the other thread.I did notice a strange thing before however: one of the objects we had in the common file was a template, and I found that when I instantiated an instance of the template in a newly loaded scene it wouldn't actually appear... but I wonder if that was due to other problems I was experiencing. It will probably be worth it for me to try that again. Thanks for the input guys, we deeply appreciate it! =)
#7
07/21/2010 (1:38 am)
Thanks for the tips Duncan! I've read through some of Pino's thread, but I was wondering if that fix you mentioned (new copies of textures are not loaded) will be making it into the next release of TX2D? If that's the case, then I can wait for that. If not, I'll look into using those modifications. That helps us a lot, thanks!
#8
07/21/2010 (10:23 am)
I haven't had the chance to trial the Beta yet so I don't know. My hunch would be that it is not in the Beta though as the team is currently working on stability for this release rather than altering functionality too much. 
#9
I have no idea (yet) what challenges this will present.
I've tagged several objects (my main player included) as "persistent" thinking that this would ensure that he is not unloaded when the level/scene is unloaded. Hopefully that's a safe bet, but my plan was not to include the player object in subsequent .txscene files (which it seems the author of this thread is doing). So I'm not sure how that will work.
Also, what if my gamer has a saved game and loads a save from level 2? If my player SceneObject is not in my level 2 scene file (because I tagged him as persistent for level 1), then is my main player ever loaded at all?
I had hoped that by tagging him as persistent it would make him part of the project. Then all I'd have to do from level to level would be to set his proper starting position.
That was my hope when I first started using TX...experience has taught me...that it's never that easy. :)
I'm still hopeful that level transitions will be manageable without taking too much focus away from building my actual game.
Good luck...
--RB
07/21/2010 (12:56 pm)
This is an interesting conversation for me, because I'm nearing the point in my game where I'm going to have to unload my first level and load up my second. :)I have no idea (yet) what challenges this will present.
I've tagged several objects (my main player included) as "persistent" thinking that this would ensure that he is not unloaded when the level/scene is unloaded. Hopefully that's a safe bet, but my plan was not to include the player object in subsequent .txscene files (which it seems the author of this thread is doing). So I'm not sure how that will work.
Also, what if my gamer has a saved game and loads a save from level 2? If my player SceneObject is not in my level 2 scene file (because I tagged him as persistent for level 1), then is my main player ever loaded at all?
I had hoped that by tagging him as persistent it would make him part of the project. Then all I'd have to do from level to level would be to set his proper starting position.
That was my hope when I first started using TX...experience has taught me...that it's never that easy. :)
I'm still hopeful that level transitions will be manageable without taking too much focus away from building my actual game.
Good luck...
--RB
#10
07/21/2010 (1:59 pm)
Persistent just means it won't get unloaded. It has to be loaded at least once to exist though.
#11
Thanks for the info, Duncan.
--RB
07/21/2010 (2:06 pm)
I think I'm going to have to go the route of moving all of my persistent objects into one .txscene file that never gets unloaded then.Thanks for the info, Duncan.
--RB
Torque Owner Duncan Colvin
hth