Splitting the forest editor?

#1 Create a new brush or element for each level youre working on, even if its a duplicate of the same tree model used in another level, because it effectively lets you specify the settings for that specific level as that tree/element is only used in that level.

Also, you can port forests without any hassle, just did it yesterday from t3d 1.2 to my git install, have to make sure you copy over the .forest file in the /levels folder, aswell as copy over your datablock entries in the art/forest/ folder...


[edit] Just read your post again, sorry but im not really sure what your asking or proposing, I realise that what i just said seems to be what you are doing but I dont see what the issue is.

I ported over 9 forests that way from 1.2 to 3 without any issues. The only problems i had was the root changes for some skies and water etc, just needed their links adjusted to reflect the new location.

#2 Example: You play a game and someone submits a map that he made for this game in order to update the game with this map, but the problem is, it has a forest and a brushes.cs and managedItemData.cs, so you cannot install this map additionally to the game. You would have to overwrite the files in art/forest/ destroying the rest of the maps.

#3 well yeah, ok i see your point, currently you would have to merge them, add the new datablocks for that map into the existing one.

#4 Currently there is a bug if you mess around with these datablocks, I think mainly if you delete an item in the forest editor, causing the files to corrupt. Other even reported errors where their forests got deleted and so on, I encountered also weird errors because of this. So at the moment this is not a good solution, since it is a one way street that will corrupt all your data after a while leaving you no way back to correct it except building all from scratch again every time.

My idea would be to organize the levels in folders and making him look for brushes.cs and managedItemData.cs in the folder the mission file is in.
So levels/mymission1/ instead of art/forest/
Or using a prefix like mymission1_forest.cs.

#5 If you're going to make brushes and managed meshes mission depending, then you won't be able to make overall changes anymore. This is because when you create more levels with 'initially the same forest items', you'll end up with duplicates.

You're right about the Forest Editor being a bit wonky. This is since 1.1! I even made a batch file to auto backup the forest files every hour I'm working in T3D.

Instead of trying to change the whole thing it would be better to try to fix it first IMO.

#6 In almost any game or engine you can install additional maps/levels by just copying them in, but if its Torque and has a forest not possible...

At the moment I have a few levels with corrupted forest files resulting in not being able to make any real changes anymore and getting annoyed with bugs.

Sure you could still make overall changes with my system, you just have to use the same datablock and if you want to do something total different, you can create a new one. Even if you want to make overall changes over many datablock you could copy them over. So I don't see any drawbacks there.

#7 Actually, there aren't that many games that allow you to import your own meshes along with the changed scripts. Skyrim comes to mind as one that does, and even then very few people create custom meshes because it's a bloody nightmare to bring in. I know there are plenty of indie games out there trying to break this mold, but at the end of the day if we were to adopt the split forest approach, $50 says that within 6 months people are crying out for a more streamlined system of having a single forest cache.

#8 I don't talk about changing the game, I talk about exchanging maps and this is totally common in computer games.
Maybe today companies invented some barriers so that is no longer possible, but since the invention of FPS games, in most cases it was possible to grab the editor, make a map, send it over to another one, he could paste it into his game and it will work. This worked with quake, half-life, unreal and cry-engine and most of the games based on that, even changing scripts, importing custom objects was always possible in these engines, even though they were not even open source.

Torque is the first thing I discover where this is not so easy.

So what would you suggest to solve this problem? Adding all to one main forest data file resulting in having to scroll through hundreds of entries when placing a single tree every time while not being allowed to change anything, because this would change the other peoples maps also and eventually will trigger the evil bug destroying all data?
I like my idea better...

#9 Additionaly:In some cases you might want to paint things with the forest editor, that normally you don't, like buildings or so, for example for huge maps, to increase performance or just to make painting simpler.

And if you work on a project with more people every artist will add his own datablocks into the forest with his own settings, that likely never get used anymore in the future, but you have to keep them there all the time and confuse other people using the editor.

Also you would have to keep track by hand what items are used in what level with what kind of entries in the main datablocks, otherwise you likely break things without knowing what you destroyed. Since you have no chance fixing the misson.forest file directly, because it is written in egyptian.

I hope I could convince a little that there is some need for a fix or at least to find a work around.

#10 I see your point, and even though its more of a game feature rather than a engine feature, its something thats pretty much standard flexibility now with most games, the ability to mod it (when on PC, not those silly console things...) and this would just make that feature alot easier for dev's to include.

Thinking about it while at work tonight, one of the things that springs to mind is to keep the current managedItemData/brushes files as more of a central library for the developer, but to duplicate the entry of any brushes used in a level from the brushes.cs into the .mis file, and modify the engine to pull the information for the forest data from the .mis (reduces the amount of files being created as the .mis is already there an in use anyway)

#11 I don't understand everything that is being discussed here, but it sounds like the forest editor has a structure that makes it difficult to collaborate/mod on level development.

So here is what I am getting from this:

1. Single data structure in a file for all forest settings in a game.
2. Mixing levels developed on anther machine with a different settings file creates problems. Corrupted data for all missions.
3. The goal of splitting would be to compartmentalize this data between missions.
4. The draw back of splitting is you lose your "templated" forest settings that speed up mission creation.

Am I seeing this right?

So is the solution to create a centralized templated forest settings that are copied and used in a separate file for each mission? Possibly with tools for merging this forest data between 2 missions? This sounds like what Andy is proposing.

#12 got it in one there frank.

use the managedItemData/brushes just for saving your 'tools' so making your own level creation smoother by having templates to hand, but the actual forest information is saved in another file and pulled from there when the missions loaded.

just as a brainfart... what about storing it like a header in the .forest file ... possible ? The .mis file would already have a forest set up in its sim groups, it would just need to look at the forest file for the tree locations and tree type/settings, or am i missing something out here as to why there separate in the first place ?

#13 Is the forest data saved as a binary? From Duion's "in Egyptian" statement that sounds about like what it does.

I like that idea, Andy - just encapsulate each forest's data using a UUID or somesuch to avoid conflicts and store it in the forest data file. Problem solved, and as a bonus you can still edit it.

#14 Why don't we just add the possibility to save & load items & brushes just like for example you do in Photoshop(R)

- If you're providing a level you'd add the custom items & brushes using a "save as..." function
- When adding this level to your project you "load" the provided items & brushes if you need to work with those.
- Store the current path to items & brushes in game/tools/settings.xml
- When you want to back up your items & brushes use the "save as..." and use a different name.
- The default path and filename for items & brushes would stay unchanged: (game/art/forest/)
- The whole thing remains compatible with existing projects

Further would adding a "lock" option - protecting the file from overwriting - give a solution to the problem of corrupted items & brush files due to various reasons (a common issue)

#15 The paths to the files are stored here by default:

game.cs

function onServerCreated()
{
   ...
   %datablockFiles.add( "art/forest/managedItemData.cs" );
   ...
}

and
game/tools/forestEditor/main.cs

function ForestEditorPlugin::onWorldEditorStartup( %this )
{  
   ...
   %brushPath = "art/forest/brushes.cs";
   ...
}

Edit: If you are able to work with unique names for items and brushes, you could just add a path in those functions pointing to the levels folder and using the name of the level you're loading. Then your solution will be pretty easy; just a few lines of script. Or am I still not getting the point here... then: nevermind!

#16 @Nils
Imaging you want to give us a level you made in your template project to install in our template project.

At the moment you would have to zip up the whole file tree you used and give that us to copy it over our template, but this will not be compatible with our setup and folder structure and also overwrite some files, like the forest brushes.cs, even if you merge the conflicting files by hand, you still have potential to mess our projects up.

In the ideal case you would just give us your mission files located in maybe levels/nils_mission/ so we just have to add one folder under levels and it will run from there, without spreading or overwriting files in our projects.

#17 @Duion; Then I got already. The solution wouldn't be to hard and can be done with torque script. I'm sorry I don't have the time now to come up with something.

However; I don't believe just add a mission/level by putting stuff in the levels folder alone. You have also assets and perhaps additional scripts to deal with. My levels won't work properly without the scripts for triggers, to give an example. Or the cool particle emitter you made for that waterfall, not to forget the decals! So often you'd still need to open a script and add/change something to make it all work. The same with the items & brushes. It isn't really that hard to add a few datablocks (managedItemsData) or add a few brushes by hand while you made a copy for safety. Do you ever use torsion? -- But thats all opinion and I don't want to nag, really :)

#18 The assets and scripts are another building site, lets assume we just want to exchange mission files.
Other games often add simplified editors within the game so people can build their own maps/missions. In this case the assets and scripts are more or less fixed and if people still want to add custom assets and scripts they can provide them within their file or in additional folders.

Assets and some scripts already work pretty modular within Torque. Sure some other things would need improvement also but in my opinion the forest editor is the worst.

Sure adding some lines to a file by hand is not that hard, but you have to imaging that it can get quite annoying and problematic if you ever when you want to add content to a project where more than one people work on, you would have to tell everyone what to do in order to add your content. Something like "please copy a to b and then open file xy and add yxz to line 253, but don't touch any other or you will break everything" will add confusion, but instead something like "copy a over to b or just launch the installer" is something everyone should be able to understand and do.
I know at the moment there is not much projects with Torque who have user content or updates and add-ons, but there are people selling packs and if they contain demo levels you should be able to test them with your project by copying it in and if you are done remove it again without changing your whole setup or breaking anything.

Imaging someone offers a pack of 10 levels with 500 entries in the forest editor and you want to add level 5 with 50 entries, then you could either keep the rest 450 junk-entries forever or sort them out by hand, introducing weird bugs to all your level forests from time to time, or delete the whole forest and paint everything again with your settings which likely is even faster.

#19 Duion, having thought about this more, its a bit of a moot point as it is not just the forest editor that will be an issue with level sharing this way, depending on how they have made the level, you will also have issues with terrain materials, any precipitation and decals too, as they use similar system to the forest, decals and precipitation being stored in a central file to store the datablocks info, and dependant on file structure for the terrains materials.cs file, it could be rooted at the main /terrains/materials.cs instead of a sub directory with its own separate materials.cs.

#20 One would think the easiest way to allow this kind of mod or addition to take place is to simply make sure that all missions that require non stock models, textures, etc package their assets in a zip along with the mission file and then distribute that instead. If it's packaged correctly, your sure not to be missing anything. You will have to modify the forest and possibly other objects to deal with that, but if your modifying the forest to allow it, the rest should be a lot easier. Most of the engine already allows reading from a zip package, so I don't see that as a big hurdle.

Otherwise it's going to be way too complicated for most users that want to do a level to even deal with it, unless they are so fervently hardcore about your game.