Will T2D be able to deallocate unreferenced textures?

#21 Yeah nothing personal Smaug.

Something I have learned (and I keep relearning actually, the hard way) is that some issues can't be solved by writing/email and some issues are actually better NOT discussed in email.

In my day job I work in a company with a large IT organization; dozens of programmers, DBAs, QA testers, project managers, and thousands of end-users. If it weren't for teleconference, videoconference, travel, face 2 face, there would be utter madness.

#22

Quote:Just because TGE's mission system did it that way doesn't make it a good idea (for T2D or TGE). There are any number of reasons why one would want to keep some datablocks around, let alone any number of games that do this as a matter of course. It may work for a typical multiplayer FPS arena-based game, but even a singleplayer FPS should keep the main character's stuff around from mission to mission.

Heh, you missed that discussion/argument by about eight months. Melv originally outlined his reasons for using datablocks for T2D image storage a while, in the original fxSceneGraph2D thread where he demostrated his new whiz-bang TGE 2D scene graph add-on.

You might not agree with that method, but that's how it is. I'm not going to discuss the whole my vision/Melv's vision because as far as I am concerned, Melv has done a dang good job to make T2D as compromising as possible to fit everybody's vision of what a generic 2D game engine should be. No, my vision doesn't overlap Melv's 100% either, but that's why you and I are code monkeys. If we don't like the status quo, we change it. (Or write our own, if the code isn't available.)

Heck, the original T2D wasn't even going to have a source release, and that's lead to a lot of these problems like TGE-reference material being unavailable to T2D-only owners. But GG changed its mind -- I like to think just because of people like me and you, who just can't resist tinkering/improving a bunch of neat code.

Quote:This just sound like a bug. The datablock is what (effectively) owns the TextureHandle. It makes sense that the removal of the block should cause its deletion. Unless TextureHandles can be owned by multiple datablocks (or other objects), in which case they should be reference counted and datablock deletion should decriment the reference.

I disagree on calling this a bug -- at least, I don't consider the lack of an explicit freeing of a texture in the datablock, if it's handled by the TextureHandle's destructor. I checked and what Bryan said is true, unlock() is called on TextureManager's destruction. Therefore, the resource object gets properly dereferenced, meaning that a call to clearTextureHolds()/purgeResources() will unload it. It might make the code a little less readable, but isn't that the point of OOP? Let the children handle all the messy cleanup? ;)

TextureHandles are just class wrapper to texture upload management code. You create one when you want to reference an actual physical texture; The resource class which it is built upon will then do the nitty gritty of loading the file, parsing it, etc. The TH will also handle the part of sending it off to the GPU, binding it to a GL handle, cleanup, etc. (Of course, depending on the type of texture handle you request, that is. The bug that Melv mentioned in 1.0.2 is that the image datablocks create persistant textures, which are kept around even after they have the necessary convolutions applied and uploaded to the card.)

Smaug, you are a smart guy. Judging from your posts, smarter than me, in fact. But from what I have seen on here, you tend to let yourself get mired in arguing over trivialities. Let's talk about texture management, not whether or not so-n-so is correct or useful.

#23 Smaug this, smaug that... it's his communication style. he's a smart guy yes. if you cant stand his occasional ranting then bigger issues are afoot.

#24 Looks like I missed a little debate here; been a little busy, but you all know that right? ;)

The reason for my choice of using datablocks for handling imagery for T2D was simple. It's the standard that is used in the engine for portable static-data. Add to this that existing TGE owners were already familiar with datablocks and the choice was simple.

My previous statements about datablocks seem to have been taken out of context a little here but thats understandable I guess when someone just wants something a certain way and it seems like I'm saying it can't be done. Perhaps I should've said "yes, delete them as you require"?

First-up, if you don't want an imagemap datablock and you want to indirectly tell the texture-manager that this texture isn't needed now...

myHugeImageMapDatablock.delete();

During the imageMap deletion, the texture-handle object will be destroyed causing it to release its lock on the texture and assuming you're not using the same underlying texture elsewhere, perhaps in another T2D imagemap, your own custom objects or even the standard GUI elements then the engines' Texture-Manager will make a call to "TextureManager::freeTexture()" which (as you can see because you've all got the code) removes the video texture-object and removes the source-bitmap (used for resurrecting textures after a purge). AFAIK, texture-holds are only used for MeshTextures which are not applicable here, they're a hangover from TGE (you can even stop holding by changing the "ENABLE_HOLDING" define). Okay, a single call to delete and it's all gone, nuff said.

The reason I stated (perhaps overstated) caution is that ".delete()" is a pretty brutal call from the scripts when you're dealing with dependant objects such as imagemaps. If you do this and another object is using it then you'll be in trouble. In the latest release, the handling of this stuff has improved but this scenario would still be a bug in your code. That's the price to pay for explicit deletion of dependant objects.

Of course, there's also the issue of doing this across the network but deletion of datablocks is not so different from any other object as long as the deletion occurs on the server and not the clients (assuming it's a common network resource).

I'm not sure I see any problem at all with any of this. Maybe that's a failing in me, the fact that I've not read every word here or I'm just being a little dense?

For additional information on texels uploaded to the graphics hardware, you can define "GATHER_METRICS" which starts T2D (actually TGE) monitoring the upload/removal of textures as well as other info. It also provides a call "dumpStats()" which dumps all the texture-objects.

@Smaug: Although you're probably going to ignore what I'm about to say, I'd recommend that you don't. ;) If you want a resource-block, why not just use SimGroups/simSets? SimGroups allow you to delete all the contained objects (whatever they are) in one go, datablocks, T2D objects, GUI Elements, whatever. I'll also add that the idea of a "mission" was put forward on the simple basis that the "mission" was just a group called "missionGroup". All objects created at mission start-up (or during the mission) were added to this group. When the mission (level) ended, it was a simple case to just delete this group. It's a sound good analogy and completely relevant. It's a real shame that as a T2D owner you feel the need to ignore me and what I've done with T2D, oh well, can't please everyone I guess.

- Melv.

#25 :D

Cool! This makes it easy and straight forward to manage resources. I must have just misunderstood you from an earlier post. Happens with me sometimes :)

Thanks for clarifying Melv!

Still love T2D too, BTW! Please keep up the great work!

#26 Ding!

I had missed this thread entirely. I also came away from the original discussion thinking there was currently no way to explicitly delete the underlying texture data.

Not to open another can of worms, but if an application is continually defining and deleting datablocks, is there a way to stop a game's execution while datablocks are loading? Are texture loads done asynchronously or on the main thread?

#27 @Michael, based on an earlier post by Melv, I am pretty sure everything is loaded & launched from the main thread.

#28 @Bryan: No problem; sometimes being too cautious without providing enough background is a bad idea. I believe there was a datablock session at the IGC; maybe there's a presentation document available somewhere. I'll ask.

@Michael: Everything is currently synchronous. The thing is you shouldn't be deleting datablocks that active objects are using, certainly not in v1.0.2 although it's much safer in v1.1.

Sorry for not be clear everyone,

- Melv.

#29 :)

Sorry for extending this even further.

The datablock delete seems to work fine for me, however, what I'm doing is loading and unloading many datablocks each time I change scene. What this amounts to is a lot of:

datablock fxImageMapDatablock2D(myDatablock) {
...
};
myDatablock.delete();
...
datablock fxImageMapDatablock2D(myDatablock) {
...
};
myDatablock.delete();

etc, reusing the datablock name on each scene load. Now after doing this a good number of times, the following assertion is thrown, followed by the appropriate crash:

AssertFatal(sNextObjectId <= DataBlockObjectIdLast,
               "Exceeded maximum number of data blocks");

Digging, (though not at great length) it looks as if sNextObjectId is simply being incremented every time a datablock is created, yet not necessarily 'returned to the pool' if you will. Am I doing something I really shouldn't be doing?

#30 Yes, kind of! I mentioned it in another thread, but by core design datablocks are really not designed to be deleted and created on the fly so to speak, so the code most probably has never been stretched in this direction.

Nothing says you can't, but it's not a "by design" utilization, so you will need to smooth out the edges.

#31 Hmm... looks like you can just change DataBlockObjectIdBitSize in simBase.h to address this problem. Try 30. That'll fix it by gosh! No need for a resource if that works!

#32 Well, looks like the value of 30 is causing some weird bugs. I'm going to expiriment to find a better number.

#33 This is a brick wall for my project, so I'd definitely like to collaborate with anyone who needs this fix as well.

@bryan I haven't done too much digging in the code, but won't changing the size of that bit field just increase the number of datablocks that need to be loaded before it fails? Having an upper limit on the number of datablocks that can be allocated at once is okay, so I was thinking of inserting an ID generator instead that maintains a list of IDs that have been allocated, and returns the ID to the pool (so to speak) upon datablock deletion.

#34 20 works well. Your SimObjectIDs will start out pretty big though. Oh well.

#35 Micheal, if your Datablock IDs are 20 bits wide, that means you can instantiate and delete about 1,000,000 of them before the engine crashes. It shouldn't be a problem under any circumstances.

#36 Also, I found out that the engine starts to act funky when SimObjectIDs hit 16,000,000. So you'll have 1,000,000 datablocks instantiations available, and 15,000,000 SimObject instantiations available. That seems like a pretty good ratio.

#37 I'm only mentioning cause you never know what people might be doing with datablock allocations. (640k should be enough for anyone!) j/k

But in this case, if that's all good then I calculate that a player roaming through my game in particular would have to do nothing but run from screen to screen as fast as possible for about 30 hours straight without restart before this crash occurs.

Seeing as how I'm not exactly putting world of warcraft together here, that would probably be an acceptable limitation. I'll call it a health and safety feature to detect and limit dangerously extended play.

Thanks, I very much appreciate your experimentation.

:D