Torque DB (possibly moving to C++)

#1 I haven't looked at this closely so bear that in mind, but i'm not sure how viable a product that would be. I mean what makes this better than using an existing DB?

#2 It's quite a lot of work to not only provide a database connection, but also provide actual tables and queries that are specific to stock T2D/TGE.

Adding in a database connection to a TAP application is a matter of hours--actually persisting valid and useful game data, from the design all the way down to performance tweaks is months of work--so I would think that it could be useful for some as a product!

I also think that your support of that product would completely and totally swamp you, and with paying customers, you'll have to support it...something to keep in mind.

#3 I just don't understand what you implementation King_Bob would give me over say something like this.

#4 @John: I definately think there would be a use, if I could create enough custom integration into TAP, though that is what I'm brainstorming now

@Stephen: very true... when I first thought of it It overwhelmed beyond thinking on it any further lol... but I'm slowly coming to grips and liking the idea...

my goal would be to create a data layer for Torque...

an in game database that then could output to other database (like MySQL and SQLite) for compatability, along with its own encrypted format

Basically not only simplifying the process of adding data base integration (as you mentioned of saving off all the correct data at the right time) while also elminating the TAP database integration need, while giving developers an in game database to store any type of data...

then agian I'm still also thinking of ways i could make it specifically viable to TAP users ... maybe some sort of direct datablock integration...

Though your warnings are very valid and welcome, it would be a very massive project, though I am completely confident that If I decided to I could do it.

#5 Feel free to be brutal, this is the point where I'm trying to decide if its worth my time to start on something like this :) both of your comments are very much appreciated and welcome

@John: I would also integrate it directly with other systems, like a journal system, maybe some other data structure based systems as well

#6 What I really would like to do is find ways to tweak it to accepting TAP specific data types (maybe even down to T2D, TGE, and TSE) specfic data types... for efficiency as well as maybe added features...

right now my mind has just stopped at that point, I know there definately must be some functionality (probalby major functionality) for creating this, maybe duplicate options that TAP doesn't naturally have...

again just calling out for opinions, thoughts, concerns etc... did this on T2D since I would start with T2D because of its simplicity compared to the others

#7 @John: ALL that resource does is to allow you to connect to a completely unpopulated (and unstructured) database. You still must build each and every table, every query, and the save/load functions to -use- that database connection. My project implemented that resource 8 months ago--took us about a month to get it to handle very basic user authentication. We took one look at how much work it would be to just be able to persist RTS units for example, and started searching high and low for someone with actual database design and implementation experience. Fortunately, we found one, and it's "only" taken him and us about a month and a half of design work, and 3 weeks of implementation to get basic persistence capability.

It's a hell of a lot more work than it appears to persist data in a good way unless you are extremely well-educated in database design and structure.

@Matthew: The only big problem I can see is that you can't make a generic database (and I mean tables, relationships, key architecture, and queries) that will work for anyone that does something so simple as add "fatigue" instead of energy to their game mechanic. Personally, I'd love to have had something to work from, but I'd also cringe at thinking about how to design it well enough to keep it generic--but then again, I'm seriously not a database design guy.

#8 @Stephen
Yeah all that resource does is give you a DB connection, but I can't imagine how any application would be able to write my database for me. Theres a reason people make good money as DBAs :p

#9 Hmm... yeah, I know what your meaning about making it generic...

maybe if I could classify all the different type of needs down to different types and different places needed to call, then make a visual tool to add custom versions of these types... like persisting attributes, NPC relation tables, etc etc...


yeah structuring your data in game and in the database end is a huge feat (depending on the scale and style of game), I can see how a lot of less resourcefull (than Stephen Zepp's) groups would shortcut the style and structure to just later find it inadequate and their game falling short because of it

#10 True John... well maybe I can make a solution for this... the Indie's automated DBA lol... a complete visual tool based off of TAP

or even if not quite that maybe I can design structurs for different styles of games... like an RPG structure, a RTS structure, etc etc... could be a DB Content pack per se (with examples)... again just throwing ideas out there

#11 I think that if you shoot for a way to persist, say, the stock 1.3 demo mission (stronghold) or something along those lines, you can focus on the underlying things that will be needed for pretty much any game (there are a subset of things that need to be persisted pretty much for any of the major genres, like location, orientation, status (health, etc.)), and focus heavily on good example queries that demonstrate the concepts within the TAP context you'd have quite a nice instruction pack.

I wouldn't market it as a starter kit myself, but more of an instructional pack that demonstrates the db side of things, as opposed to just allowing the connectivity itself. I'd think that could be quite useful for some, but on the other hand, it's really only large multiplayer games that actually need this level of functionality--single player games can pretty much just serialize their data to a flat file and be done with it.

Really neat idea, don't get me wrong, but I'm not sure it would be a "product for the masses" either.

#12 Very true, you got a good point, I'd be branching out to more of a specialized product vs an all around product...

not that I'm in it for the money (if I were I'd just set up good examples and contract out)... though would definately be best to target it towards overall use vs specialized use...

maybe C++ end code that does what my current Torque DB does but more efficient and more features... something to keep basic game data, with a fully visual tool to edit and view the data dynamically, along with examples like the Journal System, maybe an Inventory System, Dialog System, etc etc... but more of an in game database (with some easier exporting tools for easier access to mysql and sqlite) vs competition with sqlite and mysql

could create my own GUI controls to view and edit the data in game as well...

#13 I have been watching TorqueDB for quite awhile, as a dynamic content webmaster for the Air Force (meaning I play with PHP, ASP, HTML, Javascript, MySQL, and MS SQL databases all day long) it really caught my attention. As for how I would use it: my plan at the moment is to develop a Zelda-ish MMORPG ( which would be released and playable, absolutely free) and I could really, really, use a native database for this kind of work - instead of hacking in MySQL support.

Maybe I'm just not really understanding, but what do you mean by making it generic? Do you plan to include pre-built table definitions that non-experienced database users could use rather than designing their own tables? If so, I'm sure you plan to include just a "blank" design, correct? One where people like myself don't even have to worry about the pre-built definitions and can just go designing our own tables?

Also, how are you doing with the datatypes? I can see how a TorqeScript perpspective would lead someone to only think of varchar, int, text, float, etc. datatypes. Will there be datetime, enum? What about primary keys, auto_incremement, and other various functions (such as md5 and password)? Will datatype expressions be as powerful in TorqueDB as they are in other databases (ie. select * from users where (date_one > date_two) limit 10)?

I just went rambling on about some database stuff there, maybe it added some ideas into your battle plan, maybe you've already taken them into account. We'll see :)

All I can ask is don't dumb it down to much for non-database experienced users. Personally, for me to purchase, implement, and use a native database - I would at least have to have everything mentioned in my rant above with completely open access to design tables as I see fit (not being restricted to your prebuilt definitions).

#14

Quote:Also, how are you doing with the datatypes? I can see how a TorqeScript perpspective would lead someone to only think of varchar, int, text, float, etc. datatypes. Will there be datetime, enum? What about primary keys, auto_incremement, and other various functions (such as md5 and password)? Will datatype expressions be as powerful in TorqueDB as they are in other databases (ie. select * from users where (date_one > date_two) limit 10)?

Some very good input and questions

as for datatypes I would support the basic "varchar, int, text, float, blob, etc etc."... along with possibly, datablocks, filepaths, and other Torque specific entries that could be useful... for example like i did with the script Torque DB... you can set an image value and it will respond differently... int he journal system it will grab that value and not display it but display the image the path points... I could include some specific types like that... imagePath filePath etc, that will send back values when accessed so the user can then use that data accordingly...

would definately support auto_increment (in fact by default the torquescript DB really supports auto_increment, just the way it is set up naturally - at least to a point)... I would include a full feature set of MySQL and SQLite paired with game features that could be useful for specialized responses and tracking

the select question is very good as well... I would definately include that, as well as a way to customize your own select specifications (based on data types)... maybe a select statement creator tool...

I would definatley include all of this as blank options, with full functionality and configurability ... Im a completel beleiver in giving the user control of everything if they want it...

definately wouldnt require a dumbed down version of use

btw keep firing those questions if you want :) it really helps

#15 Hmm, the more I think on it maybe I could develop a data layer in between the databases and game data to store data with all the features mentioned, then export it to a format that can be entered in other databases (as well as an encrypted local format) that can be read back...

#16 Very cool - I can't think of anything more at the moment, other than explaining my intended usage ( which I will do below ) but if I do think of anything, I'll be sure to let you know.

How I intend to use TorqueDB:
I had a longer description all layed out, but I think I will do it in a .plan, I'll just explan my TorqueDB usage here. Our intentions is to implement TorqueDB into our server(s). If the game does get large enough for multiple servers we intend to do cross-referencing of the databases, so the user is only aware of one server. We're also looking into inter-server communication so these users would essentially be playing in the same gameworld, just across different servers.

The TorqueDB would maintain all the information that isn't "safe" to be stored client-side. The player's current location, character information, inventory. The TroqueDB would also maintain all enemy, item, quest information - so the addition of new content is limited to the user downloading the new media only.

We intend to implement a streaming/cache process for retrieving information from the database. When a player enters an area it begins streaming all of the information for that area (enemies, items found, quests available, etc) and maintains this list within a cache. This cache will be cleared at the end/beginning of every session, so the user will never miss out on new content because they have a cached copy of the information for that area.

#17 I forgot to also note: looking forward to the inter-database features. I would love to offer player's the ability to view their characters on our website, plan out skill trees, etc. So, we would be exporting the TorqueDB to a MySQL database to do things such as that.

#18 Wow... great description, that really helps, a full practical example.

Now a question, would you deem it as a product worth 'paying' for if it were fully optimized in C++ with the feature set mentioned (tool sets, editing tools - most likely done in TAP, like Showtool, including php scripts for web display of the MySQL formats, etc ) ?

#19 Oh definitely! I'm thinking the $20-$30 price range. My only concern would be implementing it with T2D/TGE (my planned implementation is for T2D, but I'm a TGE licensee as well and can see uses for it there as well). As stated earlier I'm a webmaster, and C++ completely blows my mind. Although I do intend to learn C++ eventually, my focus at the moment is on TorqueScript only.

It sounds like you have a lot of great features planned, although many of them I personally would not use very much. Editing tools will only be used to develop the initial database layout, we will then develop in-house tools to easily alter those databases (simply because the database will be located seperately than where the client will be altering the database. We also intend to release Gamemaster clients to certain players who are willing to assist us in "ruling" the game).

Not to sure what you mean about the PHP scripts, but if you need any help with that let me know. I have 7 years of experience with PHP and plenty of references. :)

#20 I guess I may have been seeing this slightly differently, but consider this as an alternative scenario:

--step them through connecting an existing database (MySQL, etc.) to TGE.
--step them through example table designs for persisting certain aspects of the starter.fps/rts/whatever.
--step them through use cases on queries that your tables provide for game mechanics implementation guides.

I really don't think you need/want to build a database from scratch (as you did with the journal system)--that really is going to much in the direction of re-inventing the wheel.

Just my personal opinion!

EDIT: changed "real" to "existing" to better reflect what I meant!