Mysql or other database programs

#21 oh oh oh wait... so As long i as I don't distribute the server application I don't have to pay.

And $200 is a lot for me without a job. I can barely afford V12 :P

Oh yeah and text files won't work well as i need multiple servers on a network to access them!

#22 Read this the details are there
www.mysql.com/support/arrangements/policy.html

#23 A tragedy indeed. Dashed are my hopes of converting the world to .txt format :(

#24 Look at MetaKit. I don't have the url handy but it should be easy to find. I'm using to for a non-game project (digital asset management), and think it's great. Small, fast, very portable, and free.

Jason
379

#25 heh i would love it if you got me a URL so i could check it out :P

#26 www.equi4.com/metakit/

#27 Question: Under the GPL wouldn't you beable to get the source code for mySQL and change it to suite your needs maybe take out features you won't need. anyway if you do so an make a CUSTOMIZED version. I think that you would be allowed to call it something different and use it for anything you saw fit. If I'm not mistaken this would constitute a different product. I think at most you would ahve to realease the source code.

Which brins me to another point I seriously doubt that you are gonna get charged for mutilple licenses for each server you sold. you would mostlikly be able to purchase rights to the use the source code or even a one time purchase of the databes it self for a substatialy larger fee.

I don't think something like this is gonna fall into their normal pricing scheme you will most likely have to contact them directly

#28 Dont know if this is something you guys would be interested in with all this mysql talk, but Microsoft's MSDE looks pretty nice and costs nothing to distribute. Do a search on MSDE on the MSDN online site. Here is a small excerpt. Like any Windows database you will be able to access it via ODBC or ADO.
=================================

In summary, the MSDE relational database engine exhibits the following characteristics and features:

Is fully compatible with Microsoft SQL Server 7.0 and SQL Server technologies.


Is freely distributable for solutions built with MSDE for Visual Studio 6.0 or Microsoft Office 2000 Developer.


Runs on Windows

#29 heh, I was reading through thinking "Hey, I wonder if these guys know about MSDE?" till I came to that last post :)

Its a pretty good engine actually, very nice to work with, and if you're developing for a windows platform using VC, well then, it doesn't get any easier than using one of the MS databases.

Not sure about licensing, you might have to pay license fees if you use it in a commercial product, and that would probably be the killer. SQL Server licensing fees are definately not cheap...

#30 SQL Server license may not be cheap (13k - 15k per processor for the full blown Enterprise Edition) but it is worth it and much cheaper than other enterprise databases (Oracle, DB2, etc). I say the price is worth it because you will need a fast, reliable database when building an MMORPG. You can't lose data or be down for any length of time if you want to keep your customers. I would be pretty scared of a free-source database under a high volume, high reliability scenario. Personally, I have used SQL Server Enterprise Edition to build very high volum etravel web sites and it rocks.

#31 >>>
Concurrent users

MSDE is tuned for best optimum performance at five concurrent users or less. Microsoft strongly suggests that the number of concurrent users be five or less in order to enjoy the performance levels available with MSDE.
...
SQL Server 7.0 is capable of supporting hundreds or thousands of concurrent users at the highest levels of enterprise-class performance. If an application requires more than five concurrent users, SQL Server is the best option.
<<<

It's only optimized for 5 concurrent users or less, according to the MS web site. This might not be entirely practical for a MMORPG.

David W. Brown

#32 http://www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=235

Some random thoughts on Database systems are sprinkled in that thread, might be germaine here.

I STILL don't think mySQL has the proven stability and scalability at the scales folks are HOPING for in MMORPG's.

Pretty much leaves you with SQL server or Oracle (Oracle dev IS free, but, once you produce it costs a bit of money).

#33 There sure are a lot of people talking about scalability/stability of mySQL with nearly noone talking about any proof of any of these things. Has anyone tried doing something as simple as downloading it, running a script to put a lot of entries on it and then just testing queries? I would think benchmarking would be a first step when examining the suitability of any database.

Basically I am working on a product (not a game) that uses mySQL as a database backend to hold all of our data. We used to use SQL server. Why switch? Simple, SQL was overkill and it cost too much. We were pretty much just using the database as an intelligent storage solution, and all the transactional capabilities of SQL server were unnecessary. We have yet to see a single problem with it.

When talking about using a database for a MMOG, I am guessing people are thinking of storing character and world information on it. Most of this activity is basic INSERTs and SELECTs. There is no reason why mySQL cannot do this sort of activity for even millions of players or objects.

The place where I would think twice about using mySQL is if you were in a highly transactional environment. i.e. something that involves purchasing, inventory, credit cards, etc. The latest version of mySQL has transactional support but I have not tried that yet and it is fairly new, hence I would not necessasrily endorse it since I don't know.

But for anyone looking for a database to play with, download mySQL and experiment with it. All the SQL queries and scripts are not very hard to move to a different platform, so it is not like you could not change your mind at a later date. Moving from SQL server to mySQL took about 3 days, mostly because we were using some features in SQL server that were not in mySQL. Going the other way would be very simple. There are ODBC drivers for it as well, so there is really nothing too proprietary about it besides your queries.

Last but not least, if something is REALLY bugging you in mySQL or something is behaving weird, you can download the source and look at it. Hell, you can even recompile it to tune it for your specific application.

If someone has some sort of tangible results they want to share, then by all means do so. The only way to prove or disprove performance and scalability is to test it, not talk about it.

#34 I'll admit to knowing little about mySQL, but it sounds like you have some hands on experience with it. Might you know how mySQL deals with fault tolerance (ie. how to recover automatically from a database machine failing)? This is a core issue when discussing reliability and often a good comparison point when talking about database technologies.

There are a couple of ways to deal with "fault tolerance" in the SQL Server world. In the simplest case, you setup two servers running an Active/Passive SQL Server installation combined with Microsoft Clustering Services. The clustering services take care of monitoring the active database machine (lets call the machine SQL1) for failure. If any kind of critical failure is detected on SQL1, clustering services engages the second machine (SQL2) and essentially renames it on the network to look like SQL1. Finally, running SQL Server in Active/Passive mode means that one machine is "live" and taking database requests but is also updating the second machine with a copy of the data. When the fail-over occurs, SQL2 becomes the "live" server and is now taking database requests on the most up to date "back-up" of the original data from SQL1. In other words, very little interuption (if any) on the part of the customer.

It gets more complicated as you exceed the capacity of two servers. Jumping to a 3 (or more) server cluster means you have to start partitioning data across the servers. For example, "world items" on one machine and "player data" on another machine. SQL Server can understand how the data is partitioned and retrieve the data seamlessly, regardless of the server location. In other words, a request for "player data" coming into the "world item" server will be automatically routed over to the correct server.

You can further add complexity to the issue when discussing the actual storage devices required in a high-volume, reliable database architecture. In the simplest case of database server clustering I described above, those two servers were sharing a single RAID drive - in other words a single point of failure. While we were able to provide a seamless cutover between machines if a failure occurred, we still have to account for the hard drive failing. Enter another feature of SQL Server, the ability to support high-end SAN solutions seamlessly.

All of this comes down to reliability. Keeping your database servers up and accurate 24x7 is a real hard problem to solve and definately should be a serious consideration for anyone building a persistent MMO game. SQL Server allows you to start simple and as the need arises, grow into an enterprise-level high volume database solution.

It would be good to understand how the other database technologies deal with these very real issues and how they compare overall. Might anyone have some hands-on database implementation experiences to share?

I would also like to talk a little about handling database requests from 2000+ simultaneous users - but I am out of time for the morning (heh - saved by the bell). Food for thought - How could you possible simulate 2000+ simultaneous users in a lab to validate that a particular database technology can handle the load?

#35 Yes, I use it all the time as a matter of fact.

MY point, is that nobody has done rigorous testing of the product under load (don't know about others' points).

I personally have some anectdotal web evidence of traffic on the mySQL-powered sites I run, but not a "big" picture of it's scalability. Near as I can tell, machine failure could be dicey. I also have some concerns about the mySQL security implementaion.

And just running some scripts isn't a rigorous load test ;) If anyone has that data, please share. Otherwise, unless you want to be a testing lab for a living, you have enough to worry about testing your own product, let alone someone elses :) Ditto with downloading the source, you have enough code to write, why write more code if another product has that ONE feature you need?

I know this is a forum of ALOT of coders, but come on, the answer to everything isn't download the source and rewrite it :) Priorities in business, gotta have them. You're building a game, not a database solution.

More pre-coffee ranting ;)

#36 No idea how to do quoting with this thing so I guess I'll just spit out a bunch of responses:)

First off, my key point I was trying to make is there is no reason to not start with something like mySQL since a lot of the tables and queries should port over without any problems. So people who are just trying to get off the ground can get a database whose eval period won't wear out and does not cost several thousand dollars. If there is another free one, like postgres or something, that is also a valid choice. I am not here to evangelize mySQL.

mySQL in and of itself does not have fault tolerance built into it as far as I know. If the goal is for it to be distributed and fault tolerant out of the box, then it is not the right product. However, I have not seen any product that really is. When it comes time to make something scalable and highyl available, that will cost a LOT of money, both for the hardware and for the people as well as a lot of time. The scale there being that the time it takes you to port your mySQL DB to sql server or oracle being a drop in the bucket compared to all the configuration and testing work that would have to take place.

One thing that is not really being mentioned here is architecture. That really make a huge difference in terms of the kind/power of the database you need. Are there application servers which talk to the DB (probably)? How are they configured, how much do you cache locally, how redundant are they? Are they stateless or stateful with regards to the player? So much can be done in the architecture that will affect performance and reliability I think it overshadows all this talk of the back-end database. Whats the point of having a DB up 24/7 if the rest of your application cannot be as well? The testing question also falls into this category. Are there really 2000 requests coming into the database or is there 10 application servers which each have 10 connections to the database that are running 2000 users at once? Off the top of my head, an architecture where clients go directly to the database is probably a bad idea.

I do not think the answer to everything is to rip open the source. To date, I have yet to even download the mySQL source. But anyone who programs a lot will tell you there will come a time or a bug where they will wish like hell for the source code, as opposed to lodging a problem ticket with Microsoft or Oracle.

As for not worrying about it, if you are talking about being scalable, your number one priority is making a solution and then proving that it is scalable before the first user ever touches it. That requires load testing, which will probably have to be a custom written tool that uses your protocols, as well as engineers sitting in back swapping memory, processors, boxes, OS revisions, etc. to figure out what works the best. All this happens before that first "public beta". There is no solution out of the box that does not take a buttload of work to make it have the reliability we are talking about here. Why do you think those Oracle DBA's get paid so much? If it worked out of the box and did not need worrying about then they would not make that kind of money.

If someone here writes a MMG that is successful, they will need capital to make it happen unless they are independently wealthy. So instead of worrying about that now, why not work on your game, throw mySQL in the back, and if your game should make it big, then tackle these big problems. I do not know about you guys, but I am here to get away from all that stuff and program some cool games:)

That kind of puts us on the same page Edward, since we are here to write games, not DB solutions. Why are people worrying about a 24/7, distributed, highly available, fault tolerant solution? It's fun to talk about(??) I guess, but people should not think that they need to have an Oracle license to get some good work done. Grab mySQL or any other DB they can get for free and start hacking away:)

#37 Hehe, actually, it is fun to talk about :) Or at least, that's what I get paid to talk about, so, I hope it's fun :)

I take your point though. You're right, it's at least a good prototype tool. And having the source is better than not having the source.

And, for the record, I have built scalable and available solutions out of the box with Oracle and SQL. Just need to make sure the datamodel is good (ok, that IS a tall order), and your setup admins know the software ;) And yes, it costs ALOT (but, Oracle dev licenses are free, so that helps).

Here's a suggestion for everyone: Postgres (or however it's supposed to be cased). That has much better security out of the box, and I have heard good things about it's scalability. Arsdigita.com has ported some of their highly vailable/scalable web solutions to it. Maybe that's the compromise, open source answer?

Again, no hard data on it, but, worth a peek :)

#38 Taken from Gamasutra.com a post-mortem of Dark Age of Camelot...

http://www.gamasutra.com/features/20020213/firor_01.htm

5. The joys of open source software and stability. Long ago, during the development of our early titles, we decided to use Linux wherever possible as our server back-end OS, and we kept to this same practice when creating Dark Age of Camelot. We have extensive Linux experience in-house, and it made sense for us to stay with a platform that we knew could handle the task and also was, well, free.

Because running Camelot would require a considerable amount of data management, we initially planned on using Oracle to store account and character information. However, Oracle's quoted license fee of more than $900,000 quickly removed them from contention. Once we got over our shock and amusement at Oracle's pricing, we turned to a Linux-based freeware solution, MySQL, to manage Camelot's data storage, which so far has worked admirably.

Everyone developing games should at least investigate open source solutions for their servers. It's saved us a pile of money and has been stable and reliable. In fact, prior to Camelot's launch, it was axiomatic that MMORPGs were unstable and prone to crashing during their first month or so. From the outset, we were determined to buck this trend. We co-located our servers directly at UUNET, on the network backbone, which ensured a wide network pipe to the Internet. With this Internet connection, we can increase our band-
width with just a few hours' notice to UUNET.

With the combination of reliable server code and a stable Internet connection- all running on open source software - Camelot went live on October 9, 2001, with virtually no problems. That first night, the game went down for about an hour and a half due to a database configuration problem, but since then, the game has been remarkably solid and stable. As of this writing, it hasn't been down due to server error for more than a few minutes ever since the first night.

-John

#39 You guys might also want to take a look at my code for adding ODBC support to Torque script. That way, you can develop on basically any database, and should be able to switch over if something better comes along.

Resource is here, though for some reason the download is broken. You can find my code here

Josh

#40 That's really cool, Josh (and a testament to the elegance of the respective technologies (ODBC and tscript)).

I want to put in my two cents for using linux as your server environment. Unlike some here, I don't think that linux is really great for client side software (not technology wise, necessarily, but it can be overkill... And if you're developing client software, you want to make sure you have lots of clients. There aren't that many linux clients out there... There are lots of Windows clients and more Mac clients and some linux clients.).

But for serverside stuff, Linux (or ANY Posix-based OS) is an excellent choice. They've been doing server side stuff happily for the past twenty years (and doing big server side stuff, too); chances are they'll work just fine for your server, too.

I think that the directness of the environment is good for servers, too. You can write a multiprocessing server in pretty short order - say, with a few pages of code. Templates are available all over, too. You have to deal with signals, and that's about it. If you _want_ to, you can have a console text interface. You don't have to, and many posix server apps don't. They just have an admin port - which preserves the model.

I don't trust windows for server stuff because it can't handle client stuff really reliably. (I use windows a lot, sure, but I'd never trust it with anything important.) I don't trust macs for server stuff because they've had both a history of being not really good for it (see the menu bug; I know it's fixed, anyway...), and because, like Windows, they have a GUI.

Neither OS was designed for server use. Windows evolved from a GUI overlay; NT added posix support. MacOS evolved from an attempt to make the computer friendly; OSX aded posix support.

So... You _can_ use 'em for cross-platform server stuff now without much effort, but they both lack the kind of pure stress testing and maturity that *nix has.

If I want to have something really have 99.999...% uptime, then I know I'm going to have to shell out money. You don't get uptime like that (you don't count on it, anyway; you might get lucky) without quality redundant hardware and quality redundant software and a boatload of quality redundant administrators.

An MMORPG isn't medical software; you get what you put in, and most (all?) MMORPG developers don't put in the effort to acheive that kind of reliabilit; it shows. That's ok. It's just a game (and as MMORPGs become more of a commodity, they'll get more reliable).

Look at the web - it's pretty reliable (though not what I'd call trustworthy), mostly because it takes so little effort nowadays to set up a site; almost anyone can do it, 'specially if they have a little money.

Well, it's late; I'm tired.

Summary:

If you want a fault tolerant solution, you _will_ spend money - if not on software, then on admins and hardware.

MMORPGs don't really need "fault tolerance" - they just need to be mostly up, and they need to not choke at peak hours.

*NIX is good at server stuff; I'd recomend you use it (and if you don't like mySQL, try Postgres, and if you don't like that, go buy oracle. You really do get what you pay for. Oracle is like some sort of god... and you get to pay for it.)

Ultimately, though - do what works. Results are way better than using the One True OS.