Firing up the MMOG pack - Part 1
by David Dougher · in Torque Game Engine · 01/03/2005 (7:28 am) · 24 replies
OK. Things on Adellion have calmed for a bit and in the lull I thought I would turn my attention to the MMOG Kit that I was discussing before. I have at least a tentative go ahead from HonourBound Ltd. to share some of the tech we have done, but their request is that we build from a scratch 1.3 base to avoid any possible license complications. (Adellion uses just about every Pack GG, Bravetee, Synapse and Tim Aste made.)
So initially we will have to roll back to the original gorpe code and bring it up to 1.3 standards. We will also need to isolate features that some developers may or may not want. For example, Gorpe includes support for both python (Thanks Josh Ritter) and XML. It had an optional publisher and subscriber system as well to speed responses.
So, my thought would be to take each major piece and turn it into an add on that the developer can choose to combine with a basic 1.3 system or not.
This would include the terrain paging system, since it is possible to make a perfectly good zone based MMOG system without the terrain paging system. A hybrid zoned multiterrain system might also be desireable for many people.
The first thing to realize is that such a system will involve a lot more code than the current game has. MOst of this code will actually be repeated code - taken from the original engine and reused to make a server/service with a specific function.
Adellion was created by analyzing the way that information flows through an MMOG and then dividing it into those operations which must be realtime and those for which realtime response is not required.
For example, it is not critical when sorting through your inventory to make it a real-time operation. It must happen, it must be a reliable transaction, but it does not need to occur in real-time the same way that say, a jump across a chasm needs to work.
Some game operations need not even be part of the "game" at all. For example, character creation does not need to occur on the game server. The character can be created and stored to the database. When the player starts the game the newly created character is retrieved from the database and introduced to the game world.
Furthermore, the volume of transactions for some MMO operations change over time. When you first open your game world much more time will be spent by players in Character Creation and playing the game than in Guild Chat or Inventory. As the game world gets rolling people spend more time considering the objects they own (or would like to possess or make) and conversing with guild members. Also, consider the volume of data that moves when you announce your latest monthly update. There is a significant uptick on the load that would be placed on an update server.
So initially we will have to roll back to the original gorpe code and bring it up to 1.3 standards. We will also need to isolate features that some developers may or may not want. For example, Gorpe includes support for both python (Thanks Josh Ritter) and XML. It had an optional publisher and subscriber system as well to speed responses.
So, my thought would be to take each major piece and turn it into an add on that the developer can choose to combine with a basic 1.3 system or not.
This would include the terrain paging system, since it is possible to make a perfectly good zone based MMOG system without the terrain paging system. A hybrid zoned multiterrain system might also be desireable for many people.
The first thing to realize is that such a system will involve a lot more code than the current game has. MOst of this code will actually be repeated code - taken from the original engine and reused to make a server/service with a specific function.
Adellion was created by analyzing the way that information flows through an MMOG and then dividing it into those operations which must be realtime and those for which realtime response is not required.
For example, it is not critical when sorting through your inventory to make it a real-time operation. It must happen, it must be a reliable transaction, but it does not need to occur in real-time the same way that say, a jump across a chasm needs to work.
Some game operations need not even be part of the "game" at all. For example, character creation does not need to occur on the game server. The character can be created and stored to the database. When the player starts the game the newly created character is retrieved from the database and introduced to the game world.
Furthermore, the volume of transactions for some MMO operations change over time. When you first open your game world much more time will be spent by players in Character Creation and playing the game than in Guild Chat or Inventory. As the game world gets rolling people spend more time considering the objects they own (or would like to possess or make) and conversing with guild members. Also, consider the volume of data that moves when you announce your latest monthly update. There is a significant uptick on the load that would be placed on an update server.
About the author
Owner - Pariah Games, Adjunct Professor - Bristol Community College, Mentor - Game Design - Met School Newport, Mentor - Game Design - Met School Providence
#2
Clients---
Customer Client - this is the one your customers get.
Editor Client - This is the one used by your team to make changes to your next version. This is closest to what you see in a typical torque download now.
Support Client - This is a client for support people to get your customer's unstuck, check up on reports of griefing, and slap naughtly people with a "time-out".
Security Client - This is a client to handle serious security issues like racist harrassment, customer ban for foul and abusive language to other customers or staff, theft of customer property (that is not in game sanctioned.) etc. The security client is generally given access to the database in some form to allow for direct manipulation of customer records in cases of theft (or apology).
Dedicated Servers --
Master Server - handles connections to the game. For some testing you can use the GG master server, but plan on your own. If it is remaining internal to your site you might consider modifying OpenTNL and contributing your changes. Or you can buy a license - GG would like that too...
Database Server - A dedicated mySQL server is recommended with a separate internal network to connect it to the other necessary systems - taking it off the Internet to increase security is HIGHLY recommended.
Accounting Server - the only thing we need here is to document what entry record the Accounting server needs to place on the database to allow a new customer to enter the game. (If you have a free MMOG then you don't need this.) A dedicated server is usually provided by your publisher and it has a separate network card to access your Database. The database allows only one kind of record in the file to be accessed and added/updated by the Accounting Server - a new customer record. (optionally might have a ban account flag). A new customer record does NOT carry accounting information like credit card info. Always keep your accounting data away from your game -- preferably in a separate country!
Now just some thoughts... No committment here.
With all that said... I would like to recruit volunteers to help with the creation of the MMOG Pack - specifically people to help with the coding, debugging, documentation, and some artists and level designers to help make a cool example game world. This is NOT going to be Adellion. It is going to be a general purpose pack. I have access to a lot of stuff to speed the development but it will need a development cycle to assure that it is ready for primetime and many of the pieces will need special testing - for example, we are not using Mac Servers for Adellion.
My original thought was to publish different pieces for free as tutorials or useful add-ons (we will be borrowing heavily from the resources so it seems fair to put back a few new ones.) and to sell the entire Pack for a reasonable price when complete. I doubt anybody will get rich from this and I expect that HonourBound will want a cut since they will be providing a big chunk of code, my time, and probably some of our people as well at least on a short term basis.
I would also like to have at least a couple manuals on how to integrate MMOG code with things like the Synapse Lighting Pack. So if you buy that pack from GG and we can prove you own it we can give you some guidance on how to integrate it.
If GG is interested we would sell through them - probably on CD with a manual. There is too much code for a download and it might even require a separate CVS to make updates possible.
This is NOT a trivial project, even with everything I already have to help get it started. All people who seriously assist on the project would get a free license and hopefully a piece of the pie from sales - but you should NOT think you will get rich from this.
Feedback is welcome - either here on this forum, or you can e-mail me at
ddougher@pariahgames DOT com
01/03/2005 (7:29 am)
Part 3Clients---
Customer Client - this is the one your customers get.
Editor Client - This is the one used by your team to make changes to your next version. This is closest to what you see in a typical torque download now.
Support Client - This is a client for support people to get your customer's unstuck, check up on reports of griefing, and slap naughtly people with a "time-out".
Security Client - This is a client to handle serious security issues like racist harrassment, customer ban for foul and abusive language to other customers or staff, theft of customer property (that is not in game sanctioned.) etc. The security client is generally given access to the database in some form to allow for direct manipulation of customer records in cases of theft (or apology).
Dedicated Servers --
Master Server - handles connections to the game. For some testing you can use the GG master server, but plan on your own. If it is remaining internal to your site you might consider modifying OpenTNL and contributing your changes. Or you can buy a license - GG would like that too...
Database Server - A dedicated mySQL server is recommended with a separate internal network to connect it to the other necessary systems - taking it off the Internet to increase security is HIGHLY recommended.
Accounting Server - the only thing we need here is to document what entry record the Accounting server needs to place on the database to allow a new customer to enter the game. (If you have a free MMOG then you don't need this.) A dedicated server is usually provided by your publisher and it has a separate network card to access your Database. The database allows only one kind of record in the file to be accessed and added/updated by the Accounting Server - a new customer record. (optionally might have a ban account flag). A new customer record does NOT carry accounting information like credit card info. Always keep your accounting data away from your game -- preferably in a separate country!
Now just some thoughts... No committment here.
With all that said... I would like to recruit volunteers to help with the creation of the MMOG Pack - specifically people to help with the coding, debugging, documentation, and some artists and level designers to help make a cool example game world. This is NOT going to be Adellion. It is going to be a general purpose pack. I have access to a lot of stuff to speed the development but it will need a development cycle to assure that it is ready for primetime and many of the pieces will need special testing - for example, we are not using Mac Servers for Adellion.
My original thought was to publish different pieces for free as tutorials or useful add-ons (we will be borrowing heavily from the resources so it seems fair to put back a few new ones.) and to sell the entire Pack for a reasonable price when complete. I doubt anybody will get rich from this and I expect that HonourBound will want a cut since they will be providing a big chunk of code, my time, and probably some of our people as well at least on a short term basis.
I would also like to have at least a couple manuals on how to integrate MMOG code with things like the Synapse Lighting Pack. So if you buy that pack from GG and we can prove you own it we can give you some guidance on how to integrate it.
If GG is interested we would sell through them - probably on CD with a manual. There is too much code for a download and it might even require a separate CVS to make updates possible.
This is NOT a trivial project, even with everything I already have to help get it started. All people who seriously assist on the project would get a free license and hopefully a piece of the pie from sales - but you should NOT think you will get rich from this.
Feedback is welcome - either here on this forum, or you can e-mail me at
ddougher@pariahgames DOT com
#3
Makes sense for me to be responsible for the terrain deformation aspect (if any) since I'm working on the community resource now.
01/03/2005 (7:38 am)
You know my current tasking within the MMOG kit--terrain tiling/paging!Makes sense for me to be responsible for the terrain deformation aspect (if any) since I'm working on the community resource now.
#4
well I'm definately up for assisting in any way I could.
Would assist with coding, though due to my unfamiliarity with the project I would help however needed, documentation, debugging, level design etc.
This will be a great advancement for Torque
EDIT: @Stephen - great terrain modifications!
01/03/2005 (8:01 am)
Sounds great...well I'm definately up for assisting in any way I could.
Would assist with coding, though due to my unfamiliarity with the project I would help however needed, documentation, debugging, level design etc.
This will be a great advancement for Torque
EDIT: @Stephen - great terrain modifications!
#5
Very interested in seeing this project work out, so count on my support in helping to make it happen!
Drop me a line with a rough idea of what you are looking for in terms of art requirements. (lord_cronson@yahoo.com)
01/10/2005 (8:02 pm)
Well, I just barely know enough about programming to be dangerous. ;) But I'd be glad to help out in the artwork and level design departments.Very interested in seeing this project work out, so count on my support in helping to make it happen!
Drop me a line with a rough idea of what you are looking for in terms of art requirements. (lord_cronson@yahoo.com)
#7
I'd like to present a consideration concerning terrain paging and performance. I'm not a programming guru but I do understand an important aspect of game programming and that is to have a game run as smoothly as possible you have it do as little as you can possibly make it do. I think it was Andre La Mothe who said that.
Anyway, my understanding with torque, or just about any game engine for that matter, when the game unloads a map, it also unloads all the resources (map objects, models, etc) And then when it loads up the next terrain, it loads into RAM all the new map objects. It does this even if some or many of the same map objects were in the previous terrain. Now then, I assume you are going to make necessary changes to the editor to accomodate for editing each terrain. Say, upon creating a new world the person editing it would specify the size of the world, for example: 256x96 terrains. What I'm suggesting here is that whenever a map object is put into a terrain, it is recorded into an array, a lookup table of sorts. Then, when the game is running, when a new terrain is going to be loaded into RAM, the game first looks at the array and compares the current one with the new one (or whatever one is about to be unloaded from RAM). Suppose building A, B and C are in the terrain thats about to get dumped. And in the new terrain, building A, B but NOT C is present. Instead of unloading A & B from RAM, it'd just drop building C. Sure, there's a little overhead to make the comparrison, how bad can it be? A comparrison like this would take less time that it would to reload numerous objects from disk. The more disk operation you can cut out the better the overall performance will be.
Just my $0.02 worth. :)
01/24/2005 (10:10 pm)
Please contact me about any assistance in this project, I'd appreciate it.I'd like to present a consideration concerning terrain paging and performance. I'm not a programming guru but I do understand an important aspect of game programming and that is to have a game run as smoothly as possible you have it do as little as you can possibly make it do. I think it was Andre La Mothe who said that.
Anyway, my understanding with torque, or just about any game engine for that matter, when the game unloads a map, it also unloads all the resources (map objects, models, etc) And then when it loads up the next terrain, it loads into RAM all the new map objects. It does this even if some or many of the same map objects were in the previous terrain. Now then, I assume you are going to make necessary changes to the editor to accomodate for editing each terrain. Say, upon creating a new world the person editing it would specify the size of the world, for example: 256x96 terrains. What I'm suggesting here is that whenever a map object is put into a terrain, it is recorded into an array, a lookup table of sorts. Then, when the game is running, when a new terrain is going to be loaded into RAM, the game first looks at the array and compares the current one with the new one (or whatever one is about to be unloaded from RAM). Suppose building A, B and C are in the terrain thats about to get dumped. And in the new terrain, building A, B but NOT C is present. Instead of unloading A & B from RAM, it'd just drop building C. Sure, there's a little overhead to make the comparrison, how bad can it be? A comparrison like this would take less time that it would to reload numerous objects from disk. The more disk operation you can cut out the better the overall performance will be.
Just my $0.02 worth. :)
#8
Let's talk to you about it.
The bottleneck in MMOG, in my opinion, this quantity of clients on one card and speed of loading on client side of this card, especially if on it is a lot of objects.
01/24/2005 (11:30 pm)
Don Bates - Let's talk to you about it.
The bottleneck in MMOG, in my opinion, this quantity of clients on one card and speed of loading on client side of this card, especially if on it is a lot of objects.
#9
01/26/2005 (2:52 pm)
I would love to help out in any way I can.
#10
02/02/2005 (10:05 pm)
I'd like to throw my hat in on this as well..
#11
I am especially interested in getting a couple of level designers for this project, but if you have specific talents that
you think would be useful to making this happen, now is the time to speak up. Over the next two weeks I will be developing a preliminary schedule, establishing communication channels, and securing the repository for our efforts.
Persistence, skill and talent are essential. Be prepared for fits and jumps, starts, stops and backups. Tons to do, and only the self- motivated need apply! :)
02/09/2005 (7:59 am)
People interested in working on this should contact me via the email in my profile. Several of you already have. I am especially interested in getting a couple of level designers for this project, but if you have specific talents that
you think would be useful to making this happen, now is the time to speak up. Over the next two weeks I will be developing a preliminary schedule, establishing communication channels, and securing the repository for our efforts.
Persistence, skill and talent are essential. Be prepared for fits and jumps, starts, stops and backups. Tons to do, and only the self- motivated need apply! :)
#12
Like most I have been waiting to see if TSE will solve my planet-level sized missions in torque. But it hasnt stopped me from digging up good reading on Server MMOG tech.
But still if a Terrain-Paging Module is first up for the 1.3 then I will grab it. Even without the Server services.
02/22/2005 (3:37 pm)
I got to remember to check back for this heh. Like most I have been waiting to see if TSE will solve my planet-level sized missions in torque. But it hasnt stopped me from digging up good reading on Server MMOG tech.
But still if a Terrain-Paging Module is first up for the 1.3 then I will grab it. Even without the Server services.
#13
03/03/2005 (4:38 pm)
Same here, I am waiting with baited breath for a tutorial or resource on terrain-paging.
#14
03/16/2005 (11:55 am)
I am willing to provide a use of my dedicated server to host this projects web or services. 
#15
One of my goals that is not unique to my project is cross-platform support for the server in terms of multiple databases. ie: use SQLite and a remote MySQL/ODBC at the same time... from Lin/Win/Mac (I'll need help with the mac end of things though I don't anticipate too many issues with OSX being unix based) use the right tool for the right job kind of dealy.
I'm endeavoring to use SQLite for the localized database connection and having a second database connection to a full remote server using SSL (MySQL is my primary target as it's cross platform already but ODBC support is planned). This setup would eliminate the need for shared filesystems between servers in an multi-server based world while also offloading unnecessary network traffic. As an example, say you have two servers, each hosting a different "continent" of the same world, you'd want to use the local DB for things like what's where in each area (or some such mundane detail) whereas you'd want player/creature data stored on the remote DB.
Something else I was thinking for the long-term future as an interesting project to talk about... what about defining areas/maps/world completely in the DB? ie: take all the data stored in the .ter and .mis files and work out a proper DB solution. Obviously this seems to be a monumental task but would be a really really cool and incredibly torque-unique feature IMHO.
03/16/2005 (9:57 pm)
I'll offer my services as a programmer to this project. I'm currently working on a very similar framework and am willing to contribute common features (ie: stuff that's not specific to my project).One of my goals that is not unique to my project is cross-platform support for the server in terms of multiple databases. ie: use SQLite and a remote MySQL/ODBC at the same time... from Lin/Win/Mac (I'll need help with the mac end of things though I don't anticipate too many issues with OSX being unix based) use the right tool for the right job kind of dealy.
I'm endeavoring to use SQLite for the localized database connection and having a second database connection to a full remote server using SSL (MySQL is my primary target as it's cross platform already but ODBC support is planned). This setup would eliminate the need for shared filesystems between servers in an multi-server based world while also offloading unnecessary network traffic. As an example, say you have two servers, each hosting a different "continent" of the same world, you'd want to use the local DB for things like what's where in each area (or some such mundane detail) whereas you'd want player/creature data stored on the remote DB.
Something else I was thinking for the long-term future as an interesting project to talk about... what about defining areas/maps/world completely in the DB? ie: take all the data stored in the .ter and .mis files and work out a proper DB solution. Obviously this seems to be a monumental task but would be a really really cool and incredibly torque-unique feature IMHO.
#17
11/22/2005 (11:25 am)
The thread is old but the topic has not died. GG has been very, very busy lately so little progress has been made at this point. Hopefully it will pick up again soon. 
#18
12/30/2005 (2:55 pm)
David, any news on the mmog pack? Are you still working on it? 
#19
01/28/2006 (11:05 pm)
Hi I and my small team of developers are just joining the ranks of the Torque engine after a long battle with 3D game studio by Conetic anyway we are going to continue the prodject we started and build the next Gen. of MMORPG and try to improve on the current level of good emertion in a 3d inviroment...and please dont ask for that secret because its the hook and the premice of our game and a guarded secret...however when we create stuff using the Toque Game Engine, and able to give it to the public with out violating and secrets or LAWS,...and that one i emphezize that one,.....but anything that we can do to help the comunity of gamers we will try. Bobby@Imagn' Games ....So let us know if you need help with that prodject 
Torque Owner David Dougher
Pariah Games
To accommodate this shifting demand Adellion isolated the services provided to the player and allowed them to be placed onto one or more servers. In addition, the services can be present on a single server more than one time. So a large multiprocessor server might handle a number of individual update services when the monthly release comes due, but it might reduce the number of update services and add Character Creation service duties at other times.
The emphasis here is on server services and making them adjustable to the needs of the game.
Also, since the torque engine is the basis for our game, I am for the most part planning on making at least one complete example system based on that code base with no additions - that means I won't be using python or XML or Visual Basic, or any large 3rd party package for additional support.
That means the initial focus for development will be on the server services to run the game - the infrastructure of the MMO built in C++ and the torque scripting language as much as possible; dedicated clients for PC's, Mac's and Linux; dedicated server services for all three OS's; and dedicated security and editor clients as well.
I will be using mySQL for the database and establishing connections using ODBC. There is no charge for developing using MySQL and a commercial license is only 100.00 when you are ready to launch. (If it's good enough for DaoC it is fine for me.)
Here is a tenative list of the pieces to be created. (Note: These services should be created for all three platforms, except where indicated.)
Services---
Update Services - Used to update the client to the latest version before they enter the game.
Login Services - Verifies the player is a registered member in good standing (again if your game is free this is unnecessary, unless you ban players. In that case the login server should do the check to keep grouchy players from hammering the game server with a DOS attack to try and bring it down.)
Game Services - Handles all player movement, and action requests in a given area of the game. This could be requests for inventory, to actions like swimming, jumping or combat. These are generally dedicated servers since adding other services to them undoes the idea of reducing load responsiveness and opens the door for crashes when other services are compromised.
Character Creation Services - Used to create a new player character and make him available to the game server.
Inventory Services - Used to handle the items currently in the possession of the player or stored elsewhere for him/her to retrieve (i.e. banks).
Chat Services - Used to handle conversations between players, guilds, staff, etc.
NPC Services - NPC and AI is handled outside of the game server, allowing the game world to be brought up without NPC's for editing and viewing checks. Allows multiple NPC AI servers to handle traffic and quest issues. Also, if NPC/AI is hosed on one service it doesn't crash the whole game.
It is important to note that for a smaller game it is possible to combine services on one or more machines. That way as your player base grows, you can grow the hardware to match.