Persistant Character Server
by Daniel Neilsen · 04/04/2003 (9:05 am) · 105 comments
Download Code File
Torque Character Server Tutorial
by Daniel Neilsen
DISCLAIMER:
Make sure you back up all your code before doing any of this.
IMPORTANT NOTE: You MUST change this setup for security reasons and to suit your own game. This tutorial is only to give you an indication on how you can do it and shold not be copied verbatim.
This tutorial will show you how to make a remote character server that will integrate with the TGE. This system uses TGE's httpObject to access a php webserver and does NOT directly access any sql or database server for the following reasons:
1) This method requires no username or password to be stored in either the engine code or scripts and hence, has tighter security
2) This method sends the data to php pages which can then run filtering, verification and checking routines on the data to ensure it is real.
3) This method uses CRC32 hashes to verify the data that is transferred
What you will need to use this system:
- A web server that supports php
- A mysql server (or mssql or odbc if you prefer...up to you)
- The TGE (DUH!)
------------------------------------------------------------------
ENGINE MODIFICATIONS
The engine modifications are extremely minimal. The first is a single function that we add to the end of consoleFunctions.cc. This functions allows us to return the CRC32 hash of a string that is compatible with php's CRC32
The second engine modification is really a hack. At around line 143 in httpObject.cc you will find the following:
Change it to:
What this does is allows you to use the text tag \t as & in your get variable strings allowing you to send multiple variables in a single query. (TGE normally dosnt support this and & get turned into its hex equivalent)
Thats all the engine modifications!!
------------------------------------------------------------------
USERNAME/PASSWORD VERIFICATION SCRIPT
The model we are going to use for this assumes a couple of things:
1) Only the game server should EVER talk to the character server, never the client machines
2) The game server should never actually see the players real password for security reasons.
In fps/client/ui/startMissionGui.gui at approx line 153 replace this
%conn.setConnectArgs($pref::Player::Name);
with
%passcrc = getStringCRC($pref::Player::Password) @ ""; // @ "" forces number it to string
%conn.setConnectArgs($pref::Player::Name, %passcrc);
In fps/client/ui/joinServerGui.gui at approx line 327 replace this
%conn.setConnectArgs($pref::Player::Name);
with
%passcrc = getStringCRC($pref::Player::Password) @ ""; // @ "" forces number it to string
%conn.setConnectArgs($pref::Player::Name, %passcrc);
In fps/server/scripts/game.cs add the following (we needed to change this function from common)
Now, although this code WORKS it should be improved to support
some of the following aspects:
1) Multiple server retries in case of packetloss, timeout, etc
---------------------------------------------------------------
Quite obviously this method can also be used for the storage of
persistant character information such as equipment, stats, etc All of these depend upon your game and can be done in this method.
Hope you find this to be some use
Any questions, just ask em :)
Regards
Daniel Neilsen
Torque Character Server Tutorial
by Daniel Neilsen
DISCLAIMER:
Make sure you back up all your code before doing any of this.
IMPORTANT NOTE: You MUST change this setup for security reasons and to suit your own game. This tutorial is only to give you an indication on how you can do it and shold not be copied verbatim.
This tutorial will show you how to make a remote character server that will integrate with the TGE. This system uses TGE's httpObject to access a php webserver and does NOT directly access any sql or database server for the following reasons:
1) This method requires no username or password to be stored in either the engine code or scripts and hence, has tighter security
2) This method sends the data to php pages which can then run filtering, verification and checking routines on the data to ensure it is real.
3) This method uses CRC32 hashes to verify the data that is transferred
What you will need to use this system:
- A web server that supports php
- A mysql server (or mssql or odbc if you prefer...up to you)
- The TGE (DUH!)
------------------------------------------------------------------
ENGINE MODIFICATIONS
The engine modifications are extremely minimal. The first is a single function that we add to the end of consoleFunctions.cc. This functions allows us to return the CRC32 hash of a string that is compatible with php's CRC32
ConsoleFunction(getStringCRC, S32, 2, 2, "getStringCRC(string)")
{
argc;
U32 crcVal;
crcVal = calculateCRC (argv[1], dStrlen(argv[1]), -1);
crcVal = (crcVal*-1)-1;
return(S32(crcVal));
}The second engine modification is really a hack. At around line 143 in httpObject.cc you will find the following:
if(asciiEscapeTable[c])
{
dest[destIndex++] = '%';
dest[destIndex++] = getHex((c >> 4) & 0xF);
dest[destIndex++] = getHex(c & 0xF);
}
else
dest[destIndex++] = c;Change it to:
if(asciiEscapeTable[c])
{
if(c == 9)
{
dest[destIndex++] = '&';
}
else
{
dest[destIndex++] = '%';
dest[destIndex++] = getHex((c >> 4) & 0xF);
dest[destIndex++] = getHex(c & 0xF);
}
}
else
dest[destIndex++] = c; What this does is allows you to use the text tag \t as & in your get variable strings allowing you to send multiple variables in a single query. (TGE normally dosnt support this and & get turned into its hex equivalent)
Thats all the engine modifications!!
------------------------------------------------------------------
USERNAME/PASSWORD VERIFICATION SCRIPT
The model we are going to use for this assumes a couple of things:
1) Only the game server should EVER talk to the character server, never the client machines
2) The game server should never actually see the players real password for security reasons.
In fps/client/ui/startMissionGui.gui at approx line 153 replace this
%conn.setConnectArgs($pref::Player::Name);
with
%passcrc = getStringCRC($pref::Player::Password) @ ""; // @ "" forces number it to string
%conn.setConnectArgs($pref::Player::Name, %passcrc);
In fps/client/ui/joinServerGui.gui at approx line 327 replace this
%conn.setConnectArgs($pref::Player::Name);
with
%passcrc = getStringCRC($pref::Player::Password) @ ""; // @ "" forces number it to string
%conn.setConnectArgs($pref::Player::Name, %passcrc);
In fps/server/scripts/game.cs add the following (we needed to change this function from common)
function GameConnection::onConnect( %client, %name, %passcrc )
{
// Send down the connection error info, the client is
// responsible for displaying this message if a connection
// error occures.
messageClient(%client,'MsgConnectionError',"",$Pref::Server::ConnectionError);
// Send mission information to the client
sendLoadInfoToClient( %client );
// Simulated client lag for testing...
// %client.setSimulatedNetParams(0.1, 30);
// if hosting this server, set this client to superAdmin
if (%client.getAddress() $= "local") {
%client.isAdmin = true;
%client.isSuperAdmin = true;
}
// Get the client's unique id:
// %authInfo = %client.getAuthInfo();
// %client.guid = getField( %authInfo, 3 );
%client.guid = 0;
addToServerGuidList( %client.guid );
//*******
//Store the clients passcrc
%client.passcrc = %passcrc;
checkClientLogin(%client, %name, %passcrc);
//give 30 seconds for response or boot him!
%client.loginLoop = schedule(30000, 0, "clientLoginFailed", %client);
//*******
// Set admin status
%client.isAdmin = false;
%client.isSuperAdmin = false;
// Save client preferences on the connection object for later use.
%client.gender = "Male";
%client.armor = "Light";
%client.race = "Human";
%client.skin = addTaggedString( "base" );
%client.setPlayerName(%name);
%client.score = 0;
//
$instantGroup = ServerGroup;
$instantGroup = MissionCleanup;
echo("CADD: " @ %client @ " " @ %client.getAddress());
// Inform the client of all the other clients
%count = ClientGroup.getCount();
for (%cl = 0; %cl < %count; %cl++) {
%other = ClientGroup.getObject(%cl);
if ((%other != %client)) {
// These should be "silent" versions of these messages...
messageClient(%client, 'MsgClientJoin', "",
%other.name,
%other,
%other.sendGuid,
%other.score,
%other.isAIControlled(),
%other.isAdmin,
%other.isSuperAdmin);
}
}
// Inform the client we've joined up
messageClient(%client,
'MsgClientJoin', '\c2Welcome to the Torque demo app %1.',
%client.name,
%client,
%client.sendGuid,
%client.score,
%client.isAiControlled(),
%client.isAdmin,
%client.isSuperAdmin);
// Inform all the other clients of the new guy
messageAllExcept(%client, -1, 'MsgClientJoin', '\c1%1 joined the game.',
%client.name,
%client,
%client.sendGuid,
%client.score,
%client.isAiControlled(),
%client.isAdmin,
%client.isSuperAdmin);
// If the mission is running, go ahead download it to the client
if ($missionRunning)
%client.loadMission();
$Server::PlayerCount++;
}
//The following settings assumes that the php file
//game_login.php exists at http://www.bla.com/auth/game_login.php
$AuthServer = "www.bla.com:80"; //url and port
$AuthServerPath = "/auth/"; //server path
function checkClientLogin(%client, %username, %passcrc)
{
%client.authReturned = false;
%query = "user="@%username@"\t"@
"passcrc="@%passcrc@"\r";
%server = $AuthServer;
%path = $AuthServerPath;
%script = %path @ "game_login.php";
%upd = new HTTPObject(CharLogin);
%client.sendObject = %upd; //store object details just in case
%upd.clientid = %client;
%upd.get(%server, %script, %query);
}
function CharLogin::onLine( %this, %line )
{
%client = %this.clientid;
//Check if webpage returns the word Verified on the frst line
if(StrStr(%line, "Verified") != -1)
{
cancel(%client.loginLoop);
return;
}
//otherwise, auth failed!
error("- ",%line);
clientLoginFailed(%client);
}
function CharLogin::onConnectionDied( %this )
{
%client = %this.clientid;
clientLoginFailed(%client);
}
function CharLogin::onDNSFailed( %this )
{
%client = %this.clientid;
clientLoginFailed(%client);
}
function CharLogin::onConnectFailed( %this )
{
%client = %this.clientid;
clientLoginFailed(%client);
}
function clientLoginFailed(%client)
{
//Do what you want to with him. Auth failed.
//(This is bad code, you want to improve it)
%client.delete();
}Now, although this code WORKS it should be improved to support
some of the following aspects:
1) Multiple server retries in case of packetloss, timeout, etc
---------------------------------------------------------------
Quite obviously this method can also be used for the storage of
persistant character information such as equipment, stats, etc All of these depend upon your game and can be done in this method.
Hope you find this to be some use
Any questions, just ask em :)
Regards
Daniel Neilsen
About the author
#2
04/04/2003 (10:44 am)
Yes it is Mike, for several reasons. And also, integrating the mysql++ api from mysqlab is almost impossible, the first problem was the well know new macro redefinition, and the other problem I got I wasn't sure what It meant, but it was a linking error too. So basically this way it's easier and also it's more secure, because the game never touches the databases's password, it's only stored in the web server.
#3
More than that, using HTTP to talk to a server that actually updates the database is good design - a standard three-tier approach, in fact. It has huge advantages in maintainability, reliability, and security. Most development tools are already geared towards the 3tier model, as well, so you save effort there, too.
Oh yeah - all them fancy enterprise apps are written using a tiered approach, so it's a lot easier to make your game really robust and scale up to have many users. You can drop in hardware/software and go from a dozen users to a few thousand, if you've set things up right.
04/04/2003 (11:20 am)
Urm.More than that, using HTTP to talk to a server that actually updates the database is good design - a standard three-tier approach, in fact. It has huge advantages in maintainability, reliability, and security. Most development tools are already geared towards the 3tier model, as well, so you save effort there, too.
Oh yeah - all them fancy enterprise apps are written using a tiered approach, so it's a lot easier to make your game really robust and scale up to have many users. You can drop in hardware/software and go from a dozen users to a few thousand, if you've set things up right.
#4
This means that should a hacker want this information, it is there for the taking and then he can log into your database and start messing with stuff.
Using this method, database access is completely hidden and filtered as it is done by the php page.
04/04/2003 (12:03 pm)
The other major reason why this method is better than directly linking to a database is simple security. To connect to a database you require the TGE to connect with the username and password and this means that this information must be contained in either the engine or the scripts.This means that should a hacker want this information, it is there for the taking and then he can log into your database and start messing with stuff.
Using this method, database access is completely hidden and filtered as it is done by the php page.
#5
04/05/2003 (9:21 am)
Daniel, but if you want the server to store information later, you will have to let the server have the database password... for example you want to save that the client XX picked up a new item, you gotta store that in the database and that has to be done by the game server, and the gameserver will need to have some kind of password to auth against the mysql database. So, a hacker could get the php scripts password used by the server to access the script and then mess the database via the script, he doesn't need the database password. What I was thinking is that it would be even better to configure your http server to accept only connections from certain ips (your game servers) instead of from anywhere. That would be a very secure way, don't you think?
#6
1) Send data from game server to php which includes the item type, the number of items, the username and the password CRC32. The php script should store this information in a tempory mysql table called "verify" and should set an extra field in the "verify" table that is a unix timestamp set at 30 seconds time. The php script should then output some random verification number. This might be a current time or some number that is always going to be differnet.
2) The game server takes that verification number and then submits it to a second php page. This php page then takes the information out of the temporary table and puts it into the REAL data table. A verification of this event working ok is sent back to the game server.
Some things to note:
The unix timestamp is for cleanup purposes. Every script in your system should activate a cleanup routine that deletes any old data from the system. This may be also useful to clean up user accounts that havnt been logged into for 6 months (as an example)
04/05/2003 (1:02 pm)
I wouldnt do it that way. Here is how I would do it (and have done it). It requires two php pages fro security pages but works essentially the same as the login example shown above.1) Send data from game server to php which includes the item type, the number of items, the username and the password CRC32. The php script should store this information in a tempory mysql table called "verify" and should set an extra field in the "verify" table that is a unix timestamp set at 30 seconds time. The php script should then output some random verification number. This might be a current time or some number that is always going to be differnet.
2) The game server takes that verification number and then submits it to a second php page. This php page then takes the information out of the temporary table and puts it into the REAL data table. A verification of this event working ok is sent back to the game server.
Some things to note:
The unix timestamp is for cleanup purposes. Every script in your system should activate a cleanup routine that deletes any old data from the system. This may be also useful to clean up user accounts that havnt been logged into for 6 months (as an example)
#7
04/05/2003 (5:57 pm)
So you are saying that the database should authentica the user's login and password each times it wants to store some value?
#8
You need the username (or some kind of account id) to tell which account to store the inventory info under.
I was trying to show how to do a submission verification in the above example.
ie. When something is submitted to the php it then asks the game server if there was actually a submission. A system such as this would prevent (or help prevent) people simply "attacking" your character server in a web browser.
eg.
http://www.bla.com/submit.php?username=bla&item=AwesomeWeapon&count=99999
Something like this would fail because the return verification from the game server would not happen.
04/05/2003 (8:10 pm)
Well, that part is optionalYou need the username (or some kind of account id) to tell which account to store the inventory info under.
I was trying to show how to do a submission verification in the above example.
ie. When something is submitted to the php it then asks the game server if there was actually a submission. A system such as this would prevent (or help prevent) people simply "attacking" your character server in a web browser.
eg.
http://www.bla.com/submit.php?username=bla&item=AwesomeWeapon&count=99999
Something like this would fail because the return verification from the game server would not happen.
#9
John.
04/05/2003 (9:10 pm)
Just out of curiosity, if only the game servers talk to the character server, why would the character server be accessible from the web? I'm assuming only the game servers on your site would be talking to your character server, right (I might be misunderstanding the setup)? Even if you needed multiple sites the games servers can be connect to the character servers via private pipe.John.
#10
It really all depends on your game model how you set it up. This tutorial is really a guide as to how it can be done but should always be altered to suit your own game
04/05/2003 (9:30 pm)
That would certainly be the case if the game only requires dev controlled game servers. In the game that I made, ANYONE could host a server and their character experience, weapons, etc would be saved.It really all depends on your game model how you set it up. This tutorial is really a guide as to how it can be done but should always be altered to suit your own game
#11
04/11/2003 (12:40 pm)
Excellent resource, thank you very much. Do you have an example of using this to store inventory? or player profiles?
#12
Using the changes to the engine that I have documented you can easily send a great number of variables at once. How you actually do it depends on the style of the game and how your inventory and characters are set up.
04/11/2003 (8:42 pm)
The mechanics are really no different than the login example. Really the only difference is that instead of sending username and password you might send username, itemid and itemcount as well as some kind of security verification system data.Using the changes to the engine that I have documented you can easily send a great number of variables at once. How you actually do it depends on the style of the game and how your inventory and characters are set up.
#13
Using a web server to authenticate and store data is a very nice idea! :)
Just a question though.
When connection is requested, the game server will call the script via http. But if the web server is down for instance, won't the game server hang on and freeze (until timeout, ending in error 404 or sth)? During that waiting time, will the game server be halted? If the server only waits 30 sec for the page to be loaded and stops doing anything else during that time, it could be annoying (?_?)
Anyway, thanks for this resource,
Isi
04/17/2003 (12:12 pm)
Very nice resource!Using a web server to authenticate and store data is a very nice idea! :)
Just a question though.
When connection is requested, the game server will call the script via http. But if the web server is down for instance, won't the game server hang on and freeze (until timeout, ending in error 404 or sth)? During that waiting time, will the game server be halted? If the server only waits 30 sec for the page to be loaded and stops doing anything else during that time, it could be annoying (?_?)
Anyway, thanks for this resource,
Isi
#14
04/17/2003 (12:12 pm)
I submitted my post twice ^^;
#15
It essentially makes the http request then goes about its normal business until it gets a response. :)
04/17/2003 (4:46 pm)
To answer your question, no the game server will not hang.It essentially makes the http request then goes about its normal business until it gets a response. :)
#16
04/18/2003 (1:27 am)
Thanks for your quick answer, Daniel!
#17
Like finding startMissionGui.cs I find StartMissionGui.gui not .cs .. I did a search, am I missing something?
I edit that, and I still don't find the code. so either I am using something lower, or higher or what, but need some help here :-D .. please.. this is a very good aspect, I am going to change your crc32 to something else possibly. maybe add multiple means of encryption , but that's about it. least get me started here.. thanks
Brad
Please tell me I am just dumb ;) or .. send some updates!!!!!!! :]
06/07/2003 (9:56 pm)
Ok, what version are you using? when I search for the code, it's not there. Could I be lost? lol.. Like finding startMissionGui.cs I find StartMissionGui.gui not .cs .. I did a search, am I missing something?
I edit that, and I still don't find the code. so either I am using something lower, or higher or what, but need some help here :-D .. please.. this is a very good aspect, I am going to change your crc32 to something else possibly. maybe add multiple means of encryption , but that's about it. least get me started here.. thanks
Brad
Please tell me I am just dumb ;) or .. send some updates!!!!!!! :]
#18
They are the gui files you need to edit.
Ill update the resource
06/08/2003 (1:26 pm)
Sorry, you are correctThey are the gui files you need to edit.
Ill update the resource
#19
From that point on, you either use a new key (if you're paranoid), or re-use the one you already have.
Talking to a HTTP server is a nice idea, practically though anyone with knowledge of how HTTP works could spoof a game connection and seriously screw things up if they wanted to - another way to do it, and stay 3-tiered is to hack up your own authentication daemon which takes the place of the HTTP server, and add some C++ code to have a persistant TCP connection to it.
Ofcourse, it's late here and I have no idea if this makes any sense ;)
07/18/2003 (2:15 pm)
Something I've used in the past with this (not sure if the HTTP object supports it, or whether it's doable without adding any new C++ code) was to initially talk to a webserver and have it spit out a key that can be used to encrypt communications; you'd first chat up the webserver and get a key, use that key to encrypt username and password, then send the package back to the server which would then let you know whether or not the account is valid.From that point on, you either use a new key (if you're paranoid), or re-use the one you already have.
Talking to a HTTP server is a nice idea, practically though anyone with knowledge of how HTTP works could spoof a game connection and seriously screw things up if they wanted to - another way to do it, and stay 3-tiered is to hack up your own authentication daemon which takes the place of the HTTP server, and add some C++ code to have a persistant TCP connection to it.
Ofcourse, it's late here and I have no idea if this makes any sense ;)
#20
gets me 4 errors:
C:\torque\engine\game\net\httpObject.cc(145) : error C2014: preprocessor command must start as first nonwhite space
C:\torque\engine\game\net\httpObject.cc(146) : error C2059: syntax error : '}'
C:\torque\engine\game\net\httpObject.cc(149) : error C2014: preprocessor command must start as first nonwhite space
C:\torque\engine\game\net\httpObject.cc(150) : error C2440: '=' : cannot convert from 'char' to 'char *'
Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast
Does anyone have any ideas?
04/01/2004 (10:26 am)
This code:if(asciiEscapeTable[c])
{
if(c == 9)
{
dest[destIndex++] = ´&´;
}
else
{
dest[destIndex++] = ´%´;
dest[destIndex++] = getHex((c >> 4) & 0xF);
dest[destIndex++] = getHex(c & 0xF);
}
}
else
dest[destIndex++] = c;gets me 4 errors:
C:\torque\engine\game\net\httpObject.cc(145) : error C2014: preprocessor command must start as first nonwhite space
C:\torque\engine\game\net\httpObject.cc(146) : error C2059: syntax error : '}'
C:\torque\engine\game\net\httpObject.cc(149) : error C2014: preprocessor command must start as first nonwhite space
C:\torque\engine\game\net\httpObject.cc(150) : error C2440: '=' : cannot convert from 'char' to 'char *'
Conversion from integral type to pointer type requires reinterpret_cast, C-style cast or function-style cast
Does anyone have any ideas?

Torque Owner Mike Stoddart