Central server and multiple clients

#1 Joe, these are all great questions, ones that I would also like to have more feedback on.

So consider this *bumped*

#2 I guess maybe I asked too much, wasn't clear enough, or nobody knows how to answer.

I'm not asking how to write anything on the bullet list up there.

All I want to know is if I need to actually run a game on the server for each game the clients play, which seems like it'd be too much of a strain on the server to me. Alternatively I was thinking clients could send messages to the server to verify if a move is ok, and if so the server then performs their actions by issuing commands to those clients. I know the things they own (characters, items, stats etc.) would be stored in a database and accessed when they login. I'm looking for some direction about how to best handle several (turn based) games between several logged in players.

#3 From your information you've provided you will want to have the server maintain all "rules" (movement etc). It depends on your trust level of the clients. If you are not worried about cheating then have one client act as the server between the two. However if you feel that cheating is not something you can allow then the option is to have the server maintain everything.

Depending on the complexitiy of the game you would only need one server. Otherwise you would have to create a load balancing system to handle large numbers of clients (such as a MMO setup:: MMOkit www.mmoworkshop.com). TGB was not built with a dedicated server in mind so you'll have to strip it down and make it not do any rendering/texture loading (GL calls). I am currently working on this however I'm a hobbiest and it is never certain if I'll finish anything!

Waeric

#4 If you look at my thread I just posted about rakNet, I believe it might solve some of the problems you have with cheating etc. You can run it under linux as a 'server' completely untied to TGB and TGB would just be the 'clients' connecting to your linux box. This would also tie in nicely with a virtual linux hosting plan like linode.

#5 Wow.. RakNet has some great features! It does seem a bit confusing, but that could just be my newbieness. It seems to have a good amount of examples and documentation, which is nice.

Some things I like about it already are the built in automatic patching and encrypted traffic features. Can't beat the price either. This looks like a good answer to many of my questions.

Well I got it built and it seems to be working. I haven't had a chance to install the PostgreSQL database or try any of that stuff yet. I don' really understand how to use RakNet yet or how to best add it to TGB. I guess I can just make some new console functions that call the RakNet code.

#6 I'll show you the code I've done so far, you'll have varying results depending on how you write your game scripts, I haven't tried optimizing yet or read much in the raknet documentation.

Earlier, I had it looking pretty good over LAN, although over the internet it was more laggy. Right now I am just calling it with schedules every 32 ms and it works great except its delayed by 1 and a half seconds roughly. Considering I had it working in more real time earlier, I think its just a matter of reading the docs and support to figure out the best way to do it.

Note that my way of doing this is probably NOT the best way, I'm just sending raw data and apparently with bitstreams or something like that you can compress the data. However this should work as a proof of concept and might get you started.

I broke up the c++ code into client and server, although it shouldn't be very hard to work from one TGB executable and just have seperate client/server torquescript projects.
Here is code for the client:

RakPeerInterface *rakPeerClient;

struct ShipPositionMessage
{
	unsigned char messageId;
	S32 x;
	S32 y;
	S32 rotation;
	S32 xSpeed;
	S32 ySpeed;
};

ConsoleFunction(initializeRak, S32, 2, 2, "(string one, string two)"
                "Case sensitive string compare.")
{
   argc;
	rakPeerClient = RakNetworkFactory::GetRakPeerInterface();
	SocketDescriptor s1(1235, 0);
	rakPeerClient->Startup(3, 0, &s1, 1);
	rakPeerClient->Connect("192.168.1.105", 1234, 0, 0 ,0);
	Con::printf("%s", "Rak Client Initialized (C++)");
   return 1;
}

ConsoleFunction(checkConnect, S32, 2, 2, "(string one, string two)"
                "Case sensitive string compare.")
{
    argc;
	Packet *packet = rakPeerClient->Receive();
	if (packet)
	{
		if(packet->data[0]==ID_CONNECTION_REQUEST_ACCEPTED)
		{
			Con::printf("%s", "ID_CONNECTION_REQUEST_ACCEPTED");
			Con::printf("%s", "ID_CONNECTION_REQUEST_ACCEPTED");
			Con::printf("%s", "ID_CONNECTION_REQUEST_ACCEPTED");
			Con::printf("%s", "ID_CONNECTION_REQUEST_ACCEPTED");
			Con::printf("%s", "ID_CONNECTION_REQUEST_ACCEPTED");
			return 1;
			
		}
		else if (packet->data[0]==ID_CONNECTION_ATTEMPT_FAILED)
		{
				Con::printf("%s", "ID_CONNECTION_ATTEMPT_FAILED");
				Con::printf("%s", "ID_CONNECTION_ATTEMPT_FAILED");
				Con::printf("%s", "ID_CONNECTION_ATTEMPT_FAILED");
				Con::printf("%s", "ID_CONNECTION_ATTEMPT_FAILED");
				Con::printf("%s", "ID_CONNECTION_ATTEMPT_FAILED");
			return 0;
		}
		rakPeerClient->DeallocatePacket(packet);
	}
	
	
	return  -1;
}

ConsoleFunction(sendPosition, S32, 6, 6, "send ship position and velocity and rotation")
{
	argc;
	ShipPositionMessage message;
	message.messageId=ID_USER_PACKET_ENUM;
	message.x=dAtoi(argv[1]);
	message.y=dAtoi(argv[2]);
	message.rotation=dAtoi(argv[3]);
	message.xSpeed=dAtoi(argv[4]);
	message.ySpeed=dAtoi(argv[5]);
	rakPeerClient->Send((char*) &message, sizeof(message)+1, HIGH_PRIORITY, UNRELIABLE_SEQUENCED,  0, UNASSIGNED_SYSTEM_ADDRESS, true);
	return 1;
}

Here is code for the server:

struct ShipPositionMessage
{
	unsigned char messageId;
	S32 x;
	S32 y;
	S32 rotation;
	S32 xSpeed;
	S32 ySpeed;
};

RakPeerInterface *rakPeerServer;
ShipPositionMessage globalMessageHolder;

ConsoleFunction(initializeRak, S32, 2, 2, "(string one, string two)"
                "Case sensitive string compare.")
{
   argc;
	rakPeerServer = RakNetworkFactory::GetRakPeerInterface();
	SocketDescriptor s1(1234, 0);
	rakPeerServer->Startup(3, 0, &s1, 1);
	rakPeerServer->SetMaximumIncomingConnections(3);
	Con::printf("%s", "Rak Initialized (c++ message)");
	
   return 1;
}



ConsoleFunction(checkConnect, S32, 2, 2, "(string one, string two)"
                "Case sensitive string compare.")
{
    argc;
	Packet *packet = rakPeerServer->Receive();
	if (packet)
	{
		if(packet->data[0]==ID_NEW_INCOMING_CONNECTION)
		{
			Con::printf("%s", "ID_NEW_INCOMING_CONNECTION");
			Con::printf("%s", "ID_NEW_INCOMING_CONNECTION");
			Con::printf("%s", "ID_NEW_INCOMING_CONNECTION");
			Con::printf("%s", "ID_NEW_INCOMING_CONNECTION");
			Con::printf("%s", "ID_NEW_INCOMING_CONNECTION");
			return 5;
		}
	}
	return  -1;
}

ConsoleFunction(sendXPos, S32, 2, 2, "send x position")
{
	argc;
	ShipPositionMessage message;
	message.messageId=ID_USER_PACKET_ENUM;
	message.x=dAtoi(argv[1]);
	Con::printf("%i", message.x );
	rakPeerServer->Send((char*) &message, sizeof(message), HIGH_PRIORITY, UNRELIABLE_SEQUENCED,  0, UNASSIGNED_SYSTEM_ADDRESS, true);
	return 1;
}

ConsoleFunction(receiveMessages, int, 2, 2, "Receive other player's data")
{
    argc;
	Packet *packet = rakPeerServer->Receive();
	if (packet)
	{
		if(packet->data[0]==ID_USER_PACKET_ENUM)
		{
			ShipPositionMessage* receivedShipPos = (ShipPositionMessage*)packet->data;
			globalMessageHolder.x = receivedShipPos->x;
			globalMessageHolder.y = receivedShipPos->y;
			globalMessageHolder.rotation = receivedShipPos->rotation;
			globalMessageHolder.xSpeed = receivedShipPos->xSpeed;
			globalMessageHolder.ySpeed = receivedShipPos->ySpeed;
		}
	}
	return -1;
}

ConsoleFunction(receiveX, int, 2, 2, "Receive other player's x position")
{
    argc;
	return globalMessageHolder.x;
}

ConsoleFunction(receiveY, int, 2, 2, "Receive other player's x position")
{
    argc;
	return globalMessageHolder.y;
	
}

ConsoleFunction(receiveRotation, int, 2, 2, "Receive other player's x position")
{
    argc;
	return globalMessageHolder.rotation;
	
}

ConsoleFunction(receiveXSpeed, int, 2, 2, "Receive other player's x position")
{
    argc;
	return globalMessageHolder.xSpeed;
	
}

ConsoleFunction(receiveYSpeed, int, 2, 2, "Receive other player's x position")
{
	argc;
	return globalMessageHolder.ySpeed;
	
}

You may notice that my code is highly inefficient, I'm calling a seperate console function to receive each variable of the incoming data. That is because I hadn't yet read this reply:


Also you can see I know nothing of c++, not even how to loop as I could have done when printing out the console messages :)

http://www.garagegames.com/mg/forums/result.thread.php?qt=71960

#7 Heh. Not trying to play down your accomplishments in any way, but you can do everything you just did in TGB already, and in fact do it all in script. It would be slightly faster in c++, but it's not required.

What you are effectively doing is implementing an event based update system, which is perfectly feasible in TGB as it stands (I know for a fact--I just implemented it 2 weeks ago in a prototype form as part of the "Tech and Tools Team GiD" we did).

Normally when people mention "real time networking", they are speaking about a system that has at least some of the following features:

--wind/unwind (server-client resynchronization)
--network interpolation/extrapolation
--latency compensation mechanisms
--bandwidth optimization mechanisms
--latest state update ack/nacking
--host of other capabilities depending on the situation

As I mentioned, TGB as it stands today (stock), can do everything that you appear to be using RakNet for currently. commandToServer and commandToClient are simply the same thing as your code here:

rakPeerServer->Send((char*) &message, sizeof(message), HIGH_PRIORITY, UNRELIABLE_SEQUENCED,  0, UNASSIGNED_SYSTEM_ADDRESS, true);

Except that netEvents generated by commandToXX in Torque are:

--TOP_PRIORITY
--RELIABLE_SEQUENCED
--sent to specific client or server as appropriate.

Of course, you can, in c++ create your own net events very easily, and implement any of the primary event types, including Guaranteed (RELIABLE), GuaranteedOrdered (RELIABLE_SEQUENCED), Unguaranteed (UNRELIABLE), etc.

#8 Thanks for your feedback on this. While I thought it might be possible to do it, I was not aware of how. I read threads saying that real time networking wasn't possible. Also I have no understanding of c++ so I wouldn't be able to create my own net events easily. :)

It is my understanding that rakNet does optimize for bandwidth. It doesn't do interpolation, I don't know about the other features. In fact I'm glad you put that list down as now I can focus on getting each one down for any game I make. :)

I apologize if I've caused any confusion as to the capabilities of TGB. I've read various threads saying you can't do real time networking in TGB as it stands, so I guess I took that out of context. I haven't scoured the forum for tutorials or explanations on how to get that working, but wasn't aware of any. I haven't used TGB for a while either.

In any case, I'll still be using it for now because it's my understanding that "unreliable_sequenced" is good for updating position, and the multiplayer lobby and encryption, security, and patching features. I would be unable to implement these things myself.

#9 Honestly I didn't mean to put down your work at all--I just wanted to try to enforce that "event based networking" does not imply turn based only. You can do all sorts of amazing things with event networking, as you are certainly seeing with your RakNet stuff--and TGB already has fully working net event based systems set up for you.

#10 It's interesting you say that it's not very cost-effective to implement a solution for all possible TGB games, but how hard would it be to implement a solution that was perhaps adjustable that worked with a great deal of simple games? For instance maybe there could be a behavior that you stick on an object and it gets sent over the network. If some games required different kinds of networking maybe you can adjust a flag. Maybe you're only talking about sending 5 objects or so over the network.

Of course that's not ideal, but if it worked for a great number of simple games, wouldn't that be worth it? There could of course be a disclaimer that this will not work on all possible TGB games.

I know there are a lot of people in the game maker community for instance that make very simple games and are interested in making them multiplayer and if you could say : "program your simple games in TGB and they will be networkable out of the box within reason" that could be a big draw for getting people to switch to TGB (they want to already once they realize the limitations of other 2D engines).

Of course you guys know best what to work on, just offering a suggestion :)

#11 The biggest core issue regarding real time networking in my personal opinion (other devs at GG disagree of course, and all of the issues make up the current development position) is two fold:

--proper optimization of the bitStream for real time networking requires c++ programming, or a decently extensive script based interface that would take several months to develop and polish.

--the very quote you mentioned: "within reason". Making sure market expectation is set properly is very difficult, and as soon as you provide the ability to start networking some objects, people are going to want to network all objects.

Think about a cooperative top down shooter--maybe 30-40 enemies on screen, and what, 100+ bullets? You simply will not be able to network these directly, but, ironically, using net events to send over creation and behavior data, having each client control them separately (and of course deterministically), and passing re-sync net events back and forth as appropriate is a much more feasible solution.

#12 Hey Jacob your code looks a lot like the code I wrote last night, basically. I rewrote the simple test program I had so it will now use either Raknet or TGB. It's a simple "server" TGB, with two clients that login to it and send simple messages. However it uses encryption to thwart cheating.

@Stephen Zepp
With stock TGB do we get secure traffic, an auto update system, about 40 examples covering every feature, and good networking documentation?

Is stock TGB networking really comparable to Raknet? I'm not trying to be a jerk, I'm curious because I don't feel that I know what TGB is capable of. There aren't many examples or information to go on. I'm only considering Raknet because I don't think I know enough technically to make TGB do the things Raknet seems to do out of the box.

#13 Hey Joe, would you mind sharing any of that code? :)

Like I said earlier, in my first test I had smooth networking over LAN, in my second test I had about a 1 second delay. The second time I used schedules. If you got some nice twitch-based gaming to work over the net, I'd be interested in seeing how you did it. I don't know if you noticed, but there is a 'network simulator' in rakNet that might be useful, though I haven't tried it yet.

#14 I haven't delved too deep into Raknet myself. I haven't tried a "twitch" game yet because I'm more interested in the security and management features for a mainly turn based MMO game. I do have two clients on the same computer sending messages (as C structs) to the server at a pretty fast rate, at least it seems fast to me, and that's without the fastest settings. I haven't done enough testing to see if realtime is possible but it should be considering there are realtime commercial games online that use Raknet every day.

Here's what I have so far, it's not much. Like I said it's very similar to your code, except I implemented it as a "ConObject" instead of a bunch of console functions. I also gave the simple usage examples for your server and clients. To make it faster you could try changing all the "30's" to lower values, or zeros.

#include "RakPeerInterface.h"
#include "RakNetworkFactory.h"
#include "MessageIdentifiers.h"
#include <stdio.h>
#include <string.h>

using namespace RakNet;

//Edit these as needed
#define SERVER_PORT 60005
#define CLIENT_PORT 60006
#define MAX_PLAYER_CONNECTIONS 512
 
#ifndef _SIMBASE_H_
#include "console/simBase.h"
#endif


class RakNetObject : public SimObject {
   protected:  typedef SimObject Parent;
   public:
      RakNetObject();
      ~RakNetObject(); 
      bool processArguments(S32 argc, const char **argv);
      bool  onAdd();
      void onRemove();
      static void initPersistFields(); 
      void startServer( );
      void serverProcessPackets(); 
      void startClient( );
      void connectToServer(   const char *ip  );
      void commandToServer(   const char *s  );
      void clientProcessPackets();
      S32 clientIsConnectedToServer(); 
      void doCallback();
   public:
       DECLARE_CONOBJECT(RakNetObject);
       S32 isConnected;
       RakPeerInterface *client;
       RakPeerInterface *server;
};


#include "console/consoleInternal.h"  
IMPLEMENT_CONOBJECT(RakNetObject);

RakNetObject::RakNetObject( ){   isConnected = 0; }

RakNetObject::~RakNetObject(){  }

bool RakNetObject::processArguments( S32 argc, const char **argv){
  if(!argc) return true; else return true; return false;
}

bool RakNetObject::onAdd(){
   if (!Parent::onAdd())  return false;
   const char *name = getName();
   if(name && name[0] && getClassRep())   {
      Namespace *parent = getClassRep()->getNameSpace();
      Con::linkNamespaces(parent->mName, name);
      mNameSpace = Con::lookupNamespace(name);
   }
 return true;
}

void RakNetObject::onRemove(){   Parent::onRemove();  } 
void RakNetObject::initPersistFields(){   Parent::initPersistFields();} 
void RakNetObject::doCallback(){  Con::executef(this, 2, "onCallback", "string_variable");
}ConsoleMethod(RakNetObject,doCallback,void,2,2,
"Calls the RakNetObject callback script function"){object->doCallback();}


///======================================   SERVER FUNCTIONS
void RakNetObject::startServer( ){  
       server = RakNetworkFactory::GetRakPeerInterface(); 
       server->InitializeSecurity(0,0,0,0);

    SocketDescriptor socketDescriptor(SERVER_PORT,0);      
    if ( !server->Startup( MAX_PLAYER_CONNECTIONS , 30, &socketDescriptor, 1) )
		Con::printf("Server startup failed! :( \n");

    server->SetMaximumIncomingConnections( MAX_PLAYER_CONNECTIONS );
}ConsoleMethod(RakNetObject, startServer, void ,2,2,"Start server"){
    object->startServer( );
}

struct ClientCommand {
   unsigned char messageId;
   char str[256];
};

void RakNetObject::serverProcessPackets(  ){
            Packet *p = server->Receive();
			while (p)
			{
				switch (p->data[0])
				{
				case ID_CONNECTION_REQUEST_ACCEPTED:
					Con::printf("Server : ID_NEW_INCOMING_CONNECTION\n Welcome a new user!");
					break;
				case ID_CONNECTION_ATTEMPT_FAILED:
					Con::printf("Server Error: ID_CONNECTION_ATTEMPT_FAILED\n");
					break;
				case ID_ALREADY_CONNECTED:
					Con::printf("Server Error: ID_ALREADY_CONNECTED\n");
					break;
				case ID_NO_FREE_INCOMING_CONNECTIONS:
					Con::printf("Server Error: ID_NO_FREE_INCOMING_CONNECTIONS\n Server is full! ");
					break;
				case ID_DISCONNECTION_NOTIFICATION:
					isConnected=false;
					break;
				case ID_CONNECTION_LOST:
					Con::printf("Server Error: ID_CONNECTION_LOST\n");
					break;
				case ID_MODIFIED_PACKET:
					Con::printf("Server Error: ID_MODIFIED_PACKET\n");
					break;
				default:
				  Con::printf("Command Packet Recieved: %s", p->data+1 );
				  break;
				}
				server->DeallocatePacket(p);
				p = server->Receive();
			}

}ConsoleMethod(RakNetObject, serverProcessPackets, void ,2,2,"process packets recieved by the server"){
    object->serverProcessPackets( ) ;
}






///=======================================   CLIENT FUNCTIONS
void RakNetObject::startClient(   ){
	client = RakNetworkFactory::GetRakPeerInterface();
	SocketDescriptor socketDescriptor(CLIENT_PORT,0);
    if ( !client->Startup(1, 30, &socketDescriptor, 1)  )
		Con::printf("Client startup failed!\n");

}ConsoleMethod(RakNetObject, startClient, void ,2,2,"Start client"){
    object->startClient( );
}


void RakNetObject::connectToServer(    const char *ip   ){
    Con::printf(" Connecting to server %s %d",  ip,  SERVER_PORT );
    client->Connect(  ip, (unsigned short)SERVER_PORT  , 0, 0, 0 );

}ConsoleMethod(RakNetObject, connectToServer, void ,3,3,"Connect client to server ( ip  )"){
    object->connectToServer( argv[2] ) ;
}

S32 RakNetObject::clientIsConnectedToServer(  ){
    return isConnected;
}
ConsoleMethod(RakNetObject, clientIsConnectedToServer, S32 ,2, 2, "Tells you if the client is connected to the server."){
    return object->clientIsConnectedToServer(  ) ;
}

void RakNetObject::clientProcessPackets(  ){
            Packet *p = client->Receive();
			while (p)
			{
				switch (p->data[0])
				{
				case ID_CONNECTION_REQUEST_ACCEPTED:
					Con::printf("Client : ID_CONNECTION_REQUEST_ACCEPTED\n");
					isConnected=true;
					break;
					// print out errors
				case ID_CONNECTION_ATTEMPT_FAILED:
					Con::printf("Client Error: ID_CONNECTION_ATTEMPT_FAILED\n");
					isConnected=false;
					break;
				case ID_ALREADY_CONNECTED:
					Con::printf("Client Error: ID_ALREADY_CONNECTED\n");
					break;
				case ID_NO_FREE_INCOMING_CONNECTIONS:
					Con::printf("Client Error: ID_NO_FREE_INCOMING_CONNECTIONS\n Server is full! ");
					isConnected=false;
					break;
				case ID_DISCONNECTION_NOTIFICATION:
					Con::printf("Client Error: ID_DISCONNECTION_NOTIFICATION\n Lost connection to Server.");
					isConnected=false;
					break;
				case ID_CONNECTION_LOST:
					Con::printf("Client Error: ID_CONNECTION_LOST\n Lost connection to Server.\n Either the server is down, or your internet connection is not working.");
					isConnected=false;
					break;
				case ID_MODIFIED_PACKET:
				    //This should never happen unless someones hacking something...!
					Con::printf("Client Error: ID_MODIFIED_PACKET\n");
					break;
				}
				client->DeallocatePacket(p);
				p = client->Receive();
			}

}ConsoleMethod(RakNetObject, clientProcessPackets, void ,2,2,"process packets recieved by the client"){
    object->clientProcessPackets( ) ;
}


void RakNetObject::commandToServer(    const char *s   ){
    ClientCommand msg;
    msg.messageId = ID_USER_PACKET_ENUM;
    dStrcpy( msg.str , s );
    client->Send( (char*)&msg , sizeof(msg),   HIGH_PRIORITY, RELIABLE_ORDERED, 0 , UNASSIGNED_SYSTEM_ADDRESS, 1 );


}ConsoleMethod(RakNetObject, commandToServer, void ,3,3,"send a command to server ( command  )"){
    object->commandToServer( argv[2] ) ;
}

#15 (Continued...)

Here's what the server does in game.cs:

function Start_Raknet( ){          
  $RakNetObject = new RakNetObject(raknet);
   raknet.startServer();
   canvas.setcontent(mainScreenGui);
   schedule(1000,0, "processPackets");
}

function processPackets(%this){          
   raknet.serverProcessPackets();   
   schedule(30,0, "processPackets");  //may need to make this even lower, like 0-10   
}

Here's what the clients do in their game.cs:

function Start_Raknet( ){   
   $RakNetObject = new RakNetObject(raknet);          
    raknet.startClient();
    raknet.connectToServer(  "127.0.0.1"  );
    schedule( 1000, 0, "processPackets" );
}

function processPackets(){         
  raknet.clientProcessPackets(); 
  if ( !raknet.clientIsConnectedToServer() )
      GameConnection::onConnectionDropped();
  else   
      schedule( 30, 0, "processPackets" );
}

#16 Joe: If you are after MMO features why not work with the MMOKit rather than re-invent the wheel? Python is a simple language to learn and Twisted (networking library for Python) is powerful. I'm just wondering since if you start to have performance issues with scripted raknet then you could be in for a long haul getting it working 100%. I'm curious even though it's turn-based have you considered the amount of players per "shard/server" and how many object updates across each client are required? Or are you just playing with the technology now to see what you can do?

#17 I'm just playing with ideas for now, not sure how many objects would be used per player but probably not very many. I looked at the MMO kit once before but it seemed confusing to me. I'm sure it works great if you know how to use it, but I was hoping for something smaller and easier. I'm also not proficient with Python, I'm more of a Lua guy...but perhaps I will take another look.

Oh yea. And I would have to buy a license for TGE and AFX. Which I don't really need right now. Also the kit seems to be geared more toward 3D games. I'm thinking of a simple 2D game.

#18 The MMOKit doesn't really require TGE if you can strip it down; however it is daunting. I was just mentioning it because it seems just as daunting to create a whole new server architecture since this has been used for a commercial game (aka proven). If you do create a management system in RakNet I do suggest creating a tutorial for it so that other developers have more options. I will have to check out RakNet tomorrow though since it is another option.

Keep us up to date on your progress!

#19 I've been beating that "event-based != turn-based" networking drum for a while and I'm glad to see somebody from GG finally discussing it on the forums.