Dedacated servers behind firewalls and routers...
by Max Thomas · in Torque Game Engine · 03/31/2005 (8:47 pm) · 13 replies
Hey guys,
Ok, this has really been bugging me for the past week, I've searched through the forums high and low, found a few small things, but most of them were not practacle... Well, the title really says it all, but these days almost everyone is behind a firewall or router, everyone who has cable/DSL these days has a router... Now, the problem is, that no one can pick up the dedacated servers, or even the TGE demo servers... They are all behind a firewall or a router, sometimes both... I've had them run a dedicated server, I've tried ran one, but can't seem to query them.... LAN games work fine, not a single problem... I've ran a dedicated server from my computer here and on this same computer used the gnometech.com master server checker thingy to see if I could find my server, I could find it with no problem, it odly wouldn't say weither or not the player was in the game though... but it would come up, I havn't seen if it is in that when other people host it yet, but ahh well, will try that later on. I'm concidering TNL, because I know that that has no problems getting through all of this crap somehow. I just really want to know if there is any way around it withought having to dish out the cash for TNL... It just seems as though I've tried every possable thing now and still not getting a single thing... I know I've naturally missed some forum thread or some doc with critical info but I just figured that I'de ask, thats what these forums are for right!?
Well, enough rambleing, I assume that other people probably have the same problem here, I hope I'm not one out of millions!
Feedback would be greatly appreciated guys, I hope I don't bore you with things that you have heard countless times before, Max
Ok, this has really been bugging me for the past week, I've searched through the forums high and low, found a few small things, but most of them were not practacle... Well, the title really says it all, but these days almost everyone is behind a firewall or router, everyone who has cable/DSL these days has a router... Now, the problem is, that no one can pick up the dedacated servers, or even the TGE demo servers... They are all behind a firewall or a router, sometimes both... I've had them run a dedicated server, I've tried ran one, but can't seem to query them.... LAN games work fine, not a single problem... I've ran a dedicated server from my computer here and on this same computer used the gnometech.com master server checker thingy to see if I could find my server, I could find it with no problem, it odly wouldn't say weither or not the player was in the game though... but it would come up, I havn't seen if it is in that when other people host it yet, but ahh well, will try that later on. I'm concidering TNL, because I know that that has no problems getting through all of this crap somehow. I just really want to know if there is any way around it withought having to dish out the cash for TNL... It just seems as though I've tried every possable thing now and still not getting a single thing... I know I've naturally missed some forum thread or some doc with critical info but I just figured that I'de ask, thats what these forums are for right!?
Well, enough rambleing, I assume that other people probably have the same problem here, I hope I'm not one out of millions!
Feedback would be greatly appreciated guys, I hope I don't bore you with things that you have heard countless times before, Max
#2
Thanks, I knew a fair bit of that, but some stuff was good info, I think I mighta left my main question and problem out kinda... Ha, well, I mean that I want to be able for it to just be nice and simple for every client/player to get a dedicaed server up and running, or host/join a game and query them withought having to screw around with indavidual firewalls and ports and routers, some of that stuff can be overly tricky at times with certain software. I came across TGElobby a little while ago, havn't taken the time to read up about it yet though, could that solve and problems? From that it just seemed to me like a better way to find servers and the likes... No server changes or whatever. Thats really why I was liking the TNL I just want to try and find a way around that that will be simple for everyone to get around withought any tinkering or the likes. I don't want to have every player put their IP adress into the game to be able to play and the likes... Any known ways around that?
Thank you very much so, Max
03/31/2005 (10:02 pm)
Hey,Thanks, I knew a fair bit of that, but some stuff was good info, I think I mighta left my main question and problem out kinda... Ha, well, I mean that I want to be able for it to just be nice and simple for every client/player to get a dedicaed server up and running, or host/join a game and query them withought having to screw around with indavidual firewalls and ports and routers, some of that stuff can be overly tricky at times with certain software. I came across TGElobby a little while ago, havn't taken the time to read up about it yet though, could that solve and problems? From that it just seemed to me like a better way to find servers and the likes... No server changes or whatever. Thats really why I was liking the TNL I just want to try and find a way around that that will be simple for everyone to get around withought any tinkering or the likes. I don't want to have every player put their IP adress into the game to be able to play and the likes... Any known ways around that?
Thank you very much so, Max
#3
You could of course write your own, but I would bet that it's worth the money to purchase--it's not a trivial project.
03/31/2005 (10:25 pm)
It does sound like in that case you want to implement your own master server, which (if you want to use the TNL code) will require a TNL license purchase. You would need to have a server with a permanent ip address that you provided to act as the master server, with it available to the internet.You could of course write your own, but I would bet that it's worth the money to purchase--it's not a trivial project.
#4
Yeah, I was afraid that there wasn't any way but that around this... Ahh well, ok, just a few questions, with TNL there is a way for people who have the game to host a server correct? not a master server, but is it possable for them to host their own server withought a master server? Like almost have a function to get their ip adress and use it as the master server when hosting only? I'm sure it's possable, anything is afterall, just want to know if that is already implamented or it would need to be done...
Thanx for all of the help, Max
03/31/2005 (10:37 pm)
Hey,Yeah, I was afraid that there wasn't any way but that around this... Ahh well, ok, just a few questions, with TNL there is a way for people who have the game to host a server correct? not a master server, but is it possable for them to host their own server withought a master server? Like almost have a function to get their ip adress and use it as the master server when hosting only? I'm sure it's possable, anything is afterall, just want to know if that is already implamented or it would need to be done...
Thanx for all of the help, Max
#5
Generally a system is using a private IP address which is then NATed by the interconnect device (router/firewall). As Stephen said, clients generally have no problem when making connection OUT to the internet. A NAT device can not allow incoming connections automatically. An incoming packet has no mapping and cannot be forwarded to the correct device on the user network. A user could potentially setup direct mapping, but this can be tricky and has other problems associated with it.
The major function of a master server, when the game server is behind a NAT device, is to act as an introducer. The game server advertises itself to the master server. When this is done, all the public IP addressing and port information is remembered by the master server. When a client wants to make a connection it gets this info from the master server. The master server advises the game server that a client is about to connect and provides the public IP/port of the client to the game server. BOTH the game server and the client then attempt to send info to each other. This has the effect that the firewalls at each end have the correct mapping to allow the data through.
Where the game server is not directly visible to the Internet, there must be a master server to perform the function of an introducer.
If the game server has a public IP address then anyone who knows that address can connect provided they know what the correct address is. No master server is required under these circumstances.
Hope I didn't repeat too much of what Stephen said.....
04/01/2005 (2:00 am)
Anybody can host a server, the big issue is getting clients connected to it. As mentioned earlier, the biggest issue is when a firewall/router is used.Generally a system is using a private IP address which is then NATed by the interconnect device (router/firewall). As Stephen said, clients generally have no problem when making connection OUT to the internet. A NAT device can not allow incoming connections automatically. An incoming packet has no mapping and cannot be forwarded to the correct device on the user network. A user could potentially setup direct mapping, but this can be tricky and has other problems associated with it.
The major function of a master server, when the game server is behind a NAT device, is to act as an introducer. The game server advertises itself to the master server. When this is done, all the public IP addressing and port information is remembered by the master server. When a client wants to make a connection it gets this info from the master server. The master server advises the game server that a client is about to connect and provides the public IP/port of the client to the game server. BOTH the game server and the client then attempt to send info to each other. This has the effect that the firewalls at each end have the correct mapping to allow the data through.
Where the game server is not directly visible to the Internet, there must be a master server to perform the function of an introducer.
If the game server has a public IP address then anyone who knows that address can connect provided they know what the correct address is. No master server is required under these circumstances.
Hope I didn't repeat too much of what Stephen said.....
#6
Ok, so just makeing sure that I understand you correctly here... If I made an option in the GUI to dirrectly connect to any IP adress, granted, the one that is runing the game, it won't have to show up in the query master to be connected to?
Well, then going off of that if that is the case, would it be possable to search through all of the people's IP adresses that are running the game? Withough a master server...
Seems logical to me, some clarafacation would be great though, Thanx, Max
04/01/2005 (7:39 am)
Quote:If the game server has a public IP address then anyone who knows that address can connect provided they know what the correct address is. No master server is required under these circumstances.
Ok, so just makeing sure that I understand you correctly here... If I made an option in the GUI to dirrectly connect to any IP adress, granted, the one that is runing the game, it won't have to show up in the query master to be connected to?
Well, then going off of that if that is the case, would it be possable to search through all of the people's IP adresses that are running the game? Withough a master server...
Seems logical to me, some clarafacation would be great though, Thanx, Max
#7
It should be possible to allow a client to keep a list of favorite servers to which they can attempt direct connections. Each client would be responsible for maintaining this list.
04/01/2005 (1:54 pm)
Without a master server an accurate list of IP addresses is almost impossible to maintain. Every client would need to keep a list of possible game servers. This would preclude any changes to existing servers and new servers could not be added. Without some form of master server there is no way of searching for game servers.It should be possible to allow a client to keep a list of favorite servers to which they can attempt direct connections. Each client would be responsible for maintaining this list.
#8
Yeah, I thought that would be the case, I've concidered ways around that, but I think it would be all too trickey, mainly because IP adress's are always updateing... possably some sort of php server? I've seen a few things about torque having implamented php... it would just have to involve searching the net for ip's find them, store them, and then update them into a text feild in the gui, and be updated every ten mins or so... Possable I'm sure not so sure if it would be a "simple" job though.
Ok, next question, probably much more fesable, I'm assuming that torque must have every client running the same port to connect to a server and comunicate? Well, my idea is would it be possable to have the torque game look at say port (example)"0 to 60000" and find a port that is open in the firewall, not nessacaraly be able to know if a port is opened or not, just use all of the ports in there to look for servers... Because I do know that every firewall has it's own specific port/s that are opened and all... I know that my port in the game is set to "28000" at the moment, just the default that comes with the FPS starter...
Thanx for all of your time, Max
04/01/2005 (2:52 pm)
Hey,Yeah, I thought that would be the case, I've concidered ways around that, but I think it would be all too trickey, mainly because IP adress's are always updateing... possably some sort of php server? I've seen a few things about torque having implamented php... it would just have to involve searching the net for ip's find them, store them, and then update them into a text feild in the gui, and be updated every ten mins or so... Possable I'm sure not so sure if it would be a "simple" job though.
Ok, next question, probably much more fesable, I'm assuming that torque must have every client running the same port to connect to a server and comunicate? Well, my idea is would it be possable to have the torque game look at say port (example)"0 to 60000" and find a port that is open in the firewall, not nessacaraly be able to know if a port is opened or not, just use all of the ports in there to look for servers... Because I do know that every firewall has it's own specific port/s that are opened and all... I know that my port in the game is set to "28000" at the moment, just the default that comes with the FPS starter...
Thanx for all of your time, Max
#9
For an application to be NAT safe it cannot be too strict about the ports being used. The client has no simple way of knowing what port is being used to forward the data on the Internet. NAT was designed to be as transparent to the client as possible. The data may be sent on port 28000 by the client, but the NAT device will most likely map it to a completely different port on the Internet. The destination port (the game server) is not modified, but the source port is. For example, if two clients behind the same firewall both try to send on port 28000 they cannot both use this port on the Internet. They can both have the same destination port, but the source port (as far as the Internet is concerned) would have to be different.
It is this mapping of ports that makes game servers behind firewalls problematic. Only a device on the Internet is able to see what the actual port numbers being used are. It is these 'real' addresses that the master server is aware of and can then tell both the client and the server.
I'm not quite sure what you are getting at with your next question.
Perhaps I should define some of my terms a bit better. I have been mixing firewall and NAT a bit loosely.
A firewall is a device which filters packets. Whether a packet is allowed or not is defined by a set of rules. The more strict the rules, the less packets are allowed through. A firewall does not modify the packets.
NAT is used to convert from one set of network addresses to another. In most cases, a private network has a single gateway into the Internet. This gateway uses NAT to map each private address/port to a public address/port on the Internet. The mapping is essential to have return traffic to the single Internet address forwarded back to the correct private network host.
In many cases the firewall and NAT function are both contained within the same device.
For an outgoing connection, there is no need to try and find an 'open port'. Provided you are sending to the correct port on the remote host, the NAT should handle all the mapping automatically. Trying to open 60000 ports and send a packet on each one is not the wisest of options :).
Hope this is of some help and that I am not just making things murkier!
04/01/2005 (4:07 pm)
I'm a little unclear on what you mean by 'searching the net'. There is no practical way of trying to discover what addresses are hosting a game server from each client. This is one of the main drivers behind the master server. A single point that everyone can use to register a server and for all clients to go when looking for servers. There really isn't any way around it. The master server is like google for your game.For an application to be NAT safe it cannot be too strict about the ports being used. The client has no simple way of knowing what port is being used to forward the data on the Internet. NAT was designed to be as transparent to the client as possible. The data may be sent on port 28000 by the client, but the NAT device will most likely map it to a completely different port on the Internet. The destination port (the game server) is not modified, but the source port is. For example, if two clients behind the same firewall both try to send on port 28000 they cannot both use this port on the Internet. They can both have the same destination port, but the source port (as far as the Internet is concerned) would have to be different.
It is this mapping of ports that makes game servers behind firewalls problematic. Only a device on the Internet is able to see what the actual port numbers being used are. It is these 'real' addresses that the master server is aware of and can then tell both the client and the server.
I'm not quite sure what you are getting at with your next question.
Perhaps I should define some of my terms a bit better. I have been mixing firewall and NAT a bit loosely.
A firewall is a device which filters packets. Whether a packet is allowed or not is defined by a set of rules. The more strict the rules, the less packets are allowed through. A firewall does not modify the packets.
NAT is used to convert from one set of network addresses to another. In most cases, a private network has a single gateway into the Internet. This gateway uses NAT to map each private address/port to a public address/port on the Internet. The mapping is essential to have return traffic to the single Internet address forwarded back to the correct private network host.
In many cases the firewall and NAT function are both contained within the same device.
For an outgoing connection, there is no need to try and find an 'open port'. Provided you are sending to the correct port on the remote host, the NAT should handle all the mapping automatically. Trying to open 60000 ports and send a packet on each one is not the wisest of options :).
Hope this is of some help and that I am not just making things murkier!
#10
Lol, yes, very true, it's isn't advisable to go searching through 60000 ports for servers, I just wish I could find some way for torque to like break through a firewall, or whatnot however TNL does it... But then that would really be defeating half of TNL wouldn't it? Ya, this is very much so a great bit of help, it has just been agravating me for I have been tinkering with this till 4 in the morning every night and can't seem to get it working no matter what I do... So, question, do you know of any way to work around a firewall? I mean, is there some salution to getting through whatever it may be that is stopping servers from being queryed?.. It just seems to me as though it is quite simple actually, just lots of little things are holding fast to the wrong things...
Hope I'm not being overly vaugh...
Thanks, Max
04/01/2005 (4:24 pm)
Hey,Lol, yes, very true, it's isn't advisable to go searching through 60000 ports for servers, I just wish I could find some way for torque to like break through a firewall, or whatnot however TNL does it... But then that would really be defeating half of TNL wouldn't it? Ya, this is very much so a great bit of help, it has just been agravating me for I have been tinkering with this till 4 in the morning every night and can't seem to get it working no matter what I do... So, question, do you know of any way to work around a firewall? I mean, is there some salution to getting through whatever it may be that is stopping servers from being queryed?.. It just seems to me as though it is quite simple actually, just lots of little things are holding fast to the wrong things...
Hope I'm not being overly vaugh...
Thanks, Max
#11
Lol, yes, very true, it's isn't advisable to go searching through 60000 ports for servers, I just wish I could find some way for torque to like break through a firewall, or whatnot however TNL does it... But then that would really be defeating half of TNL wouldn't it? Ya, this is very much so a great bit of help, it has just been agravating me for I have been tinkering with this till 4 in the morning every night and can't seem to get it working no matter what I do... So, question, do you know of any way to work around a firewall? I mean, is there some salution to getting through whatever it may be that is stopping servers from being queryed?.. It just seems to me as though it is quite simple actually, just lots of little things are holding fast to the wrong things...
Hope I'm not being overly vaugh...
Thanks, Max
04/01/2005 (5:30 pm)
Hey,Lol, yes, very true, it's isn't advisable to go searching through 60000 ports for servers, I just wish I could find some way for torque to like break through a firewall, or whatnot however TNL does it... But then that would really be defeating half of TNL wouldn't it? Ya, this is very much so a great bit of help, it has just been agravating me for I have been tinkering with this till 4 in the morning every night and can't seem to get it working no matter what I do... So, question, do you know of any way to work around a firewall? I mean, is there some salution to getting through whatever it may be that is stopping servers from being queryed?.. It just seems to me as though it is quite simple actually, just lots of little things are holding fast to the wrong things...
Hope I'm not being overly vaugh...
Thanks, Max
#12
I wanted to clarify Michael's statement here, since it could be taken slightly incorrectly for anyone that isn't a network guru:
You don't -have- to have a master server if:
A) You have a static IP address(es) (doesn't change over time) that is internet addressable (not in the 10.x.x.x or 192.168.x.x address spaces --I think that second is right, I always forget if I don't look it up each time),
B) Your server is set up on the platform that is assigned the address defined above. If you have a server that is assigned an internal network address, then you also need to do C:
C) Set up proxy serving on the platform (computer, router, etc.) that has that publically addressable address. This is basically an explicit "forwarder" which tells the router:
--hey, if someone asks for port 28000 at your ip address, connect them to 10.x.x.x on port 28000 instead--and remember that you told them that. In other words, a proxy turns a normally "not publically addressable" network service (such as TGE on an internal network address:port) into one that is publically addressable, using a single IP:port.
I have to go back to the real purpose of the "Master Server" concept here as well--the purpose of a Master Server is to allow players to start up non-permenent, non-static IP address games on their home computers, and let others across the internet connect to them. It is NOT designed to allow for a game like Lore:Dark Horizons, or any other game that have their own permanent, addressable servers to "advertise". It can of course be twisted to do that, but that's not the reason for it.
If you find yourself trying to twist the master server concept for a permanently hosted server, you might want to re-think your design.
04/01/2005 (5:31 pm)
Quote:Where the game server is not directly visible to the Internet, there must be a master server to perform the function of an introducer.
I wanted to clarify Michael's statement here, since it could be taken slightly incorrectly for anyone that isn't a network guru:
You don't -have- to have a master server if:
A) You have a static IP address(es) (doesn't change over time) that is internet addressable (not in the 10.x.x.x or 192.168.x.x address spaces --I think that second is right, I always forget if I don't look it up each time),
B) Your server is set up on the platform that is assigned the address defined above. If you have a server that is assigned an internal network address, then you also need to do C:
C) Set up proxy serving on the platform (computer, router, etc.) that has that publically addressable address. This is basically an explicit "forwarder" which tells the router:
--hey, if someone asks for port 28000 at your ip address, connect them to 10.x.x.x on port 28000 instead--and remember that you told them that. In other words, a proxy turns a normally "not publically addressable" network service (such as TGE on an internal network address:port) into one that is publically addressable, using a single IP:port.
I have to go back to the real purpose of the "Master Server" concept here as well--the purpose of a Master Server is to allow players to start up non-permenent, non-static IP address games on their home computers, and let others across the internet connect to them. It is NOT designed to allow for a game like Lore:Dark Horizons, or any other game that have their own permanent, addressable servers to "advertise". It can of course be twisted to do that, but that's not the reason for it.
If you find yourself trying to twist the master server concept for a permanently hosted server, you might want to re-think your design.
#13
I hope I'm not being an ass about this in anyways... Well, I'm oging to try out that whole IP connect thing tonight, and see how that goes. I hope I'm not understanding you uncorrectlyor reading it wrong and getting the wrong message more of, But the problem is really, that my game isn't picking up anything... Not so sure if this has anything to do with it, but the server shoes up in the gnimetech garagegames master server thingy, but it's name is listed as "Timed Out" while the dedicated server is running.... I've gone over masses of forum threads dealing with problems with this, but it seems as though all of them are resolved with the people opening the ports on their firewalls or routers... That just seems too complacated for the average game player to do... Playing games really doesn't invollve learning much or doing much at all, I just want it to be the same way for finding servers, not messing around with ports and IP's...
Do you know what the Realm Wars uses for it's servers?
Well, long story short if I managed to completly go roundabout with all of that, I want to find a good way to have all people who run the game be hosting a server, and have clients be able to find that server with no troubble whatsoever...
Thats basically what TNL is for right? Hah, I think this is almost a pre TNL thread for me.... correct me if I'm wrong though....
Thank you sooo much for all of your help, I am on the path to being a network guru like yourself, Max
04/01/2005 (7:29 pm)
Hey,I hope I'm not being an ass about this in anyways... Well, I'm oging to try out that whole IP connect thing tonight, and see how that goes. I hope I'm not understanding you uncorrectlyor reading it wrong and getting the wrong message more of, But the problem is really, that my game isn't picking up anything... Not so sure if this has anything to do with it, but the server shoes up in the gnimetech garagegames master server thingy, but it's name is listed as "Timed Out" while the dedicated server is running.... I've gone over masses of forum threads dealing with problems with this, but it seems as though all of them are resolved with the people opening the ports on their firewalls or routers... That just seems too complacated for the average game player to do... Playing games really doesn't invollve learning much or doing much at all, I just want it to be the same way for finding servers, not messing around with ports and IP's...
Do you know what the Realm Wars uses for it's servers?
Well, long story short if I managed to completly go roundabout with all of that, I want to find a good way to have all people who run the game be hosting a server, and have clients be able to find that server with no troubble whatsoever...
Thats basically what TNL is for right? Hah, I think this is almost a pre TNL thread for me.... correct me if I'm wrong though....
Thank you sooo much for all of your help, I am on the path to being a network guru like yourself, Max
Torque 3D Owner Stephen Zepp
The reason for this is that by default, a majority of firewalls will allow any outgoing connections, but severely restrict incoming connections. For a server, this means that you must allow connections to be opened to your server through your firewall. For players, in some cases you can leave the firewall alone, but for many firewalls, a UDP service request is not considered a "connection", and therefore you must unblock the firewall's restriction for the UDP port(s) assigned for game traffic. FYI, default for TAP is 28000.
While I haven't studied how the TNL master server works in depth, the basic concept is that the master server is open to connections (no firewall, or a proxy to the server), and waits for a server to "register". Since the game server opened a connection to the master server, the game server's firewall allows return data across that port (normally).
Next, a player asks for a connection to a game server by telling the master server "hey, tell this server to let me in". The master server relays the request, and then the game server opens a connection to the client.
Now, I admit, I don't understand the part about how that connection can "break through" any client firewall, so either my understanding of how the master server works is incomplete, or it actually can't do that. Anyone that knows for sure able to answer that side of things, or correct me if my understanding is out in left field? :)
Edit: and when your dedicated server is on an internal network (which is standard for home networks), your router must proxy the dedicated server (or use the master server), otherwise external connections don't know who to talk to--all they are able to connect to via IP address is the router--none of your internal network computers have internet locateable addresses. The technical description of this is NAT, or Network Address Translation, if you want to do research on it outside GG.