Using TNL in TGE (and TSE in the future)
by OneST8 · in Torque Game Engine · 06/17/2006 (10:10 am) · 6 replies
This thread is started based on the reply from the brilliant Ben Garney found here with relevant part quoted below:
I am very interested in using TNL and all of it's fabulous features within TGE (and TSE once stable) but am having a bit of trouble figuring out where to start. I have no fears of massive amounts of labour intensive coding and am actually very much looking forward to it, however I find myself lacking the experience with TGE to know where to get the ball rolling from. I mean like it's a huge mountain of work so where to begin? Normally it's "find the top of the mountain and let the ball roll down" but figuring out where everything starts in respect to networking with TGE has been a little confusing for me.
Any and all help is greatly appreciated and I'm not against sharing patches/tutorials of the work I do in this regard. In fact, I'd be happy to put together a TNL'd Stock TGE for GG to do with as they see fit.
PS: One thing I will state is that after having gone through the examples and sources of TNL, I am absolutely impressed! I mean like it's a serious "WOW" in terms of the good stuff it provides! Regardless of how many people on the GG IRC channel told me "you don't need it", I am very happy to be a proud owner of an Indy License for it!
2) You'd have to rip out all the existing net code and rewrite it using TNL. It wouldn't be really hard, but definitely labor intensive. Same for TSE.
I am very interested in using TNL and all of it's fabulous features within TGE (and TSE once stable) but am having a bit of trouble figuring out where to start. I have no fears of massive amounts of labour intensive coding and am actually very much looking forward to it, however I find myself lacking the experience with TGE to know where to get the ball rolling from. I mean like it's a huge mountain of work so where to begin? Normally it's "find the top of the mountain and let the ball roll down" but figuring out where everything starts in respect to networking with TGE has been a little confusing for me.
Any and all help is greatly appreciated and I'm not against sharing patches/tutorials of the work I do in this regard. In fact, I'd be happy to put together a TNL'd Stock TGE for GG to do with as they see fit.
PS: One thing I will state is that after having gone through the examples and sources of TNL, I am absolutely impressed! I mean like it's a serious "WOW" in terms of the good stuff it provides! Regardless of how many people on the GG IRC channel told me "you don't need it", I am very happy to be a proud owner of an Indy License for it!
#2
First off it's the encryption, second off, after having gone through the example code and libs, I've grown on the idea of the separated networking layer. This separation would open the door for a lot of secondary applications and offloaded server functionality for the game I'm working on. Also, the connection handshaking and spoof protection that brings is absolutely invaluable. Nevermind all the master server/client code.
One of the major roadblocks I've encountered with TGE so far is that we've identified at least 3 ISPs where the stock TGE client cannot connect to a publicly hosted (non-firewalled) dedicated server. The client can do an initial query of our master server but fails completely to ping the dedicated server at all. Even directly connecting/querying the dedicated server is blocked somehow. At this point I'm suspecting the ISP's transparent proxy server(s) and/or stringent firewalling to be the culprit. We've tried a wide range of different ports and ideas, yet still no luck.
Now, I suppose I should elaborate a bit about the game and our goals with the game. Firstly, it's a multiplayer-only RPG. Secondly, we're endeavouring to support a player base over the 20,000 range. Granted we're not naive enough to believe that we can reach those numbers on any single machine, but rather we want to abstract as much of the RPG processing and offload it to secondary servers (in a clustered fashion). Now, on top of all this, we want to allow for custom worlds to be built but there are some serious requirements with respect to the integration of these custom worlds within the "master" network.
In order to disclose any further details regarding the game, I'll have to request that the interested parties sign an NDA and fax it back to me. At which point I'd be more than willing to share our executive summary and design document. This game has been in the planning stages for over two years and we're now beginning the process of constructing a prototype. Obviously we're using the wonderful technologies provided by Garage Games and I am fully confident in all of the purchasing decisions we've made so far with your company. The group of individuals involved in this monumental project have extensive experience with managing and maintaining a large and active community on a number of different levels.
Again, any and all help is greatly appreciated and will result in reciprocated efforts for the general Garage Games community.
06/19/2006 (3:23 pm)
That's a fair question.First off it's the encryption, second off, after having gone through the example code and libs, I've grown on the idea of the separated networking layer. This separation would open the door for a lot of secondary applications and offloaded server functionality for the game I'm working on. Also, the connection handshaking and spoof protection that brings is absolutely invaluable. Nevermind all the master server/client code.
One of the major roadblocks I've encountered with TGE so far is that we've identified at least 3 ISPs where the stock TGE client cannot connect to a publicly hosted (non-firewalled) dedicated server. The client can do an initial query of our master server but fails completely to ping the dedicated server at all. Even directly connecting/querying the dedicated server is blocked somehow. At this point I'm suspecting the ISP's transparent proxy server(s) and/or stringent firewalling to be the culprit. We've tried a wide range of different ports and ideas, yet still no luck.
Now, I suppose I should elaborate a bit about the game and our goals with the game. Firstly, it's a multiplayer-only RPG. Secondly, we're endeavouring to support a player base over the 20,000 range. Granted we're not naive enough to believe that we can reach those numbers on any single machine, but rather we want to abstract as much of the RPG processing and offload it to secondary servers (in a clustered fashion). Now, on top of all this, we want to allow for custom worlds to be built but there are some serious requirements with respect to the integration of these custom worlds within the "master" network.
In order to disclose any further details regarding the game, I'll have to request that the interested parties sign an NDA and fax it back to me. At which point I'd be more than willing to share our executive summary and design document. This game has been in the planning stages for over two years and we're now beginning the process of constructing a prototype. Obviously we're using the wonderful technologies provided by Garage Games and I am fully confident in all of the purchasing decisions we've made so far with your company. The group of individuals involved in this monumental project have extensive experience with managing and maintaining a large and active community on a number of different levels.
Again, any and all help is greatly appreciated and will result in reciprocated efforts for the general Garage Games community.
#3
Note that the connection handshaking and spoof protection is actually part of stock Torque networking, but the remainder of what you state isn't--and TNL does provide a better structure for offloading processing requirements to other threads/processes/applications.
I have noticed the issues with "firewall punching/non-cooperative ISPs" myself, and while I haven't nailed down the exact issues, I have a feeling that it's a bandwidth limiting mechanism some ISP's are using (use of "non-approved" ports for example has by some ISP's been considered unacceptable--I've even had issues with Comcast deciding that my SSH sessions into remote servers was a bandwidth drain and they would disconnect my after 60 seconds of even active use. Not saying the stock Torque implementation of firewall utilization isn't a culprit as well, but that code hasn't been touched much in quite a while, so it appears that the ISPs are getting more strigent again--just my personal opinion of course!
It sounds like you've done your homework, and since you have put in the work to analyze the solutions required, I don't see much of an issue for you in performing the steps necessary to remove stock networking and replace it with TNL...in fact, I'd be interested in hearing your progress, either here or in email!
06/19/2006 (4:19 pm)
With that summary and amount of (obvious) pre-research, I can say that TNL is certainly a viable option, and slightly more applicable to some of the circumstances you describe.Note that the connection handshaking and spoof protection is actually part of stock Torque networking, but the remainder of what you state isn't--and TNL does provide a better structure for offloading processing requirements to other threads/processes/applications.
I have noticed the issues with "firewall punching/non-cooperative ISPs" myself, and while I haven't nailed down the exact issues, I have a feeling that it's a bandwidth limiting mechanism some ISP's are using (use of "non-approved" ports for example has by some ISP's been considered unacceptable--I've even had issues with Comcast deciding that my SSH sessions into remote servers was a bandwidth drain and they would disconnect my after 60 seconds of even active use. Not saying the stock Torque implementation of firewall utilization isn't a culprit as well, but that code hasn't been touched much in quite a while, so it appears that the ISPs are getting more strigent again--just my personal opinion of course!
It sounds like you've done your homework, and since you have put in the work to analyze the solutions required, I don't see much of an issue for you in performing the steps necessary to remove stock networking and replace it with TNL...in fact, I'd be interested in hearing your progress, either here or in email!
#4
Work has officially begun in our project to merge TNL into TGE 1.5 (with AFX). We've got one programmer working on it full-steam and then I'm helping him out however I can while trying to get other (non-related) systems implemented at the same time so that we're still meeting our project deadlines.
We've hit a few hurdles already but seem to be able to find solutions to the problems on our own based on my knowledge of TGE and my cohort's knowledge of TNL/C++.
No working code yet as migrating to TNL isn't something one can just do in a weekend but I'll try to post updates here when we make some tangible progress.
01/21/2007 (10:32 pm)
Just posting a minor update to this (forgotten) thread...Work has officially begun in our project to merge TNL into TGE 1.5 (with AFX). We've got one programmer working on it full-steam and then I'm helping him out however I can while trying to get other (non-related) systems implemented at the same time so that we're still meeting our project deadlines.
We've hit a few hurdles already but seem to be able to find solutions to the problems on our own based on my knowledge of TGE and my cohort's knowledge of TNL/C++.
No working code yet as migrating to TNL isn't something one can just do in a weekend but I'll try to post updates here when we make some tangible progress.
#5
Cheers,
Rich Wermske
05/02/2007 (4:48 pm)
Quote:In order to disclose any further details regarding the game, I'll have to request that the interested parties sign an NDA and fax it back to me. At which point I'd be more than willing to share our executive summary and design document.Ok. As an interested party, I'll bite. Is the non-disclosure available for consideration online or do we direct the request to you (IM, email, IRC) to obtain it?
Cheers,
Rich Wermske
#6
I seem to have mistaken the IRC convo that was happening between you and I last night. I had thought I was coming into an existing conversation between you and someone else, rather than you directing it at me just as I'd logged in.
Anyways, regarding the NDA and such we'll have to talk in private a bit but things are insanely hectic at this time for our team. Getting all setup and running with the Prairie Games MMO Kit has opened up some important near-future goals for us and as such has been consuming 99% of our time.
So, when I've got some time and we're both on IRC; let's chat and take things from there.
05/02/2007 (11:18 pm)
Heya Rich,I seem to have mistaken the IRC convo that was happening between you and I last night. I had thought I was coming into an existing conversation between you and someone else, rather than you directing it at me just as I'd logged in.
Anyways, regarding the NDA and such we'll have to talk in private a bit but things are insanely hectic at this time for our team. Getting all setup and running with the Prairie Games MMO Kit has opened up some important near-future goals for us and as such has been consuming 99% of our time.
So, when I've got some time and we're both on IRC; let's chat and take things from there.
Torque 3D Owner Stephen Zepp
Sometimes it's not completely clear--but TNL is simply an extracted version of the stock Torque networking, cleaned up for use as a library instead of an integrated networking layer, and given a bit more flexibility and functionality.