Game Networking Schemes
by Matt Fairfax · in Technical Issues · 02/15/2001 (12:05 pm) · 36 replies
Since I asked for this forum, I figured I would start the discussions:
Questions for anyone who has made a networked game:
How did you implement your networking scheme? How well did it work? How well did it handle latency? What resources did you use to help you create/implement your scheme? How easily can it be ported to other platforms?
How did the genre of your game effect your networking scheme?
I'm not asking for anyone to give away any company secrets, but anything you could post would be a big help! Assume that the readers of this has basic knowledge of sockets (BSD and Winsock) since it is fairly easy to find information on them. The biggest goal of this post is to find out how networking is being implemented in games. Some topics you may want to talk about are: TCP or UDP, Synchronous or Asynchronous, Blocking or Non-Blocking, ...
Thank you!
Questions for anyone who has made a networked game:
How did you implement your networking scheme? How well did it work? How well did it handle latency? What resources did you use to help you create/implement your scheme? How easily can it be ported to other platforms?
How did the genre of your game effect your networking scheme?
I'm not asking for anyone to give away any company secrets, but anything you could post would be a big help! Assume that the readers of this has basic knowledge of sockets (BSD and Winsock) since it is fairly easy to find information on them. The biggest goal of this post is to find out how networking is being implemented in games. Some topics you may want to talk about are: TCP or UDP, Synchronous or Asynchronous, Blocking or Non-Blocking, ...
Thank you!
About the author
I am a Game Designer at PopCap who has worked on PvZ Adventures, PvZ2, Peggle Blast, and Bejeweled Skies. I am an ex-GarageGames employee who helped ship TGE, TGEA, Torque 3D, and Constructor.
#2
03/16/2001 (10:44 pm)
Ok, I've been knee deep in network programming for the last two weeks and I have some questions for those who have gone ahead of me =) I am trying to build my net code with an eye towards portability so I started out using select() to tell me if there was data waiting to be read. This worked fine but it was a huge cycle hog. My framerate halved. This is simply not acceptable. So I started looking for alternatives. I learned of two ways of doing it: using a MSG_PEEK in a recv() or recvfrom() to determine if there is any data waiting to be read or using ioctlsocket() or ioctl() to determine if there is any data waiting. Both ways only incure the slightest slowdown and seem to work pretty well. My only concern is that I keep reading in different network programming guides that this is very bad form and should be avoided b/c of unreliability and b/c it locks the network buffer while the call is excuted so that it can't receive any new information. Another alternative I have been considering is using multi-threaded networking but I haven't done any testing on that yet. My question to you network programmers is what did you find to be the best way to do your net code for your game? Any thoughts on what I have been messing with? 
#3
Well, we had to use the select() method with Worms World Party on the DC, I went through a few different methods, but in the end the select call was the only one that was working reliably.
As for cross platform, well, Ive seen some hideous messes :)), at the end of the day though, I wouldnt worry about the specifics of which calls to use, more about the interface and how your game interacts with it.My point being that you cant necassarily rely on a single implementation, so try and abstract the implementation away from the interface.
Ive gone through a few different things with the last few network games Ive done, starting with winsock, then onto directplay, then back to winsock. I wrote a directplay lobbyable application (quite possibly the first one available in a commercial game) and was hit quite badly by lack of features support in the directplay of the time. Luckily I had wrapped the network code into a nice class for our other programmers, and I was able to rewrite the internals to use part directplay, and part winsock without changing thier code.
One thing keeps coming back to me though, keep the networking interface as SIMPLE as you can. Connect, Send, Receive, Isconnected and Disconnect. Something that simple as an interface. From there on, well, build up in layers, I did a simple little message class that took care of marshalling (i.e. flattening the data into packets), it also makes sense to build your network code into modules that can be turned on and off in realtime.
Its kind of a big thing to try and explain in a single reply.. :))
03/18/2001 (12:13 pm)
Hi Matt,Well, we had to use the select() method with Worms World Party on the DC, I went through a few different methods, but in the end the select call was the only one that was working reliably.
As for cross platform, well, Ive seen some hideous messes :)), at the end of the day though, I wouldnt worry about the specifics of which calls to use, more about the interface and how your game interacts with it.My point being that you cant necassarily rely on a single implementation, so try and abstract the implementation away from the interface.
Ive gone through a few different things with the last few network games Ive done, starting with winsock, then onto directplay, then back to winsock. I wrote a directplay lobbyable application (quite possibly the first one available in a commercial game) and was hit quite badly by lack of features support in the directplay of the time. Luckily I had wrapped the network code into a nice class for our other programmers, and I was able to rewrite the internals to use part directplay, and part winsock without changing thier code.
One thing keeps coming back to me though, keep the networking interface as SIMPLE as you can. Connect, Send, Receive, Isconnected and Disconnect. Something that simple as an interface. From there on, well, build up in layers, I did a simple little message class that took care of marshalling (i.e. flattening the data into packets), it also makes sense to build your network code into modules that can be turned on and off in realtime.
Its kind of a big thing to try and explain in a single reply.. :))
#4
Under windows a large server will be better implemented using io completion ports and multiple threads. If you only have a small number of sockets you could use WaitForMultipleObjects calls. The beauty of both of these is your thread waits until something shows up, you don't get those nasty polling processor surges. I would (personally) try and stay away from anything involving windows message pumps.
Under different flavors or unix select() is your only real option. I would try multiple threads. I.e. Thread 1 waits on the select statement and pulls messages out of the kernel while Thread 2 does game stuff and receives the processed network messages from Thread 1. I imagine right now you're have your code wait for a short duration on the select statement then continuing your game code. Not knowing your code I imagine adding a second thread which only does the select would buy you quite a bit.
Gideon
03/18/2001 (4:30 pm)
The use of multiple implementations is interesting and probably the most effective. Under windows a large server will be better implemented using io completion ports and multiple threads. If you only have a small number of sockets you could use WaitForMultipleObjects calls. The beauty of both of these is your thread waits until something shows up, you don't get those nasty polling processor surges. I would (personally) try and stay away from anything involving windows message pumps.
Under different flavors or unix select() is your only real option. I would try multiple threads. I.e. Thread 1 waits on the select statement and pulls messages out of the kernel while Thread 2 does game stuff and receives the processed network messages from Thread 1. I imagine right now you're have your code wait for a short duration on the select statement then continuing your game code. Not knowing your code I imagine adding a second thread which only does the select would buy you quite a bit.
Gideon
#5
03/19/2001 (6:51 am)
Hey Phil! Glad to see you on GG! I have been moving towards a multiple implementation scheme for a while but I wanted to hear from some other developers first =). I am going to stick with ioctlsocket() for the time being (just can't ignore the framerate hit of select()) but if it turns out to be unreliable, I am thinking of moving towards a multi-threaded scheme using select(). I can't wait to get a hold of the V12's networking code and see how they did it! I am still very interested in hearing anything about how someone did networking in their game. This seems to be a subject that a lot of people are tight-lipped about. Once I have a bit more experience and code I may start a pratical, code-filled tutorial on doing networking in games. I will probably wait until I have seen the V12 net code so that I can include anything I learn from it.
#6
Well, I guess most people dont think its interesting for others to hear about how they did it.
I'm more interested in seeing how they managed to make a stable server platform out of windows :)), we had a nightmare time with winsock under NT (or rather *I* had a nightmare time), basically, I started using MFC sockets.. that was a REALLY bad idea. I didnt keep any stats from when we started the Worms 2 network service, but basically, it was impossibly unstable.
I rewrote the server again, over a period of a month, basically hacking out the MFC ness, but still it was crashing (other than naive buffer ovverrun exploits that hackers like to throw at ya), To this day i feel bad about the Worms 2 server, because it wasnt really reliable enough, in the end we wrote a little "server monitor" program that restarted it if it died.
But you live and learn. What I learnt was quite important, but the main thing was this:
NEVER try and write network code for a game, interface code for a game,protocols and lobbying systems for a game AND the server for the game in a 9 month period and THEN start hacking away at the server after 6 months of 18 hour days.
Hehe, if ever there was a feeling of burnout after a project, well, this was it.
Phil.
03/20/2001 (3:24 pm)
Hi Matt,Well, I guess most people dont think its interesting for others to hear about how they did it.
I'm more interested in seeing how they managed to make a stable server platform out of windows :)), we had a nightmare time with winsock under NT (or rather *I* had a nightmare time), basically, I started using MFC sockets.. that was a REALLY bad idea. I didnt keep any stats from when we started the Worms 2 network service, but basically, it was impossibly unstable.
I rewrote the server again, over a period of a month, basically hacking out the MFC ness, but still it was crashing (other than naive buffer ovverrun exploits that hackers like to throw at ya), To this day i feel bad about the Worms 2 server, because it wasnt really reliable enough, in the end we wrote a little "server monitor" program that restarted it if it died.
But you live and learn. What I learnt was quite important, but the main thing was this:
NEVER try and write network code for a game, interface code for a game,protocols and lobbying systems for a game AND the server for the game in a 9 month period and THEN start hacking away at the server after 6 months of 18 hour days.
Hehe, if ever there was a feeling of burnout after a project, well, this was it.
Phil.
#7
I'm embarassed to say I'm not familiar with networked Worms. How big is your server? What did you replace MFC with? From my experience MFC is crap for developing large (or small) scale servers. I've spent a lot of time developing large servers servers on NT and 2k. Some of the code I inherited used MFC and proved to be essentially useless. And under high-load/multiply threaded environments extremely unstable. I'm not entirely comfortable with the asynchronous WSA calls because of the overhead of the message pump. Did you switch to WSA async io or the standard socket API?
I see that neither you nor Matt seemed to be into the idea of io completion ports and a separate thread to handle all the networking code. Using select without a separate thread seems like a painful idea. Besides async io using WSA which requires you to have a message pump what other option would there be? Even under unix it's fairly common to fork a separate thread to handle network activity. I'm not super familiar with thread behavior under 98 and ME so I may be missing something here.
Gideon
03/21/2001 (10:22 am)
Phil,I'm embarassed to say I'm not familiar with networked Worms. How big is your server? What did you replace MFC with? From my experience MFC is crap for developing large (or small) scale servers. I've spent a lot of time developing large servers servers on NT and 2k. Some of the code I inherited used MFC and proved to be essentially useless. And under high-load/multiply threaded environments extremely unstable. I'm not entirely comfortable with the asynchronous WSA calls because of the overhead of the message pump. Did you switch to WSA async io or the standard socket API?
I see that neither you nor Matt seemed to be into the idea of io completion ports and a separate thread to handle all the networking code. Using select without a separate thread seems like a painful idea. Besides async io using WSA which requires you to have a message pump what other option would there be? Even under unix it's fairly common to fork a separate thread to handle network activity. I'm not super familiar with thread behavior under 98 and ME so I may be missing something here.
Gideon
#8
Well, we had to use the select call polling method with the DC because it didnt have any other working method :)), it was placed in a thread, but still, the software modem of the DC really was a cycle hog.
The MFC stuff was me being stupid really, I figured it would actually work. In the end, I basically wrote a threaded server using blocking calls. One thing I did wrong was that I used a thread per client, which for our expected client usage was ALMOST ok, but unfortunately the thread overhead was quite large.
If I were to do the same thing again, I'd definitely go with completion ports and a thread pool, and write the server as a service rather than an app.
But to be honest, I'd basically write it for linux/freebsd before I'd do it for windows, because I dislike the overhead of windows GUI and threading compared to a similar linux box.
I'm definitely interested in seeing the V12 net code, its obviously more complex than my stuff for Worms (lets face it, worms wasnt pushing the envelope much in terms of network code :))
Phil.
03/21/2001 (11:07 am)
Hi Gideon,Well, we had to use the select call polling method with the DC because it didnt have any other working method :)), it was placed in a thread, but still, the software modem of the DC really was a cycle hog.
The MFC stuff was me being stupid really, I figured it would actually work. In the end, I basically wrote a threaded server using blocking calls. One thing I did wrong was that I used a thread per client, which for our expected client usage was ALMOST ok, but unfortunately the thread overhead was quite large.
If I were to do the same thing again, I'd definitely go with completion ports and a thread pool, and write the server as a service rather than an app.
But to be honest, I'd basically write it for linux/freebsd before I'd do it for windows, because I dislike the overhead of windows GUI and threading compared to a similar linux box.
I'm definitely interested in seeing the V12 net code, its obviously more complex than my stuff for Worms (lets face it, worms wasnt pushing the envelope much in terms of network code :))
Phil.
#9
03/21/2001 (1:54 pm)
Does anyone know where I can find some good references on using multi-threading and completion ports and the like? I am definitely interested in trying out this approach. I know that the V12 net code is going to blow my stuff outta the water but I figured I would keep plugging away on my own so that I will have learned all the basics really well =) BTW: I tried MFC netowrking a while back *shudder* and it wasn't fun.
#10
As for finding IOCompletion port references, check out MSDN on the microsoft site. Here are a few links. Best bet is to do a search on IOCP.
support.microsoft.com/support/kb/articles/Q192/8/00.ASP
msdn.microsoft.com/library/periodic/period00/Winsock.htm
If anyone has had success using IOCP's, I'd be interested to hear about it.
03/21/2001 (6:57 pm)
I've recently been doing some server work for a non-game project where I tried using IOCompletion ports. They seem like the best thing to use, but I kept on running into this obscure bug where I would get random stack corruption that would invariably set my stack pointer to 16. I switched to select() and the problem disappeared. When I have some time though, I REALLY want to find out what the heck was going on. Anyway, thats another story...As for finding IOCompletion port references, check out MSDN on the microsoft site. Here are a few links. Best bet is to do a search on IOCP.
support.microsoft.com/support/kb/articles/Q192/8/00.ASP
msdn.microsoft.com/library/periodic/period00/Winsock.htm
If anyone has had success using IOCP's, I'd be interested to hear about it.
#11
03/21/2001 (8:00 pm)
I have been looking through some of the MSDN documentation about i/o completion ports and it seems to be indicating that I will have trouble using completion ports on Win9x if I can use them at all. Does anyone who has used them know anything about this? Here is basically what I am thinking about using multi-threading with networking: Have a thread that executes select() on the sockets and if there is data wating to be read it reads it off and copies it to a queue where the data waits to be processed by the main thread. Am I on the right track here or way off base. I am going to spend a lot of time this weekend and next week profiling some of the different networking schemes I have learned about to see what is the fastest.
#12
Another, probably simpler way is to just make a thread that blocks on select() for all of your sockets, and when it gets data, just read off the sockets it returns and dump them into a queue. Your main game code can just check this queue for any data that comes in and process it normally. Use another queue to hold all your outgoing packets and a thread that takes the packets off that queue and sends them.
Either of these should work fine for game servers in the 3D genre. They tend to do a lot of work so I think the scaling issue will hit your CPU a lot harder than the networking code.
03/22/2001 (9:00 am)
If you are looking for your server to support Win9x, you are not going to be able to use IO Completion Ports. How many connection are you planning to support? One option is to use the overlapped IO functions in Winsock2 for your IO. You can define events for each of your send and receive calls in the overlapped structures and use WaitForMultipleObjects to gather all of your IO completions into one place. This should act similar to an IO completion port but is not quite as scalable to large numbers of sockets. I've never tried this so I don't know for certain if this works well under 9x. The nice thing about the WSAReceive overlapped function is the driver will use your buffers for the data it receives. Otherwise, it uses its own internal buffers and does a memcpy into the one that you pass in.Another, probably simpler way is to just make a thread that blocks on select() for all of your sockets, and when it gets data, just read off the sockets it returns and dump them into a queue. Your main game code can just check this queue for any data that comes in and process it normally. Use another queue to hold all your outgoing packets and a thread that takes the packets off that queue and sends them.
Either of these should work fine for game servers in the 3D genre. They tend to do a lot of work so I think the scaling issue will hit your CPU a lot harder than the networking code.
#13
It's difficult to comment on your specific implementation without some usage information.
I'm getting the impression that your looking at a peer to peer model?
Or is it a dedicated server? If it is a peer to peer situation are you looking at n-way communication (i.e. every peer is connected to every other peer) or are trying to reduce the number of connections in some way? N-way is fine if you limit the number of connections each box can make. It's not so good if you allow unlimited connections. The different models that have been suggested will be applicable in different situations. IOCP stuff probably isn't a great idea for small scale servers. It really pays off when you have the potential for lots of sockets and a pool of threads serving them.
I'm interested in hearing about your testing. Do you feel comfortable posting your results and methodology?
Cheers,
Gideon
03/26/2001 (10:07 am)
Matthew, It's difficult to comment on your specific implementation without some usage information.
I'm getting the impression that your looking at a peer to peer model?
Or is it a dedicated server? If it is a peer to peer situation are you looking at n-way communication (i.e. every peer is connected to every other peer) or are trying to reduce the number of connections in some way? N-way is fine if you limit the number of connections each box can make. It's not so good if you allow unlimited connections. The different models that have been suggested will be applicable in different situations. IOCP stuff probably isn't a great idea for small scale servers. It really pays off when you have the potential for lots of sockets and a pool of threads serving them.
I'm interested in hearing about your testing. Do you feel comfortable posting your results and methodology?
Cheers,
Gideon
#14
Sorry about this, the threading model in these forums isn't the best, nor is the quotation system.
But, you wrote:
>> But to be honest, I'd basically write it for
>> linux/freebsd before I'd do it for windows, because I >> dislike the overhead of windows GUI and threading
>> compared to a similar linux box.
I agree that (to my Unix-grown way of thinking) the Unix networking seems easier and without a lot of the overhead that Win32 forces on the programmer. But I do like the thought that's gone in to Win32 threading stuff. IOCP's are really nice for big servers. The speed of using critical sections in Win32 is excellent (no context hit unless you really need it). Some of the new darkside features on 2k seems alright for big server stuff.
I'm interested in hearing what you like about threading under linux or where you feel the big hits are in Win32 development for servers.
I do like the Solaris model of Threads versus LWP's. It seems to provide a lot of possibilities and extremely lightweight thread creation (or is that LWP, I've been known to get the terms reversed). I'm going to run over and check out the Linux 2.4 threading changes. I must confess that I've been immersed in Windows for the last year and haven't kept up on my Linux and BSD info.
I'm definitely excited to check out the V12 networking code (I don't know if that's sad or not).
Gideon
03/26/2001 (10:17 am)
Phil,Sorry about this, the threading model in these forums isn't the best, nor is the quotation system.
But, you wrote:
>> But to be honest, I'd basically write it for
>> linux/freebsd before I'd do it for windows, because I >> dislike the overhead of windows GUI and threading
>> compared to a similar linux box.
I agree that (to my Unix-grown way of thinking) the Unix networking seems easier and without a lot of the overhead that Win32 forces on the programmer. But I do like the thought that's gone in to Win32 threading stuff. IOCP's are really nice for big servers. The speed of using critical sections in Win32 is excellent (no context hit unless you really need it). Some of the new darkside features on 2k seems alright for big server stuff.
I'm interested in hearing what you like about threading under linux or where you feel the big hits are in Win32 development for servers.
I do like the Solaris model of Threads versus LWP's. It seems to provide a lot of possibilities and extremely lightweight thread creation (or is that LWP, I've been known to get the terms reversed). I'm going to run over and check out the Linux 2.4 threading changes. I must confess that I've been immersed in Windows for the last year and haven't kept up on my Linux and BSD info.
I'm definitely excited to check out the V12 networking code (I don't know if that's sad or not).
Gideon
#15
03/26/2001 (11:29 am)
Actually my current needs are pretty lite. I am developing a newer version of Scorched Earth (it is ironic that Phil (Worms) is involved in this thread isn't it =) so my networking requirements are pertty small. It is turn-based with a lot of idle time so I am definitely not stressing the bandwidth. I am however trying to build a fairly flexible networking model so that I can use it for later projects which could include a first person shooter (very bandwidth intensive). I am looking at a server/client model with one player hosting and participating in the game along with a small number of clients. Pretty much along the same lines of your standard Quake game. I may add a dedicated server executable but that is not a concern right now. My biggest concern with the networking is that it not cause a big hit in my framerate b/c I am targetting fairly low-end machines. My ultimate goal would be for a low-lag game over a modem (server and clients). I didn't get a chance to do any profilling this weekend (life interrupted as usual =) but I will try to get some results posted here soon.
#16
Gideon: the biggest issue I found was that there was some inherent memory leak in accept called under NT4 at the time, to be honest I shouldnt really
blame NT too much, I never really had time to play with the thing, so it was basically my first effort that shipped (i.e. almost no profiling, almost no load testing etc).
My main problem was that the threading was pretty expensive in terms of a thread per connection. The amount of context switching was too much,
Ive since learnt that its better to use a thread pool with IOCP. But at the time, I was restricted to just getting a server up ASAP and making it work on NT and 95.
Since we moved to linux, performance for the same machine has almost doubled.
What we ended up with, is basically a two tier process, the matchmaking was strictly client/server, using winsock but done in a way that was call compatible with Directplay
(because we had used DP for the peering code). In Armageddon I replaced the Directplay stuff entirely, and wrote a VERY simplistic connection base class. From there we added
compression and all the other useful stuff directplay didnt have. To start with for armageddon I used a http based protocol for matchmaking, with an IRC server under NT4 which
connected via ASP to a MS SQL backend (which did game logs and stats and stuff for our rankings).
After we shipped, I ported this over to linux, because SQL server was dying every other day (probably poor configuration on my part, but out of the box this bitch doesnt run! :)), in the
end, we changed over to MySQL and used a modified IRC server to poke values into the database. I couldnt believe MS SQL server would die, but since Ive seen lots of asp errors since,
I guess this is pretty natural.
So my main change from Worms 2 to Worms Armageddon was that the server end was moved from a custom written app to a commercial bunch of applications with an open protocol.
Of course we got hacked to death (as any network game tended to), so we ditched the ranking system a while after. But this is after several million games were logged.
Matt: one thing that will amuse you, if you do a game similar to worms, you can pretty much get away with almost ANY connection :)), I tested worms 2 over a 2400 baud modem and it worked fine :))
Hope this sheds some light on things.
Phil.
03/26/2001 (12:16 pm)
Hey Guys..Gideon: the biggest issue I found was that there was some inherent memory leak in accept called under NT4 at the time, to be honest I shouldnt really
blame NT too much, I never really had time to play with the thing, so it was basically my first effort that shipped (i.e. almost no profiling, almost no load testing etc).
My main problem was that the threading was pretty expensive in terms of a thread per connection. The amount of context switching was too much,
Ive since learnt that its better to use a thread pool with IOCP. But at the time, I was restricted to just getting a server up ASAP and making it work on NT and 95.
Since we moved to linux, performance for the same machine has almost doubled.
What we ended up with, is basically a two tier process, the matchmaking was strictly client/server, using winsock but done in a way that was call compatible with Directplay
(because we had used DP for the peering code). In Armageddon I replaced the Directplay stuff entirely, and wrote a VERY simplistic connection base class. From there we added
compression and all the other useful stuff directplay didnt have. To start with for armageddon I used a http based protocol for matchmaking, with an IRC server under NT4 which
connected via ASP to a MS SQL backend (which did game logs and stats and stuff for our rankings).
After we shipped, I ported this over to linux, because SQL server was dying every other day (probably poor configuration on my part, but out of the box this bitch doesnt run! :)), in the
end, we changed over to MySQL and used a modified IRC server to poke values into the database. I couldnt believe MS SQL server would die, but since Ive seen lots of asp errors since,
I guess this is pretty natural.
So my main change from Worms 2 to Worms Armageddon was that the server end was moved from a custom written app to a commercial bunch of applications with an open protocol.
Of course we got hacked to death (as any network game tended to), so we ditched the ranking system a while after. But this is after several million games were logged.
Matt: one thing that will amuse you, if you do a game similar to worms, you can pretty much get away with almost ANY connection :)), I tested worms 2 over a 2400 baud modem and it worked fine :))
Hope this sheds some light on things.
Phil.
#17
- No TCP (normally)
- No threading.
- No IO completion ports
- No overlapped IO
- No select
Tribes 1 & 2 do support TCP, but it's only used for special connections such a being able to telnet into the server.
All game traffic uses a single non-blocking UDP port. The Tribes 1 networking engine simply polls this port using recv to generate networking events. Tribes 2 was changed to use asynchronous Winsock calls and events, but this was only because it simplified the TCP support (which in T1 was also simply polled with recv since there was never more than one or two connections). The networking engine provides guaranteed events through its own Notified Delivery Protocol (described a little here).
The only thing I would probably change is adding support for overlapped IO. Though copying the buffers is not a big deal, the thought of it bothers me.
03/26/2001 (5:16 pm)
FYI, Tribes 1 and 2 networking uses:- No TCP (normally)
- No threading.
- No IO completion ports
- No overlapped IO
- No select
Tribes 1 & 2 do support TCP, but it's only used for special connections such a being able to telnet into the server.
All game traffic uses a single non-blocking UDP port. The Tribes 1 networking engine simply polls this port using recv to generate networking events. Tribes 2 was changed to use asynchronous Winsock calls and events, but this was only because it simplified the TCP support (which in T1 was also simply polled with recv since there was never more than one or two connections). The networking engine provides guaranteed events through its own Notified Delivery Protocol (described a little here).
The only thing I would probably change is adding support for overlapped IO. Though copying the buffers is not a big deal, the thought of it bothers me.
#18
Functionally I have no idea if it matters or not. I would probably have used a thread which blocked on recv, dumped a packet into a queue and possibly fired off an event if necessary. I do wonder what Windows is doing in it bowels to monitor the async IO, since there must be a thread somewhere which is reponsible for it. If anyone knows offhand, I'd be interested to find out what is going on there.
Well, I can't wait to get into this code. Hope you guys are ready soon:)
03/27/2001 (5:44 pm)
For the most part that model makes complete sense. Going into stuff like IOCP's is only useful when a lot of sockets are being created. I'm not sure about the asynchronous winsock calls. Although I haven't used them, everything I read seems to think that it was a useful feature back back in the Win3.1 days but unnecessary once we got real threads.Functionally I have no idea if it matters or not. I would probably have used a thread which blocked on recv, dumped a packet into a queue and possibly fired off an event if necessary. I do wonder what Windows is doing in it bowels to monitor the async IO, since there must be a thread somewhere which is reponsible for it. If anyone knows offhand, I'd be interested to find out what is going on there.
Well, I can't wait to get into this code. Hope you guys are ready soon:)
#19
Unless you are processing the network events on the interrupt as they occur, threads seem like a waste of time.
As for the Winsock support, I didn't add that :) It did actually simplify some of the networking code (which is worth something), and does not appear to have had any affect on performance.
03/27/2001 (8:21 pm)
The OS is already event driven and takes care of buffering data from the network, why waste time on another context switch? As long as the system buffer is large enough to accommodate the data that accumulates between calls to recv, threads simply complicate the code and if anything, slow things down.Unless you are processing the network events on the interrupt as they occur, threads seem like a waste of time.
As for the Winsock support, I didn't add that :) It did actually simplify some of the networking code (which is worth something), and does not appear to have had any affect on performance.
#20
I'm sure the performance of TribesII/V12 is mostly a function of its protocol versus how it shuffles packets around. Quite honestly, that will be a LOT more fun to get into.
I need to explore asynchronous IO though, as I think I previously discarded it out of hand. I find threads are a much cleaner/portable implementation, but I really need to find out that performance answer for myself.
03/28/2001 (5:03 pm)
I think the truth here is that either method is probably plenty efficient for the task. I'm sure the performance of TribesII/V12 is mostly a function of its protocol versus how it shuffles packets around. Quite honestly, that will be a LOT more fun to get into.
I need to explore asynchronous IO though, as I think I previously discarded it out of hand. I find threads are a much cleaner/portable implementation, but I really need to find out that performance answer for myself.
Torque Owner Tim Gift
TRIBES Networking Model article I co-authored with Mark gives an overview of what we did for Starsiege: TRIBES and TRIBES II