Client Server Implementation- Discussion
by Gareth Fouche · in RTS Starter Kit · 04/13/2005 (12:30 am) · 5 replies
Alrighty, this little topic is inspired by the research I've been doing the last few days (in between being sick with flu). Pretty much, after I posted that projectile resource, Stephen posed the question of what would an observer see when a projectile was fired? Well, in my quest to uncover the answer to that question, I journeyed into dark and myserious places, deep into the heart of Torques network code, and communed with the ghosts that reside therein (haha, :-p)
So, I spent the weekend learning about ghosted objects and scoping etc. Armed with that knowledge, I went back to the RTS-SK, delved around a bit, and came to a very worrying conclusion. The code I presented earlier WILL work, but only under certain circumstances. The problems arise from scoping, and as far as I can see they aren't easily solved (well, the easy solutions create other bigger problems).
So, I've created this thread to present my thoughts on the matter, and invite discussion. I will detail the RTS-SK client server model as I understand it, how it works, what problems that arise, and my thoughts on how to go about sorting things out. I would appreciate any feedback. Perhaps I have got it wrong, perhaps I'm just imagining the problems, or perhaps you know of easy solutions to the problems. Or maybe you agree with me? Note also that this isn't an attack on the pack designers, it's an excellent foundation, but there are SOME problems we have to solve on our own.
Ok, so to start with, ghosting. Typically, objects are created on the server during a game, and then ghosted across to a client only if the object is in scope for that client. This is most often based on whether the clients camera can see the object. A new object of the same type is created on the client, and its data is set to replicate the server object. Now, this bit I'm a bit shaky on, but from what I can see, if the object then later goes out of scope, its ghost is deleted on the client. Then if it comes back into scope it is re-created. This can happen again and again. Creation and deletion. I'm not quite sure what the point of deleting the object is, if you aren't updating it whats the harm of letting it hang around till you need it again (if its something like a player or vehicle)? But this certainly seems to be happening in the RTS-SK at least, and I think it happens standard. The datablock system greatly reduces the cost of this ghost creation though.
Anyway, on to the RTS-SK. It can be a bit tricky sorting out what exactly is going on in the code at each end of the game, as quite a few classes can exist on both sides, albeit with slightly different behaviors. Plus, the classes are strongly interlinked. These 2 facts combined can make life...interesting ;-).
Right, so this it what it looks like. On the server side, there is an RTSConnection for each of the clients. This connection has simsets to store references to the clients units, his current selection, and his camera. Also stored in the connection is a list of the units that are visible to that client. This includes enemies. Now, what must be remembered is that this information is sitting on the server, from what I can see, it isn't sent to the client directly. In conjunction with the vismanager and the ghosting system, this causes units to be ghosted across. What happens is that the vismanager on the server works out which client can see what, and then updates the corresponding connection. (Remember, still entirely on the server) Then, in the RTScamera scope query (which is called for each client), instead of the usual FPS result based on what the player can see given his FOV etc, the RTSconnection is asked whether it can see each unit. If yes, the unit is in scope for that client. THAT is when the data gets sent to the client. All the units in scope for the client are ghosted across to him.
So, I spent the weekend learning about ghosted objects and scoping etc. Armed with that knowledge, I went back to the RTS-SK, delved around a bit, and came to a very worrying conclusion. The code I presented earlier WILL work, but only under certain circumstances. The problems arise from scoping, and as far as I can see they aren't easily solved (well, the easy solutions create other bigger problems).
So, I've created this thread to present my thoughts on the matter, and invite discussion. I will detail the RTS-SK client server model as I understand it, how it works, what problems that arise, and my thoughts on how to go about sorting things out. I would appreciate any feedback. Perhaps I have got it wrong, perhaps I'm just imagining the problems, or perhaps you know of easy solutions to the problems. Or maybe you agree with me? Note also that this isn't an attack on the pack designers, it's an excellent foundation, but there are SOME problems we have to solve on our own.
Ok, so to start with, ghosting. Typically, objects are created on the server during a game, and then ghosted across to a client only if the object is in scope for that client. This is most often based on whether the clients camera can see the object. A new object of the same type is created on the client, and its data is set to replicate the server object. Now, this bit I'm a bit shaky on, but from what I can see, if the object then later goes out of scope, its ghost is deleted on the client. Then if it comes back into scope it is re-created. This can happen again and again. Creation and deletion. I'm not quite sure what the point of deleting the object is, if you aren't updating it whats the harm of letting it hang around till you need it again (if its something like a player or vehicle)? But this certainly seems to be happening in the RTS-SK at least, and I think it happens standard. The datablock system greatly reduces the cost of this ghost creation though.
Anyway, on to the RTS-SK. It can be a bit tricky sorting out what exactly is going on in the code at each end of the game, as quite a few classes can exist on both sides, albeit with slightly different behaviors. Plus, the classes are strongly interlinked. These 2 facts combined can make life...interesting ;-).
Right, so this it what it looks like. On the server side, there is an RTSConnection for each of the clients. This connection has simsets to store references to the clients units, his current selection, and his camera. Also stored in the connection is a list of the units that are visible to that client. This includes enemies. Now, what must be remembered is that this information is sitting on the server, from what I can see, it isn't sent to the client directly. In conjunction with the vismanager and the ghosting system, this causes units to be ghosted across. What happens is that the vismanager on the server works out which client can see what, and then updates the corresponding connection. (Remember, still entirely on the server) Then, in the RTScamera scope query (which is called for each client), instead of the usual FPS result based on what the player can see given his FOV etc, the RTSconnection is asked whether it can see each unit. If yes, the unit is in scope for that client. THAT is when the data gets sent to the client. All the units in scope for the client are ghosted across to him.
#3
The only thing that you may be letting slide by you is the fact that you -can- obviously re-write the scoping and the visibility determinations yourself, so that the case you are worried about never exists.
You could force any unit that is attacking a particular client's units to be scoped, and then add a flag of some sort that would have the visManger still not -render- it. It would stay in scope (since it's attacking), and therefore generate projectiles just fine, but the client wouldn't be able to see the unit until you remove the flag.
This would alleviate the need to run the full sim on all clients (and that is quite a bit of network traffic if you are talking about large missions, etc., while still solving your concern.
04/13/2005 (6:42 am)
Just to validate, just about everything you've mentioned is entirely correct!The only thing that you may be letting slide by you is the fact that you -can- obviously re-write the scoping and the visibility determinations yourself, so that the case you are worried about never exists.
You could force any unit that is attacking a particular client's units to be scoped, and then add a flag of some sort that would have the visManger still not -render- it. It would stay in scope (since it's attacking), and therefore generate projectiles just fine, but the client wouldn't be able to see the unit until you remove the flag.
This would alleviate the need to run the full sim on all clients (and that is quite a bit of network traffic if you are talking about large missions, etc., while still solving your concern.
#4
I did actually think about doing what your talking about, but there are still exceptions. There are cases where you aren't actually attacking a client, but you still want him to see the projectiles.
- What happens if you have a "siege unit", that can attack the ground? This is a common scenario.
- What happens if you are firing a projectile at Client A (or a ground point) which is not in Client B's viewpint, but it crosses Client B's view?
Ok, I suppose the first option is easy to solve, just check whether the destination point is in the clients view (its the same calc as checking if a unit is in the field of view), and scope if yes. The second problem is trickier, but is far less common, and maybe I can just ignore it. Perhaps I'm just overthinking it, making problems for myself ;-)
My other concern is whether the scoping would get the ghost created on the client before the attack netEvent arrived. I suppose you could wait for the ghost ID to exist on the client before sending the event, maybe put it into a scheduled wait loop. But wait to long and you are out of synch, which kind of destroys the usefulness. Can you think of a better way to do this? Or is the scoping mechanism so fast I don't really need to worry
(Honestly, I don't really want to re-write the current structure, but I REALLY don't want to come back and find I've got to change the fundamental structure I've been building on.)
04/13/2005 (7:15 am)
Thanks for taking the time to read that Stephen.I did actually think about doing what your talking about, but there are still exceptions. There are cases where you aren't actually attacking a client, but you still want him to see the projectiles.
- What happens if you have a "siege unit", that can attack the ground? This is a common scenario.
- What happens if you are firing a projectile at Client A (or a ground point) which is not in Client B's viewpint, but it crosses Client B's view?
Ok, I suppose the first option is easy to solve, just check whether the destination point is in the clients view (its the same calc as checking if a unit is in the field of view), and scope if yes. The second problem is trickier, but is far less common, and maybe I can just ignore it. Perhaps I'm just overthinking it, making problems for myself ;-)
My other concern is whether the scoping would get the ghost created on the client before the attack netEvent arrived. I suppose you could wait for the ghost ID to exist on the client before sending the event, maybe put it into a scheduled wait loop. But wait to long and you are out of synch, which kind of destroys the usefulness. Can you think of a better way to do this? Or is the scoping mechanism so fast I don't really need to worry
(Honestly, I don't really want to re-write the current structure, but I REALLY don't want to come back and find I've got to change the fundamental structure I've been building on.)
#5
04/13/2005 (7:22 am)
Ok, thinking about it, I do have an equation somewhere about finding the closest point on a line to another point. If the second case really is a problem, I could take the line from the projectiles start to finish, apply it to find the nearest point, do RTS unit range check on the point as normal, and scope the unit as necessary. I think I was just creating unnessary worry for myself. 
Torque 3D Owner Gareth Fouche
So that is the structure as I understand it, and from it arises my fundamental problem. You see, units in the fog of war aren't just hidden on the client, they don't exist! So lets look at what happens.
- My projectile modification. Projectiles are created independently of each other on the client and server. Since the server has all the objects, damage will be applied correctly, no problem. But on the client side, projectiles will only be created if you can actually see the unit creating them. If it has a longer sight range than your closest unit, then, since it's in the fog, it won't be ghosted to the client. Since it's not on the client, its obviously not processing on that client, and therefore not creating projectiles! So you could have artillery shelling your base and you would have no idea why your units are dying! You could be attacked from out of the fog and not even know which direction its coming from, since you cannot see any tracer fire. In a pitched battle you wouldn't even notice until your men died!
- Or you might have a radar scan like in Starcraft. You might scan a region where a huge battle is going on, but you would just see the units doing their attack animations, falling over, taking damage, but no projectiles for a few seconds at least, until they fire again while you're actually viewing them.
Anyway, you see my point, although it might seem minor, this will quickly annoy the player.
I've been having a look at how most RTS's handle this, and (as it actually says in the RTS-SK docs), they handle it by having the whole simulation running on each client, with the server just passing around messages and performing validation. In these cases, all the units are sent to each client, and the fog of war just hides them.
So the point I'm at currently is that I think I'm going to do it the same way. I'll remove any scoping/visManager from the server side of things, and just send every unit. On the client side the visManager can just hide unseen units. This way, since all the units will exist on each client, and be processing, the projectiles will be created correctly.
There is another issue. All those units being on each client would probably increase network traffic a great deal. Currently, unit synch updates are being sent whenever the unit moves. Doing this for every unit might be costly (although note, you'd have to consider this even if you went with the current method, what if all your players armies met in the middle for a big showdown?) However, you can take advantage of the fact that the sim is deterministic (given the same input it *should* reach the same outcome), and only send unit updates infrequently, every so many ticks, just to make sure everyone is synched. This is how most RTSs do it.
There is a problem in terms of security though. Cheaters can hack your client to uncover the fog of war and see all the units. This might be a concern for you. However, for me this is a negligible problem, if I got to the point where my game was popular enough for people to hack it I'd be happy.
So that's it, what do you guys think? Thoughts, opinions, anyone solved these probs and want to share? Have I got it totally backwards?