Plan for Ben Garney
by Kyle Carter · 01/10/2005 (9:25 am) · 17 comments
2PM Friday, Jan 7
I get called into a Zap planning meeting. We've decided that we need to address the biggest problem with Zap at the moment, vis a vis, a lack of critical mass on the servers. Many approaches are discussed... The key thing is that Jeff says, "Well, we can probably let Ben spend a day on bots for Zap."
This might have been a bad idea.
9:40 AM Saturday, Jan 8
Zap has working, competent bots. This took only twelve hours of coding - the rest of the intervening time was spent on eating, hanging out, etc. From the e-mail I sent at that point, before going home to sleep:
Then I go out to a local cafe with PatW for breakfast. (Mmm, pancakes!) Buy a book for my German class, and then go home for some well-deserved rest.
3 AM Sunday, Jan 9
After hanging out with MattF and TimA for dinner (and seeing some movies - liked Bill and Ted's Bogus Adventure, didn't like the Butterfly Effect). Oh, and some anime. Slayers. Pretty wild stuff. It would be neat to do a shader demo that matched that 80's cartoon style...
Back on track. ;)
Notice the festive debug visualizations. You can see lines showing the calculated paths for the bots (which I believe are all trying to reach me in the middle), as well as some little triangle-reticle-widgets showing their exact goal positions.
At this point, all the bots are named Marco, and they all act the same way, which can cause some problems. The navgraph generator is also a bit primitive, you often get cases where nodes are in unreachable positions, causing bots to grind up against walls and such. This all leads up to these changes over the next eight hours or so:
This doesn't produce a lot of visual change, but it does make the bots play a lot better. They're beginning to be kinda fun now. They're also REALLY hard to kill at this point, as they have essentially perfect energy management - you have to wear through all their energy before you can hurt them! This combined with the fact that they stick in a pack makes it REALLY hard to beat them.
5 PM Sunday, Jan 9
Another restful day. Before work, I manage to vacuum the main area of my apartment with the shiny new vacuum I got for Christmas. Much nicer there now. I leave as my roomie is in the process of shampooing and cleaning it again - first time we've vacuumed in a while, so it's good to go the extra mile and get it really clean. ;)
Go out for dinner with Matt, Josh, and the new intern, Zac Zadell. I'm sure you'll be hearing more about him and from him in the coming months. :) We come back an hour or two later.
At this point, I sit down to implement the major missing piece. I've got the pathfinding, I've got shooting, I've got shields and repair pack support. I've got them working well with most of the gametypes (Soccer's a bit of a mystery for them still - they shoot the ball but don't understand that it needs to go in a goal :P). But they all tend to make the same decisions and act in the same way.
Zap is a team game and for it to work well, every player must fulfill a unique role. Some should attack, some should defend. All must balance multiple conflicting priorities, and do it fast enough to avoid getting... well...zapped. If everybody tries to attack, or everybody tries to defend, the game breaks down.
Also, all the bots are named Marco, which is kinda cool, but it gets old after a bit. And they're hard to tell apart.
In addition, as I add more functionality to the bots, I start ending up with some really hairy nested if-statements to deal with different situations. Hard to read, hard to debug, hard to tweak.
So I decide to fix ALL THREE problems in one fell swoop!
Basically, what I do is implement two things. First, an "AIProfile" system, which lets me associate tweakable constants with each bot. I can set their skill level (which controls how well they shoot, use their shields, etc), and control how they prioritize different tasks. Second, an "AITask" system, which lets me describe the various goals of bots using subclasses of a base AITask. The bot determines a priority for each task, based on its context (Are enemy ships nearby? Is anyone shooting at it? Does it have the flag?), then modifies those priorities based on values in its AIProfile.
So, for instance, you might have a bot that sits around and defends all day, which you would achieve by making it weight all tasks marked as DefenseTasks twice as much as any other type. It will still pay attention to other concerns, but given the opportunity will prefer to repair or guard the flag or what have you.
The other nice about the AITask architecture is that it's easy to extend it to deal with specific game type situations. I'm currently using a simplistic kludge to deal with gametype goals, but eventually will extend the AI to have specific tasks for things like Hunters maps or CTF maps.
I can also easily expose map annotations to the bots with this architecture. I want to have the ability to have various AI markers that the map creater can place - saying, for instance, "defend this area" or "wander to this area every so often." These hints can help the bot play the map MUCH more effectively than they do now - they tend to focus only on central areas of the map.
About 2:50am, the profiles are done. At 3:40am, the skill parameter works, making the bots miss, over use or under use their shield, or simply fail to shoot. Josh, Zac, and I spend the next three hours debugging and preparing a build for Jay to take to MacWorld.
And playing some really kickass games of Hunters. Once you get a game going, the bots are pretty much indistinguishable from real players, especially now that they have profiles and such going.
Right now a lot of stuff is hardcoded, but I expect once we integrate with Torque App, most of it'll move to script, giving people a LOT of power in terms of playing with AI.
And that's about it. I could probably rant all day about how cool this stuff is, but I think I'll just leave you with some screenshots and a promise it won't be too much longer till we get this out to you. :)
SCREENSHOTS
Blue guys healing one another.
Gotta chase your dreams...
Early version of the bot horde.
Scores from playing 20-person Hunters.
Will I make it...?
The Marcos.
Like a blossoming flower...
The yellow flags whizz lightly through the air...
Sucks if you don't quite make it to the nexus, though.
I get called into a Zap planning meeting. We've decided that we need to address the biggest problem with Zap at the moment, vis a vis, a lack of critical mass on the servers. Many approaches are discussed... The key thing is that Jeff says, "Well, we can probably let Ben spend a day on bots for Zap."
This might have been a bad idea.
9:40 AM Saturday, Jan 8
Zap has working, competent bots. This took only twelve hours of coding - the rest of the intervening time was spent on eating, hanging out, etc. From the e-mail I sent at that point, before going home to sleep:
Quote:
They fly around, they play most game types (usually with a little nudging from the human players), they use shields and repair packs, they shoot really well, they pathfind, they deal with all sorts of objects, and so forth.
Areas to work on:
- Optimize pathfinding
- Cache navgraph/optimize navgraph generation
- There are bugs in the navgraph generator, I think mostly relating to resetting the data in it.
- Reduce bot-bobbing, enhance flocking behavior
- Add more personality to bots
- add flexible behavior prioritizer system
- add quote support
- add "personality" definition capability
- reaction time
- different interest in different actions at different times (right now all hard coded)
- Make easier bots?
- Confirm that networking works properly, optimize move generation for bots.
- Confirm bots will leave/join as new players do.
- Add wall avoidance forces to bot movement code.
- Generate appropriate navgraph for speed zones
Ben
Then I go out to a local cafe with PatW for breakfast. (Mmm, pancakes!) Buy a book for my German class, and then go home for some well-deserved rest.
3 AM Sunday, Jan 9
After hanging out with MattF and TimA for dinner (and seeing some movies - liked Bill and Ted's Bogus Adventure, didn't like the Butterfly Effect). Oh, and some anime. Slayers. Pretty wild stuff. It would be neat to do a shader demo that matched that 80's cartoon style...
Back on track. ;)
Notice the festive debug visualizations. You can see lines showing the calculated paths for the bots (which I believe are all trying to reach me in the middle), as well as some little triangle-reticle-widgets showing their exact goal positions.
At this point, all the bots are named Marco, and they all act the same way, which can cause some problems. The navgraph generator is also a bit primitive, you often get cases where nodes are in unreachable positions, causing bots to grind up against walls and such. This all leads up to these changes over the next eight hours or so:
Quote:- Now produces simplified paths. (Added simple LOS peephole optimization to path emitter code.)
- Nodes are nudged post-graph-creation to avoid on-edge situations with world geometry (simple iterative spring constraint system)
- Moves are now generated with accurate times, to help prevent lag.
- Updated retrieve game type to more accurately report gamepoints to AI.
- Remove linked/unlinked lists, vastly reducing navgraph creation time.
- Added missing SpeedZone.h.
- Added a little tiny bit of sanity checking around map load code.
- Reordered interest identification code a bit to reduce checks done on "simple" cases like projectiles.
- Fixed mGoalPathPos crash by adding initializer.
- AI now prioritizes gamepoint goals by distance.
Bots can play semicompetently on...
- All CTF maps.
- All Zone Control maps.
- All Hunters maps.
- All Rabbit maps.
- All Retrieve maps.
- All ZapMatch maps. (Though they lack seeking behavior.)
- All Soccer maps. (Though they will not aim the ball, simply shoot at it.)
- Fixed path cache bug with (0,0) co-ordinates.
- Now does some additional culling of nodes, if there is no geometry around, to help enhance pathfinding performance.
This doesn't produce a lot of visual change, but it does make the bots play a lot better. They're beginning to be kinda fun now. They're also REALLY hard to kill at this point, as they have essentially perfect energy management - you have to wear through all their energy before you can hurt them! This combined with the fact that they stick in a pack makes it REALLY hard to beat them.
5 PM Sunday, Jan 9
Another restful day. Before work, I manage to vacuum the main area of my apartment with the shiny new vacuum I got for Christmas. Much nicer there now. I leave as my roomie is in the process of shampooing and cleaning it again - first time we've vacuumed in a while, so it's good to go the extra mile and get it really clean. ;)
Go out for dinner with Matt, Josh, and the new intern, Zac Zadell. I'm sure you'll be hearing more about him and from him in the coming months. :) We come back an hour or two later.
At this point, I sit down to implement the major missing piece. I've got the pathfinding, I've got shooting, I've got shields and repair pack support. I've got them working well with most of the gametypes (Soccer's a bit of a mystery for them still - they shoot the ball but don't understand that it needs to go in a goal :P). But they all tend to make the same decisions and act in the same way.
Zap is a team game and for it to work well, every player must fulfill a unique role. Some should attack, some should defend. All must balance multiple conflicting priorities, and do it fast enough to avoid getting... well...zapped. If everybody tries to attack, or everybody tries to defend, the game breaks down.
Also, all the bots are named Marco, which is kinda cool, but it gets old after a bit. And they're hard to tell apart.
In addition, as I add more functionality to the bots, I start ending up with some really hairy nested if-statements to deal with different situations. Hard to read, hard to debug, hard to tweak.
So I decide to fix ALL THREE problems in one fell swoop!
Basically, what I do is implement two things. First, an "AIProfile" system, which lets me associate tweakable constants with each bot. I can set their skill level (which controls how well they shoot, use their shields, etc), and control how they prioritize different tasks. Second, an "AITask" system, which lets me describe the various goals of bots using subclasses of a base AITask. The bot determines a priority for each task, based on its context (Are enemy ships nearby? Is anyone shooting at it? Does it have the flag?), then modifies those priorities based on values in its AIProfile.
So, for instance, you might have a bot that sits around and defends all day, which you would achieve by making it weight all tasks marked as DefenseTasks twice as much as any other type. It will still pay attention to other concerns, but given the opportunity will prefer to repair or guard the flag or what have you.
The other nice about the AITask architecture is that it's easy to extend it to deal with specific game type situations. I'm currently using a simplistic kludge to deal with gametype goals, but eventually will extend the AI to have specific tasks for things like Hunters maps or CTF maps.
I can also easily expose map annotations to the bots with this architecture. I want to have the ability to have various AI markers that the map creater can place - saying, for instance, "defend this area" or "wander to this area every so often." These hints can help the bot play the map MUCH more effectively than they do now - they tend to focus only on central areas of the map.
About 2:50am, the profiles are done. At 3:40am, the skill parameter works, making the bots miss, over use or under use their shield, or simply fail to shoot. Josh, Zac, and I spend the next three hours debugging and preparing a build for Jay to take to MacWorld.
And playing some really kickass games of Hunters. Once you get a game going, the bots are pretty much indistinguishable from real players, especially now that they have profiles and such going.
Right now a lot of stuff is hardcoded, but I expect once we integrate with Torque App, most of it'll move to script, giving people a LOT of power in terms of playing with AI.
And that's about it. I could probably rant all day about how cool this stuff is, but I think I'll just leave you with some screenshots and a promise it won't be too much longer till we get this out to you. :)
SCREENSHOTS
Blue guys healing one another.
Gotta chase your dreams...
Early version of the bot horde.
Scores from playing 20-person Hunters.
Will I make it...?
The Marcos.
Like a blossoming flower...
The yellow flags whizz lightly through the air...
Sucks if you don't quite make it to the nexus, though.
#2
01/10/2005 (9:36 am)
Very cool work Ben, did you use the stuff from the AI project or develop your own work based on the preexisting AI information built into Torque? 
#3
And it's not in Torque yet, the bots will be out before the move to Torque.
01/10/2005 (9:48 am)
Ben coded this all from scratch.And it's not in Torque yet, the bots will be out before the move to Torque.
#4
And will TGE licensees be able to get their dirty mitts on the source in the same way that it's in TNL cvs? Or is the TNL cvs version something entirely different?
Thanks,
Gary (-;
01/10/2005 (10:09 am)
So, without sounding too dumb, what's the deal with integrating it with TGE?And will TGE licensees be able to get their dirty mitts on the source in the same way that it's in TNL cvs? Or is the TNL cvs version something entirely different?
Thanks,
Gary (-;
#5
As a Geman I really like to know, what's your intention for taking a German class?
-- Markus
01/10/2005 (10:58 am)
Hey, Ben.As a Geman I really like to know, what's your intention for taking a German class?
-- Markus
#6
01/10/2005 (12:28 pm)
This is not in TNL, it is in Zap! The version of Zap that is open source, will not get this. We are not planning on putting it in TGE since this is 2D AI. From time to time, we do things that are not released to the public immediately. AI would have a huge support component that we do not want to take at the moment. Eventually, almost everything we work on ends up in the community in one way or another. For instance, this work may end up in Torque 2D.
#7
01/10/2005 (1:22 pm)
Keep the bots smart, or at least have a flag to turn off your dumbing-down code. If the bots are really that good, we can always have five humans vs one or two bots :)
#8
01/10/2005 (1:54 pm)
Great job ... I love all night AI coding sessions. I remember a crazy weekend a few years back where I did just that. Then I had my friend playing the game and everytime he'd win I'd then go and tweak the AI to make it tougher.
#9
That would be excellent!
I for one think that would make T2D a killer add-on for Torque.
01/10/2005 (5:36 pm)
" For instance, this work may end up in Torque 2D."That would be excellent!
I for one think that would make T2D a killer add-on for Torque.
#10
Tonight as we set-up the presentation (Alex and I scripted our presentation almost as fast as Ben wrote the Bot code) we ran through TGE, TSTPro (not sure if we'll get time to talk art pipeline - we only have 25 minutes in our 6 sessions)... then we put up Zap! with bots playing Hunters - it was awesome to watch. Just seeing bots chase 400+ Yard Sale flags on a 72" plasma screen with alex posting 7 figure scores with just a mouse and keyboard is breath taking and then the crowd started to form... an Apple exec yells out 'what game is that? - It looks amazing!' and he was over 30 meters away from the theater area.
We do know how to have fun around the Garage. We'll be making some announcements about T2D tomorrow and you'll see that GG is fully committed to a multiplatform gaming strategy long term (if you're making a game you want GG to publish think OS X).
01/10/2005 (7:44 pm)
Looks like GG will get to be center stage at MacWorld in the Game Area Theater. Tonight as we set-up the presentation (Alex and I scripted our presentation almost as fast as Ben wrote the Bot code) we ran through TGE, TSTPro (not sure if we'll get time to talk art pipeline - we only have 25 minutes in our 6 sessions)... then we put up Zap! with bots playing Hunters - it was awesome to watch. Just seeing bots chase 400+ Yard Sale flags on a 72" plasma screen with alex posting 7 figure scores with just a mouse and keyboard is breath taking and then the crowd started to form... an Apple exec yells out 'what game is that? - It looks amazing!' and he was over 30 meters away from the theater area.
We do know how to have fun around the Garage. We'll be making some announcements about T2D tomorrow and you'll see that GG is fully committed to a multiplatform gaming strategy long term (if you're making a game you want GG to publish think OS X).
#11
01/10/2005 (7:57 pm)
thinking OSX...thinking OS X....hey i got my own mac for christmas! that helps!
#12
01/10/2005 (8:20 pm)
@Jay: Excellent news man, always good to see exec's yelling anything from across the room for hilarity purposes, but when it's for something positive, that's outstanding!
#13
@Eric: It'll all be tweakable, if not on the initial release, certainly after we port to TorqueApp and have it controlled by script.
@Jeremy: Thanks. ;) It certainly was fun... and I see now why it's something you'd do once. :P
@Matt: Me, too. We'll see if it manages to get high enough on the task list, though. There's just too dang much cool stuff going on here. :)
@Jay: Sweet! We seem to have a tendency to make our way to the center of attention... :P I look forward to the debriefing. Say, has Alex gone up against his bot counterpart yet? ;)
01/11/2005 (7:46 am)
@Markus: Took the language for six years, now feel like learning more about the culture, given its tremendous influence on western thought over the past few centuries.@Eric: It'll all be tweakable, if not on the initial release, certainly after we port to TorqueApp and have it controlled by script.
@Jeremy: Thanks. ;) It certainly was fun... and I see now why it's something you'd do once. :P
@Matt: Me, too. We'll see if it manages to get high enough on the task list, though. There's just too dang much cool stuff going on here. :)
@Jay: Sweet! We seem to have a tendency to make our way to the center of attention... :P I look forward to the debriefing. Say, has Alex gone up against his bot counterpart yet? ;)
#14
01/11/2005 (9:54 pm)
@Jay: Wheeeeen? :P
#15
The reason is that this helps suspend disbelief in the game. They feel much more real. If they can snipe you from across the screen, but get caught behind a turret while trying to pick up the flag you dropped... well, it's something everyone will look at and go 'bot'. It breaks immersion. If you can't fix the piloting errors, then deliberately crank the accuracy down so they shoot like a noob too. The bots will suck, but at least they'll suck evenly.
As the piloting and teamplay get more consistent and good, then you can ramp the aiming back up.
Oh, and one last tip for 'realistic' feeling to bad aiming. Instead of a random error effect, having a wandering error direction. For example, a bot with the accuracy toned down could aim .25 seconds behind the target, with a wandering error margin of +/- .4 seconds. That 'wandering error' means that sometimes they'll be spot on... other times aiming far behind the target. But at any particular second, they'll EITHER be shooting spot on, OR behind the target... it should not look like a scattergun effect, because the error margin changes slowly, perhaps taking a full second to change halfway. You might even keep the error margin consistent for .5 seconds, and then have the bot 'adjust its aim' for .25 seconds, and have it consistent again for .5 seconds. Obviously if it is hitting, even a stupid bot would not 'adjust its aim'.
This also means that even a stupid bot is gonna hit someone who is not moving, which is a good thing.
Visually, it should look similar to how players shoot... usually shooting behind, and every little bit trying a different aiming method.
One more thing, the error margin should be clamped to aim in the proper 90 degree arc... so even if .6 seconds lag would put the bot aiming BEHIND itself (even though the player is in front), the clamping should have it aiming in the right general direction still. The arc could be updated on a periodic basis, not instantaneous... say, twice a second. This would also simulate a player having difficulty keeping up in a close in battle where the direction switches sides so fast.
All these values (.25 seconds behind, +/- .4 seconds, 90 clamp updated every .5 seconds, etc...) are for a relatively mediocre aiming... but I think the aiming will be better than you might think. When tailing the bot it will still aim just fine, since it will not have to lead. When the opponent is sitting still it will still aim just fine. Most of this error margin will affect firing at someone the bot is flying parallel to... which is hard for people to do anyway.
I'm sure there is more, but I wanted to discuss this because while I am not afraid that Ben will have the bots aiming perfectly (like the turrets), I am afraid that the 'miss' algorithm would be a random +/- x degree jitter... which feels VERY bot like when you see it in action. It looks like the bot is shaking its hand in an arc back and forth very fast, so the shots spray out like a scattercannon. Also, that simple kind of bad aiming looks even stupider when the bot can't hit a stationary target.
01/12/2005 (1:40 am)
One recommendation about bots and their skill level... make their aiming about even with their movement and teamplay. If they run into walls and push the soccer ball into their own goal, make them shoot like noobs too. Team tactics are always the hardest AI to write... I recommend making a deliberate effort to have their shooting feel about as skillful as their teamplay.The reason is that this helps suspend disbelief in the game. They feel much more real. If they can snipe you from across the screen, but get caught behind a turret while trying to pick up the flag you dropped... well, it's something everyone will look at and go 'bot'. It breaks immersion. If you can't fix the piloting errors, then deliberately crank the accuracy down so they shoot like a noob too. The bots will suck, but at least they'll suck evenly.
As the piloting and teamplay get more consistent and good, then you can ramp the aiming back up.
Oh, and one last tip for 'realistic' feeling to bad aiming. Instead of a random error effect, having a wandering error direction. For example, a bot with the accuracy toned down could aim .25 seconds behind the target, with a wandering error margin of +/- .4 seconds. That 'wandering error' means that sometimes they'll be spot on... other times aiming far behind the target. But at any particular second, they'll EITHER be shooting spot on, OR behind the target... it should not look like a scattergun effect, because the error margin changes slowly, perhaps taking a full second to change halfway. You might even keep the error margin consistent for .5 seconds, and then have the bot 'adjust its aim' for .25 seconds, and have it consistent again for .5 seconds. Obviously if it is hitting, even a stupid bot would not 'adjust its aim'.
This also means that even a stupid bot is gonna hit someone who is not moving, which is a good thing.
Visually, it should look similar to how players shoot... usually shooting behind, and every little bit trying a different aiming method.
One more thing, the error margin should be clamped to aim in the proper 90 degree arc... so even if .6 seconds lag would put the bot aiming BEHIND itself (even though the player is in front), the clamping should have it aiming in the right general direction still. The arc could be updated on a periodic basis, not instantaneous... say, twice a second. This would also simulate a player having difficulty keeping up in a close in battle where the direction switches sides so fast.
All these values (.25 seconds behind, +/- .4 seconds, 90 clamp updated every .5 seconds, etc...) are for a relatively mediocre aiming... but I think the aiming will be better than you might think. When tailing the bot it will still aim just fine, since it will not have to lead. When the opponent is sitting still it will still aim just fine. Most of this error margin will affect firing at someone the bot is flying parallel to... which is hard for people to do anyway.
I'm sure there is more, but I wanted to discuss this because while I am not afraid that Ben will have the bots aiming perfectly (like the turrets), I am afraid that the 'miss' algorithm would be a random +/- x degree jitter... which feels VERY bot like when you see it in action. It looks like the bot is shaking its hand in an arc back and forth very fast, so the shots spray out like a scattercannon. Also, that simple kind of bad aiming looks even stupider when the bot can't hit a stationary target.
#16
01/12/2005 (1:46 am)
Er, one more thing I forgot to mention. When I say 'aiming .25 seconds behind', what I mean is that the bot aims and arcs its shot, leading and all, just like the turrets, so that the bullet passes through the space .25 seconds after the ship leaves it. This means that the bot using this algorithm will still lead shots... just usually not quite enough... again, a trait of the typical noob, and even the typical not-so-noob. 
Torque 3D Owner Stephen Zepp
Sweet work man!