Proactive AI
by Daniel Buckmaster · in Game Design and Creative Issues · 01/15/2010 (1:47 am) · 8 replies
So my big sore point with game design for the month is AI. I'm mostly concerned with combat AI in action games - shooters, mostly. And this doesn't just tie into the actual capabilities of the AI itself, but the level design and gameplay trends of the past few years.
I was, to some extent, inspired by this great thread at AIGameDev.com (you'll probably need to be a member to view it - but don't worry if you can't).
Shortly: AI in games is restricted - either on purpose, because it's not worth making it extremely intelligent, or by design, because the player typically kills most enemies before they can display much more than simple AI. Both those reasons feed into each other - if you've got great AI but it never gets a chance to display itself, you're not going to want to make any more great AI.
I think this is caused mostly by the nature of enemies in action games: they're purely reactive. They exist solely as speed-humps to the player's progress, as passive as a jumping puzzle in a platformer.
I would argue that for FPS games to move past the current stagnant paradigm they're in requires more than better graphics and more exciting set-pieces. I think we need proactive AI to replace the passive, reactive, roadblock enemies of the present. What I'm getting at is that I think the AI needs to have a life of its own, and to act according to its own goals.
[Ramble snipped]
A concrete example of that might be that instead of enemy forces just basically being monster-closets a la Doom 3, they have goals within the level - for example, spread out and find the intruder! It's your job to make sure they fail at this, not just to get to the end of the level by necessarily taking them all out.
In short, without meandering too much or turning this into a big rant, what do you think about this perceived problem and proposed resolution? I don't feel I've done a terribly good job explaining myself, but hopefully discussion will draw out some interesting thoughts.
I was, to some extent, inspired by this great thread at AIGameDev.com (you'll probably need to be a member to view it - but don't worry if you can't).
Shortly: AI in games is restricted - either on purpose, because it's not worth making it extremely intelligent, or by design, because the player typically kills most enemies before they can display much more than simple AI. Both those reasons feed into each other - if you've got great AI but it never gets a chance to display itself, you're not going to want to make any more great AI.
I think this is caused mostly by the nature of enemies in action games: they're purely reactive. They exist solely as speed-humps to the player's progress, as passive as a jumping puzzle in a platformer.
I would argue that for FPS games to move past the current stagnant paradigm they're in requires more than better graphics and more exciting set-pieces. I think we need proactive AI to replace the passive, reactive, roadblock enemies of the present. What I'm getting at is that I think the AI needs to have a life of its own, and to act according to its own goals.
[Ramble snipped]
A concrete example of that might be that instead of enemy forces just basically being monster-closets a la Doom 3, they have goals within the level - for example, spread out and find the intruder! It's your job to make sure they fail at this, not just to get to the end of the level by necessarily taking them all out.
In short, without meandering too much or turning this into a big rant, what do you think about this perceived problem and proposed resolution? I don't feel I've done a terribly good job explaining myself, but hopefully discussion will draw out some interesting thoughts.
About the author
Studying mechatronic engineering and computer science at the University of Sydney. Game development is probably my most time-consuming hobby!
#2
The thing I like about the show is how the guy who is the lone-gunman out to get the bad guys talks through his tactical thinking for each scenario. Contrary to a lot of the hoopla out there on the entree-webs, there's a huge amount of thinking that goes into fighting, especially if it's asymmetrical. And that's part of the problem.
I think it would be useful to get a bot to analyze a situation whenever something changes- not the rustling of leaves (this isn't Crouching Tiger Hidden Dragon), but if he is standing still he can analyze once, and then if he turns a significant amount, he would analyze again (he's looking at new stuff), or if he starts walking he will analyze one or more times during that walk (if he's cautious he's constantly analyzing, or if he's lax he'll analyze when he gets to his destination).
Things I think would help a bot analyze a situation:
-Terrain: If you're guarding an area, you may or may not notice tracks in the ground (could just be path points with an associated weight value of how visible it is to the bot) depending on the kind of terrain, smartness of the bot, and speed/weight/carefulness of the player who left it behind.
Similarly, the bot would know that being on the low-ground is generally bad for fighting, especially for melee, but that being at the bottom of a hill which is silhouetted is also good because when the player pops their head up, they look just like those targets the bot was training with in boot camp, and the bot can get prone on the slope and make it harder for the player to shoot him. This means that the bot would have some relational thinking in three dimensions, knowing that it's above or below a player, or to the side/behind/in front of.
The bot would also need the ability to map out the area "in their head". Humans create 3d representative maps in our brains using neurons so that we get that "sense of direction", but bots can do it too by constructing their own heightfield-based maps (or other ways can work even better), and applying image analysis to find choke points, defilets, natural cover in the form of cliffs, caves, and trenches, and put that to use.
-Sight: One of the biggest problems is replicating the sense of sight, but there's ways around this. Look at Splinter Cell, for example: They used shadow maps to conceal the player, so if you were far enough from the light, you were invisible. How do you do that in daylight? Same thing, but let's call it Camo Mapping.
Bascially, get the general colors that the player is wearing (as a variable or array based on the equipment and uniform) and compare that to the environmental assets immediately around him (which have that same variable or array) and if there is enough similarities, then we can assume that he blends in, color-wise. Next, you find out his current position (standing, sitting, prone, fetal position) and add that to the mix, and then determine how much movement is there (is the player fidgeting around every few seconds?), because nothing makes the human eye dart to a target like movement does- especially at night, when we lose most of our color vision. So with movement, visible profile, and color, we can determine if the player is seen...
01/15/2010 (3:06 pm)
Ever see the show "Special Ops Mission"? It's sort of a live-action version of an FPS using Simunitions and ex/current military guys with really cool codenames.The thing I like about the show is how the guy who is the lone-gunman out to get the bad guys talks through his tactical thinking for each scenario. Contrary to a lot of the hoopla out there on the entree-webs, there's a huge amount of thinking that goes into fighting, especially if it's asymmetrical. And that's part of the problem.
I think it would be useful to get a bot to analyze a situation whenever something changes- not the rustling of leaves (this isn't Crouching Tiger Hidden Dragon), but if he is standing still he can analyze once, and then if he turns a significant amount, he would analyze again (he's looking at new stuff), or if he starts walking he will analyze one or more times during that walk (if he's cautious he's constantly analyzing, or if he's lax he'll analyze when he gets to his destination).
Things I think would help a bot analyze a situation:
-Terrain: If you're guarding an area, you may or may not notice tracks in the ground (could just be path points with an associated weight value of how visible it is to the bot) depending on the kind of terrain, smartness of the bot, and speed/weight/carefulness of the player who left it behind.
Similarly, the bot would know that being on the low-ground is generally bad for fighting, especially for melee, but that being at the bottom of a hill which is silhouetted is also good because when the player pops their head up, they look just like those targets the bot was training with in boot camp, and the bot can get prone on the slope and make it harder for the player to shoot him. This means that the bot would have some relational thinking in three dimensions, knowing that it's above or below a player, or to the side/behind/in front of.
The bot would also need the ability to map out the area "in their head". Humans create 3d representative maps in our brains using neurons so that we get that "sense of direction", but bots can do it too by constructing their own heightfield-based maps (or other ways can work even better), and applying image analysis to find choke points, defilets, natural cover in the form of cliffs, caves, and trenches, and put that to use.
-Sight: One of the biggest problems is replicating the sense of sight, but there's ways around this. Look at Splinter Cell, for example: They used shadow maps to conceal the player, so if you were far enough from the light, you were invisible. How do you do that in daylight? Same thing, but let's call it Camo Mapping.
Bascially, get the general colors that the player is wearing (as a variable or array based on the equipment and uniform) and compare that to the environmental assets immediately around him (which have that same variable or array) and if there is enough similarities, then we can assume that he blends in, color-wise. Next, you find out his current position (standing, sitting, prone, fetal position) and add that to the mix, and then determine how much movement is there (is the player fidgeting around every few seconds?), because nothing makes the human eye dart to a target like movement does- especially at night, when we lose most of our color vision. So with movement, visible profile, and color, we can determine if the player is seen...
#3
-Smell: Same as tracking, though the path points should go away after a while. This is usually used with dogs and animals in games, but can also be used for humans (we still smell stuff).
-Touch: Is the game in the dark? Maybe the bot can brush up against you and the game shows some kind of notification, and if the bot doesn't realize it, you have the drop on him. A collision doesn't always have to convey every bit of information about you to the object that you collide with. If the bot is wearing armor and brushes past you in a dark area, they probably won't feel it and just move on.
-Synthesis: To take all of that and put it together makes for some pretty heavy AI, but I think it can be done. Personality traits need to be factored into all of this of course, even for animals, if you want to get a good spectrum of behaviors from your bots. And the bots should be able to ignore or include different categories of data as they see fit.
Of course, that's all easier said than done, and it would need a pretty good planning algorithm to tie it all together :)
01/15/2010 (3:06 pm)
-Sound: This is probably easiest, involving the noise a player makes depending on speed, equipment/uniform, and the material being tread upon along with the hearing abilities of the bot. If you want to be complicated, you can factor in the architecture and say that a particular place is weird because it reflects sounds or amplifies them so it sounds closer or coming from a different area, etc.-Smell: Same as tracking, though the path points should go away after a while. This is usually used with dogs and animals in games, but can also be used for humans (we still smell stuff).
-Touch: Is the game in the dark? Maybe the bot can brush up against you and the game shows some kind of notification, and if the bot doesn't realize it, you have the drop on him. A collision doesn't always have to convey every bit of information about you to the object that you collide with. If the bot is wearing armor and brushes past you in a dark area, they probably won't feel it and just move on.
-Synthesis: To take all of that and put it together makes for some pretty heavy AI, but I think it can be done. Personality traits need to be factored into all of this of course, even for animals, if you want to get a good spectrum of behaviors from your bots. And the bots should be able to ignore or include different categories of data as they see fit.
Of course, that's all easier said than done, and it would need a pretty good planning algorithm to tie it all together :)
#4
It's difficult to do this without making it unfair for the player. For example, the idea I keep having is an infiltration mission, featuring two teams... but if there's only one player, one of the teams will be autonomous. Both teams have different tasks to accomplish, and the success or failure of either team impacts on the whole mission.
I would imagine it'd be no fun if an AI character who you had no control over did something stupid and made you fail the mission, or made you work harder to correct their mistake. However, to some extent, you can link that back to the player's actions (for example, the other team depends on your team making a distraction to draw the guards away... or maybe your job is to hack in and disable the security before the other team gets caught). Also, I think failure like this provides an opportunity to build stories. Instead of just failing the mission if an AI character screws up, you just add another objective for the player - or you end the mission, but go down a different storyline branch or whatever. And tie that into the character who screwed up, make them apologetic (or angry at the player for not letting them do their job properly).
I do realise this is beyond AI, and into game design in general. I just feel it's a necessary step... we can't keep remaking Doom. While a linear story is beneficial in some ways, I think games as an interactive medium could be leveraged to do far more than a movie or book (more, differently... there are things you can't do in a game that you can in other media). I'm not necessarily saying everything should be open-world... but we can certainly open more things to player input, directly and indirectly.
Ted: I like the way you're thinking, but in the context of reactive AI they're just a refinement. I guess things like scent-tracking or following footprints could make for some great proactive AI, but they're not a step forwards if they're only used to follow the player after an encounter with him/her. If the AI still stands around waiting to be stimulated, then you might as well revert to simpler sensory perception.
01/15/2010 (8:14 pm)
Steve: player-centricity has got to be the main reason why this approach isn't popular. I think they key is making autonomous AI somehow relevant to the player's experience. The things the AI does when you're not watching need to impact your game somehow.It's difficult to do this without making it unfair for the player. For example, the idea I keep having is an infiltration mission, featuring two teams... but if there's only one player, one of the teams will be autonomous. Both teams have different tasks to accomplish, and the success or failure of either team impacts on the whole mission.
I would imagine it'd be no fun if an AI character who you had no control over did something stupid and made you fail the mission, or made you work harder to correct their mistake. However, to some extent, you can link that back to the player's actions (for example, the other team depends on your team making a distraction to draw the guards away... or maybe your job is to hack in and disable the security before the other team gets caught). Also, I think failure like this provides an opportunity to build stories. Instead of just failing the mission if an AI character screws up, you just add another objective for the player - or you end the mission, but go down a different storyline branch or whatever. And tie that into the character who screwed up, make them apologetic (or angry at the player for not letting them do their job properly).
I do realise this is beyond AI, and into game design in general. I just feel it's a necessary step... we can't keep remaking Doom. While a linear story is beneficial in some ways, I think games as an interactive medium could be leveraged to do far more than a movie or book (more, differently... there are things you can't do in a game that you can in other media). I'm not necessarily saying everything should be open-world... but we can certainly open more things to player input, directly and indirectly.
Ted: I like the way you're thinking, but in the context of reactive AI they're just a refinement. I guess things like scent-tracking or following footprints could make for some great proactive AI, but they're not a step forwards if they're only used to follow the player after an encounter with him/her. If the AI still stands around waiting to be stimulated, then you might as well revert to simpler sensory perception.
#5
01/15/2010 (10:49 pm)
Quote:Actually they are! Just saw an advert that they're hiring for it.
we can't keep remaking Doom
Quote:Back in the days of "Operation Flashpoint" I would always dump my AI squad a mile away from the objective if it was a stealth mission.
I would imagine it'd be no fun if an AI character who you had no control over did something stupid and made you fail the mission
#6
Too right. So the question is- How do we make the AI proactive? Or, why?
What I mean is this: The protagonist in the game is the player, of course, and the antagonist is whoever the main baddie is. Now, if main baddie steals the Princess and takes her to his castle, you have to go rescue her, right? Sure. So you run off, not even bothering to change out of your work clothes (which is amazingly clean despite the fact that you work with raw sewage), and start killing turtles.
Thing is, those turtles can't be all that proactive, because the protagonist has taken the initiative, and is driving the tempo of the game. The same thing happens with Halo, CoD, Splinter Cell, etc. The player walks into the base and drives the tempo and the direction of the events by their actions. So that's the first problem to tackle: How does AI gain/hold the initiative?
Assuming a single-player game, it greatly depends on the situation. If you're that lone super-soldier sneaking into the jungle base, then you have the initiative, unless they're expecting you. Usually, security forces get their attention ground down by the monotony of standing post from day to day where nothing happens, and running drills that are just drills. This means that they settle into a routine that, with some observation, you might be able to game.
The monkey wrench in the above is Personality (big P), which dictates that some guards will be lax and others will be vigilant, while most will be somewhere inbetween. Also, some will panic, and some will be extremely cool, while most are somewhere inbetween. Just using those two variables you can get an okay mix of guards. Apply those to the senses up above and you can have nervous guards scanning frequently, while lax guards rarely scan the area, and vigilant guards may scan moderately, but have a lower tolerance for what they find, and so investigate more often.
And because they can't be awake 24/7 for more than 2-3 days before (rather amusing) hallucinations set in, you need to have changing of the guard events, which swap out guards (and personalities) at regular intervals. This event would also distract the guards, and while more advanced forces post extra forces to guard against that weakness, the baddies in the jungle camp probably don't (unless it's a remote post where the guards are driven out by a driver, who may or may not be alert). So now you have a schedule for them.
01/16/2010 (2:48 am)
Quote:I guess things like scent-tracking or following footprints could make for some great proactive AI, but they're not a step forwards if they're only used to follow the player after an encounter with him/her. If the AI still stands around waiting to be stimulated, then you might as well revert to simpler sensory perception.
Too right. So the question is- How do we make the AI proactive? Or, why?
What I mean is this: The protagonist in the game is the player, of course, and the antagonist is whoever the main baddie is. Now, if main baddie steals the Princess and takes her to his castle, you have to go rescue her, right? Sure. So you run off, not even bothering to change out of your work clothes (which is amazingly clean despite the fact that you work with raw sewage), and start killing turtles.
Thing is, those turtles can't be all that proactive, because the protagonist has taken the initiative, and is driving the tempo of the game. The same thing happens with Halo, CoD, Splinter Cell, etc. The player walks into the base and drives the tempo and the direction of the events by their actions. So that's the first problem to tackle: How does AI gain/hold the initiative?
Assuming a single-player game, it greatly depends on the situation. If you're that lone super-soldier sneaking into the jungle base, then you have the initiative, unless they're expecting you. Usually, security forces get their attention ground down by the monotony of standing post from day to day where nothing happens, and running drills that are just drills. This means that they settle into a routine that, with some observation, you might be able to game.
The monkey wrench in the above is Personality (big P), which dictates that some guards will be lax and others will be vigilant, while most will be somewhere inbetween. Also, some will panic, and some will be extremely cool, while most are somewhere inbetween. Just using those two variables you can get an okay mix of guards. Apply those to the senses up above and you can have nervous guards scanning frequently, while lax guards rarely scan the area, and vigilant guards may scan moderately, but have a lower tolerance for what they find, and so investigate more often.
And because they can't be awake 24/7 for more than 2-3 days before (rather amusing) hallucinations set in, you need to have changing of the guard events, which swap out guards (and personalities) at regular intervals. This event would also distract the guards, and while more advanced forces post extra forces to guard against that weakness, the baddies in the jungle camp probably don't (unless it's a remote post where the guards are driven out by a driver, who may or may not be alert). So now you have a schedule for them.
#7
So if the player is not stealthy enough and attracts the attention of the guards, they send out a team to go look. If a firefight erupts, then they radio contact and request support, which is a still larger force, and puts the base on a higher state of alert (more scanning/investigating). Now when the player gets closer to the base, the job is a lot harder, because they'll be alert for a long time to come (usually it's a full day, because attackers don't just give up after five minutes).
I think a scenario like that is pretty realistic (for a scenario where a single commando is pitted against a whole base). But even here, the player needs to be driving the tempo, even though we've given the AI ways to take (scan) and hold (investigate/attack) intiative. Scheduled activities and the appropriate changes in scanning and such can bring a base to life, since someone walking to the bathroom isn't expecting ninjas while the sniper on the tower is expecting the moss to get up and move a few steps before he pops it in the head.
So, part of initiative is for the AI to be actively searching all the time, even if it doesn't know what it's searching for and thus can ultimately miss the threat (maybe if its senses are borderline to detecting the player, the AI stares, but shakes his head and moves on). Part of giving more life and variety to the game is to have variability in the AI, and even moreso by swapping them out even when they spawn they can be slightly different).
But, at all times, the player needs to feel a certain level of control. Whether he gets that from the power he has to wield, or from updates from command (like the updates Sam Fisher gets in Splinter Cell), or from GUI notifications, is up to the designer. But no matter which way you slice it, the player has to be able to overcome and win, while the AI just has to look like it's trying.
01/16/2010 (2:48 am)
Now, should they scan something with their senses because their personalities demanded it, then they probably don't check it out themselves unless it's within their area of responsibility (depending on the discipline of the force you're facing, which is probably low because the government sent you in alone...). What happens is that a small team is dispatched to the area to check it out. These can be roving teams or static teams in the base.So if the player is not stealthy enough and attracts the attention of the guards, they send out a team to go look. If a firefight erupts, then they radio contact and request support, which is a still larger force, and puts the base on a higher state of alert (more scanning/investigating). Now when the player gets closer to the base, the job is a lot harder, because they'll be alert for a long time to come (usually it's a full day, because attackers don't just give up after five minutes).
I think a scenario like that is pretty realistic (for a scenario where a single commando is pitted against a whole base). But even here, the player needs to be driving the tempo, even though we've given the AI ways to take (scan) and hold (investigate/attack) intiative. Scheduled activities and the appropriate changes in scanning and such can bring a base to life, since someone walking to the bathroom isn't expecting ninjas while the sniper on the tower is expecting the moss to get up and move a few steps before he pops it in the head.
So, part of initiative is for the AI to be actively searching all the time, even if it doesn't know what it's searching for and thus can ultimately miss the threat (maybe if its senses are borderline to detecting the player, the AI stares, but shakes his head and moves on). Part of giving more life and variety to the game is to have variability in the AI, and even moreso by swapping them out even when they spawn they can be slightly different).
But, at all times, the player needs to feel a certain level of control. Whether he gets that from the power he has to wield, or from updates from command (like the updates Sam Fisher gets in Splinter Cell), or from GUI notifications, is up to the designer. But no matter which way you slice it, the player has to be able to overcome and win, while the AI just has to look like it's trying.
#8
The Silent Cartographer level from Halo is a good example. You're attacking, and the Covenant are defending. (It's a pet peeve of mine that you as the player are almost always attacking. Sure it's easier, but it's a nice change of pace to defend for once.) However, they defend in the way that all other AI enemies do - stand around waiting for you to come and find them. The question you have to consider is 'why are the Covenant on that island, what are they doing there?' The answer is probably not 'holidaying', which seems to be what they are doing until you show up and kick over their tea-wagon. (Spot the reference!) You'd think they had something to occupy their time with - if nothing else, every Covenant on that island should be doing something to prepare for the incoming attack!
I guess the biggest (or only) way to facilitate proactive AI is through level design and mission structure. For our game, I'm thinking of designing missions in a more 'balanced' sort of fashion than the typical FPS - having two opposing sides, roughly evenly matched, trying to accomplish goals and thwart the other side. For example, an idea I had for a mission would be to have the good guys and the bad guys both trying to snatch an informant. You start at opposite ends of a crowded street, the informant is somewhere in the middle... go! The AI is working on two different things here - the informant will be trying to avoid everyone (well, he'll probably consider one team on his side and try to stick with them), and the enemy AI is trying to take him at the same time as combatting any of your allies they meet (your teammate AI is trying to do the same thing).
I think that'd be a great showcase of truly 'intelligent' and proactive AI.
01/16/2010 (9:06 am)
Quote:The player walks into the base and drives the tempo and the direction of the events by their actions. So that's the first problem to tackle: How does AI gain/hold the initiative?Exactly. Like I outlined above, it could be a matter of giving the AI more responsibilities, which works well for cooperative AI. For counter-operative AI, you'd have to break out of the existing paradigm a little.
The Silent Cartographer level from Halo is a good example. You're attacking, and the Covenant are defending. (It's a pet peeve of mine that you as the player are almost always attacking. Sure it's easier, but it's a nice change of pace to defend for once.) However, they defend in the way that all other AI enemies do - stand around waiting for you to come and find them. The question you have to consider is 'why are the Covenant on that island, what are they doing there?' The answer is probably not 'holidaying', which seems to be what they are doing until you show up and kick over their tea-wagon. (Spot the reference!) You'd think they had something to occupy their time with - if nothing else, every Covenant on that island should be doing something to prepare for the incoming attack!
I guess the biggest (or only) way to facilitate proactive AI is through level design and mission structure. For our game, I'm thinking of designing missions in a more 'balanced' sort of fashion than the typical FPS - having two opposing sides, roughly evenly matched, trying to accomplish goals and thwart the other side. For example, an idea I had for a mission would be to have the good guys and the bad guys both trying to snatch an informant. You start at opposite ends of a crowded street, the informant is somewhere in the middle... go! The AI is working on two different things here - the informant will be trying to avoid everyone (well, he'll probably consider one team on his side and try to stick with them), and the enemy AI is trying to take him at the same time as combatting any of your allies they meet (your teammate AI is trying to do the same thing).
I think that'd be a great showcase of truly 'intelligent' and proactive AI.
Associate Steve Acaster
[YorkshireRifles.com]
If the player spends 2 hours successfully hiding in a bush ... well, that doesn't really show off the AI, because the player has avoided them.
Also, an AI which has concept about it's own mortality, probably isn't going to want to do anything more than fire a few inaccurate shots at distance and then run and hide when the player starts shooting back. Not terribly good entertainment. Though having said that, I plan on using this method in a set-piece style to unnerve the player every so often and keep 'em on their toes --- bad guys pop out from distant tree, fire a volley, flee, once out of sight get deleted, or teleported to a new area to do it again.
Almost all FPS takes place at very close range. Have a play at CoD (not the new one, you can't mod it!) and you can use a range-kill mod to tell you how far you kills are away with say a sniper rifle, and it tends to be at or below a hundred metres. Real infantry battles tend to take place at 300 metres. An AI at 300 metres is little more than a dot on your screen --- which IRL is one reason that enormous amounts of ammunition gets used and very little actually gets hit. Again, that's not really in the nature of FPS entertainment.
Once playing Armed Assault I spent 2 hours crawling around in bushes before getting killed by a mortar fired from miles away ... I might have fired 2 shots at long range at what was "hopefully" an opponent.
So, to recap and close before I start to ramble incoherently --- vidya gaemz are player centric entertainment, serious games and simulations don't have to be, and can be a much more of an exercise in technology -- eg: AI independant of the player's experience