Pure Neural A.I.
by decryptoid · in Technical Issues · 07/30/2006 (1:07 pm) · 17 replies
I have a snapshot posted with more details but it is waiting for approval and I am eager to share some of my latest work with Neural Networks......
Vid of some bots fighting:
www.decryptoid.com/downloads/aidemo.wmv
... and a small video of a bot running thru a small maze-like group of boxes. He spins a bit but still makes it thru!
www.decryptoid.com/downloads/maze.wmv
These bots have no helper functions and are wired directly to the Move Pointer. They even handle the targeting with mouse moves (rather than just target picking and me moving them with code).
Fighting against these bot is a very strange experience. In some ways they are like computer players (fast response) but in others they very much resemble human players (like the way they run away when you jack them, or try to hide with low health, ha!)
They CAN get caught on certain geometry (like a close corner between two really long walls) but for the most part can navigate completely autonomously without any stupid pathfinding, no-go marking, "level scanning" or any other special whatsa-housits.
To obtain a nice Neural net that made correct generic decisions about the environment was not horribly difficult in all honesty. The difficult part was building a neural targeting system accurate to 5 decimal places (so the bots can get a few long shots in there!).
So there you have it... PURE NEURAL. Between sensors and the move pointer.... there are only neurons.
Vid of some bots fighting:
www.decryptoid.com/downloads/aidemo.wmv
... and a small video of a bot running thru a small maze-like group of boxes. He spins a bit but still makes it thru!
www.decryptoid.com/downloads/maze.wmv
These bots have no helper functions and are wired directly to the Move Pointer. They even handle the targeting with mouse moves (rather than just target picking and me moving them with code).
Fighting against these bot is a very strange experience. In some ways they are like computer players (fast response) but in others they very much resemble human players (like the way they run away when you jack them, or try to hide with low health, ha!)
They CAN get caught on certain geometry (like a close corner between two really long walls) but for the most part can navigate completely autonomously without any stupid pathfinding, no-go marking, "level scanning" or any other special whatsa-housits.
To obtain a nice Neural net that made correct generic decisions about the environment was not horribly difficult in all honesty. The difficult part was building a neural targeting system accurate to 5 decimal places (so the bots can get a few long shots in there!).
So there you have it... PURE NEURAL. Between sensors and the move pointer.... there are only neurons.
About the author
#2
The net is about 200 neurons, with about 128,000 synapes, with a complex multi-stage learning system of my own devise that required around 45,000,000 training iterations (14 hours), even after genetic optimization. I train the bots outside of TSE, then load the weights as the main system loads. The system is very complex, but I can still run about 5 bots at a time before it starts chuggin' (on a 1.8GHz opteron). Since I load the nets while the main system loads there is no lag time when a bot dies and respawns.. he gets his own brain back and keeps on thinkin'.
Yeah, a know that's a little nutz, but I never really have more than 3 bots active at once (out of view bots will sleep eventually) so it seems to work great. I will probably lighten up on the sensors and see how fast I can make them while still making interesting decisions.
My upcoming experiments will include -
1) Memory structures - a bot should follow you over a hill and remember that you just ducked behind a building.
2) Human training - With memory structures in place, I will begin training my first bot to play a complete mission autonomously exactly the way I would play it.
07/30/2006 (1:42 pm)
Thanks! Sure... It has 46 inputs including all the normal stuff (health, positions, etc), a long range radar (for targets), and a proximity sensor system for local navigation. The net is about 200 neurons, with about 128,000 synapes, with a complex multi-stage learning system of my own devise that required around 45,000,000 training iterations (14 hours), even after genetic optimization. I train the bots outside of TSE, then load the weights as the main system loads. The system is very complex, but I can still run about 5 bots at a time before it starts chuggin' (on a 1.8GHz opteron). Since I load the nets while the main system loads there is no lag time when a bot dies and respawns.. he gets his own brain back and keeps on thinkin'.
Yeah, a know that's a little nutz, but I never really have more than 3 bots active at once (out of view bots will sleep eventually) so it seems to work great. I will probably lighten up on the sensors and see how fast I can make them while still making interesting decisions.
My upcoming experiments will include -
1) Memory structures - a bot should follow you over a hill and remember that you just ducked behind a building.
2) Human training - With memory structures in place, I will begin training my first bot to play a complete mission autonomously exactly the way I would play it.
#3
keep us informed, and if you feel like sharing how you did it that would kick ass.
07/30/2006 (7:19 pm)
Sweet man,keep us informed, and if you feel like sharing how you did it that would kick ass.
#4
After all, the idea is for the network to generalize a set of behaviors rather than memorizing a certain pattern.
Having trained a lot of them myself, I found knowing when to stop was key. Nice work.
07/31/2006 (7:42 am)
It doesn't seem reasonable to train the NN so exhaustively. What type of behavior do you get if you train the NN for 1/10th the number of iterations, for example, and how is that behavior so different qualitatively that it would lead you to train ten times more?After all, the idea is for the network to generalize a set of behaviors rather than memorizing a certain pattern.
Having trained a lot of them myself, I found knowing when to stop was key. Nice work.
#5
The key is that I don't pick a target with the net and turn with math... the neurons are actually directly attached to the Move-> pointer. As a result, if there was even the tiniest bit of instability or inaccuracy it made the bot jerk around, shooting was just wacky, and the small errors at close range would be magnified when he tried to shoot me at long distance.
The "concious" part of the net makes nice decisions at about 70% correlation, while the targeting "reflex" system is trained to 99.97% correlation... that's why he can shoot you at 100 meters but still makes really smooth decisions like backing up when you are too close, or deciding when to run away when damaged. For those of you who know what it means, overall AIC on this net is -42,687.9 and MDL is -34,799.9... no, that's not a typo. (and a correction to one of my above posts... this net has 12,800 synapes... I put an extra 0 in there, 'doh!)
Most people are not training networks this complex and are not familiar with special staged learning techniques to control specialization and over-training. Nets with this many synapes require huge datasets to make aliased decisions, otherwise tiny sets of the large array learn to memorize each specific instance and the bot becomes more like a state engine... and just as boring.
To give you guys a few hints and reasons why I did it like this....
1) In a previous life I was a mechanical engineer, so I thought, why not make the bot the same way I would make a robot, with industry standard sensors and everything (or at least similiar)
2) My education is in Biology and Biochem, so I thought, why not design a network that emulates an actual organism?
3) I saw a PBS show on brains or something... and I saw a picture that made it all clear to me. Even after messing with nets for over 7 years I never had a true comprehension like I now do... and not of simple things like just how they work, but I could build ultra complex systems with mixed topologies, learning techniques, memory systems, comprehension of the time domain and spatial awareness, all in one single unfied brain structure.
4) elegence in the complexity - with special structures, I kept the 200 neuron net to 12,800 synapes (that post above had an extra 0) and if it was a standard net it would have closer to 40,000. That yeilds a net almost 3 times faster than standard techniques for the same size. That's how I can get 3 or 4 bots in there instead of just one. SWEEEEEEEET.
5) I feel that most implementations don't use neural nets to thier full potential because of a lack of understanding and more importantly, difficulty implemented complex structures which actually work prefectly! It is time to put an end to function approximators and start building "aware" systems. The technology is here so I am gonna use TO THE MAX.
07/31/2006 (9:12 am)
Thanks! For the most part you are correct... most of the system worked well with about 1000 iterations of a 9000 sample set. The problem was targeting. Consider the rest of the iterations as me taking the bot to a targeting range and shooting skeet with him. The key is that I don't pick a target with the net and turn with math... the neurons are actually directly attached to the Move-> pointer. As a result, if there was even the tiniest bit of instability or inaccuracy it made the bot jerk around, shooting was just wacky, and the small errors at close range would be magnified when he tried to shoot me at long distance.
The "concious" part of the net makes nice decisions at about 70% correlation, while the targeting "reflex" system is trained to 99.97% correlation... that's why he can shoot you at 100 meters but still makes really smooth decisions like backing up when you are too close, or deciding when to run away when damaged. For those of you who know what it means, overall AIC on this net is -42,687.9 and MDL is -34,799.9... no, that's not a typo. (and a correction to one of my above posts... this net has 12,800 synapes... I put an extra 0 in there, 'doh!)
Most people are not training networks this complex and are not familiar with special staged learning techniques to control specialization and over-training. Nets with this many synapes require huge datasets to make aliased decisions, otherwise tiny sets of the large array learn to memorize each specific instance and the bot becomes more like a state engine... and just as boring.
To give you guys a few hints and reasons why I did it like this....
1) In a previous life I was a mechanical engineer, so I thought, why not make the bot the same way I would make a robot, with industry standard sensors and everything (or at least similiar)
2) My education is in Biology and Biochem, so I thought, why not design a network that emulates an actual organism?
3) I saw a PBS show on brains or something... and I saw a picture that made it all clear to me. Even after messing with nets for over 7 years I never had a true comprehension like I now do... and not of simple things like just how they work, but I could build ultra complex systems with mixed topologies, learning techniques, memory systems, comprehension of the time domain and spatial awareness, all in one single unfied brain structure.
4) elegence in the complexity - with special structures, I kept the 200 neuron net to 12,800 synapes (that post above had an extra 0) and if it was a standard net it would have closer to 40,000. That yeilds a net almost 3 times faster than standard techniques for the same size. That's how I can get 3 or 4 bots in there instead of just one. SWEEEEEEEET.
5) I feel that most implementations don't use neural nets to thier full potential because of a lack of understanding and more importantly, difficulty implemented complex structures which actually work prefectly! It is time to put an end to function approximators and start building "aware" systems. The technology is here so I am gonna use TO THE MAX.
#6
How long did it take to design, code, debug and implement ?
Also, is it extendable (using the tool ?) Maybe an in game interface for setting
dynamic features ? or other enhancments you've thought of.
I could help with coding if interested ?
just drop me an e-mail.
07/31/2006 (11:58 am)
Looks really nice.How long did it take to design, code, debug and implement ?
Also, is it extendable (using the tool ?) Maybe an in game interface for setting
dynamic features ? or other enhancments you've thought of.
I could help with coding if interested ?
just drop me an e-mail.
#7
07/31/2006 (12:11 pm)
Outstand work. Will this be release as resource or pack or you doing it for gmae. 
#9
yup.. you're that dude from terminator who invented skynet and caused the destruction of the human race arent you? :P
lol cant comment on anything other then whats read here because nothing i have seems to want to open your video files for some reason, quicktime, wmp, winamp, divx... but sounds like youre doing a good job lol. Any ideas why I cant get the video to play (1st one.. the second doesnt even have a video attached to the link)
11/02/2006 (1:25 pm)
Quote:It is time to put an end to function approximators and start building "aware" systems. The technology is here so I am gonna use TO THE MAX.
yup.. you're that dude from terminator who invented skynet and caused the destruction of the human race arent you? :P
lol cant comment on anything other then whats read here because nothing i have seems to want to open your video files for some reason, quicktime, wmp, winamp, divx... but sounds like youre doing a good job lol. Any ideas why I cant get the video to play (1st one.. the second doesnt even have a video attached to the link)
#10
By the way, how do they go on indoor levels?
11/08/2006 (8:52 am)
That's pretty impressive. The only letdown seems to be the sheer amount of resources it takes to run them - having a max of so few of them operating really reduces their practicality. How much faster would it be if you could use the neural net for just the decision making and then relegate other functions to non-neural algorithms? I'm trying to do some simple neural network stuff, so this is pretty interesting.By the way, how do they go on indoor levels?
#12
12/05/2006 (2:35 pm)
It has to be approved first. Do you plan on releasing any of this as a resource or any documents about Neural AI?
#13
In the meantime entertain yourselves with the following:
www.decryptoid.com/downloads/PureNeural_demo2.wmv
...and
...more advanced neural network design including:
1) Threat/scenario analyzer
2) Target Processor/Selector
3) Recurrent Linear Servo-Controller for mouse moves
4) Standard Linear Controllers for Strafe and Fire
@Matt... I never really planned on releasing anything for developers in the beginning... BUT, there is a small chance I will release some portion of this in the future. I have changed jobs and depending on how useless this technology becomes to me over the next year I may at the very least write some articles on how to get started for real (rather than a bunch of theoretical nonsense you find in most books). To be honest, it's not hard to make a neural network... it just hard to make a sensor and normalization routine for the data. 99.999999999% of the time I spent was fixing math errors on my sensor function. Sad eh?
12/05/2006 (2:51 pm)
Sorry, I always forget about the approval of snapshots 'cause I am all excited...In the meantime entertain yourselves with the following:
www.decryptoid.com/downloads/PureNeural_demo2.wmv
...and
...more advanced neural network design including:
1) Threat/scenario analyzer
2) Target Processor/Selector
3) Recurrent Linear Servo-Controller for mouse moves
4) Standard Linear Controllers for Strafe and Fire
@Matt... I never really planned on releasing anything for developers in the beginning... BUT, there is a small chance I will release some portion of this in the future. I have changed jobs and depending on how useless this technology becomes to me over the next year I may at the very least write some articles on how to get started for real (rather than a bunch of theoretical nonsense you find in most books). To be honest, it's not hard to make a neural network... it just hard to make a sensor and normalization routine for the data. 99.999999999% of the time I spent was fixing math errors on my sensor function. Sad eh?
#14
12/05/2006 (3:29 pm)
Awesome! Please release something! Anything at all would magnify the awesome to unbelievable dimensions.
#15
12/05/2006 (4:14 pm)
I am very impressed.
#16
12/25/2006 (11:12 pm)
The link isnt working, But it looks hella awesome. Good work 
#17
01/20/2012 (10:02 am)
Has any of the code been released ? 
Associate Orion Elenzil
Real Life Plus
can you give an idea of what the different inputs are and how big the net is ?