1 Topic by Henningstone (edited by Henningstone 2017-01-11 17:23:31)

  • Henningstone
  • Henningstone
  • Member
  • Offline

Topic: Neural network genetic algorithm ddrace bot (by Paszczak)

Hey there!

I want to present you a DDRace bot, which learns to play a race map by itself, using a neural network genetic algorithm!
Important: It is entirely done by Paszczak, I only write this post for him because he asked me to. I did nothing to the bot itself.

This is still work-in-progress, but developing quite fast into a very appreciable project. He implemented a genetic algorithm into the teeworlds client, enabling it to learn how to play race on it's own.

For now, only the distance from the starting point is used to calculate the fitness, so it can only finish maps which go straight in one direction, not like Aip-Gores. If the distance from the spawn didn't increase for a defined amount of time, it will reset (kill) and go over to the next simulation. 100 simulations make one generation. After 4-5 generations, you can see a significant improvement.

The bot learns in real-time right while playing a map, so it takes quite long to gather some skill. But waiting is worth I can tell you ^-^

If you are interested in seeing the bot doing it's job, there is a twitch livestream online from time to time where you can watch it. You can also chat with the author of it using the twitch chat smile

WATCH IT HERE! http://www.twitch.tv/twneuronnetwork

If you are curious how it works, this video might we very interesting for you: https://www.youtube.com/watch?v=qv6UVOQ0F44. It is something similar to this bot for Super Mario.

I will keep you up-to-date about any further development, this is definitely a very interesting idea.

SOURCE CODE: https://github.com/paaszczak/teeworlds-neural-network

2 Reply by unsigned char* (edited by unsigned char* 2016-01-30 01:45:35)

  • unsigned char*
  • unsigned char*
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

Ufff... if works like "http://rednuht.org/genetic_cars_2/" you will need ~10 years for finish a map xDD

This "learning" can be used in other maps?

P.S: Improve it increasing population..... O_o

Why I'm a pointer? http://3.bp.blogspot.com/-Zem9Ozn7BOo/T … itchie.jpg

3 Reply by heinrich5991

  • heinrich5991
  • heinrich5991
  • Moderator / Donor
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

Looks awesome. smile

4 Reply by unsigned char*

  • unsigned char*
  • unsigned char*
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

of course is very interesting!

P.S: I just start see this youtube channel for see an introduction to I.A. :B

Why I'm a pointer? http://3.bp.blogspot.com/-Zem9Ozn7BOo/T … itchie.jpg

5 Reply by xush

  • xush
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

Hype! He is like a son for me.

Real programmers don't comment their code - it was hard to write, it should be hard to understand.
Proudly verkeckt since 2010.

6 Reply by Paszczak

  • Paszczak
  • New member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

@unsigned char*
of course it can, but for now I wanted to give him something easy to see how long is it going to take

7 Reply by Netherland

  • Netherland
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

Very interesting, there should be some videos showing the bot in action, tried to watch the stream yesterday but nothing happened.

A.I. is cool (until it starts to develop itself).

privet

8 Reply by Henningstone (edited by Henningstone 2016-02-01 19:03:19)

  • Henningstone
  • Henningstone
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

Netherland wrote:

Very interesting, there should be some videos showing the bot in action, tried to watch the stream yesterday but nothing happened.

"Nothing happened" because he streams from his home computer, and of course it isn't running 24/7.

@Paszczak: Well, maybe record some video for youtube once with low-gen and once with high-gen?

9 Reply by TeecTac

  • TeecTac
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

Neural networks seems like overkill to me.

Given that teeworlds physics is deterministic, there should be an analytical solution using pathfinding and newtonian physics.

Developper of teeworlds-stats.info

10 Reply by xush

  • xush
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

But then there is no guaranteed solution.

Real programmers don't comment their code - it was hard to write, it should be hard to understand.
Proudly verkeckt since 2010.

11 Reply by heinrich5991

  • heinrich5991
  • heinrich5991
  • Moderator / Donor
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

xush wrote:

But then there is no guaranteed solution.

As far as I understand the concepts, an analytical solution will always give you a result, and "neural networks" can take a very long time ("have no guaranteed solution").

12 Reply by xush

  • xush
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

The analytical solution will only give you a theoretical guaranteed solution.
The genetic algorithm only takes that long because he is doing one simulation at a time right now, usually you would do a whole population (100 in this case) simultaneously.

Real programmers don't comment their code - it was hard to write, it should be hard to understand.
Proudly verkeckt since 2010.

13 Reply by TeecTac

  • TeecTac
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

« only a theoretical guaranteed solution » is good enough and it is computationally effective.

NN are guaranteed to give a solution for a sufficient number of training steps.
That means it could take a very long time before finding a solution to complex maps.

On the other hand, its implementation is certainly simpler !

Developper of teeworlds-stats.info

14 Reply by rand()

  • rand()
  • Member
  • Offline

Re: Neural network genetic algorithm ddrace bot (by Paszczak)

Analytical solution may be not computable.
NN approximates a solution, expecting a global solution.  Moreover, it would be fun, given a working NN for a map, to test it on another map (close enough)