This is the last of the series of articles about this project, if you haven’t read the previous rounds yet, I recommed you to check them out. If you already read them thank you very much for your interest and let’s get started!
The last weeks was a very stressful one, I tried to train the model using different methods (changing the reward function many times, normalyzing the inputs, reshaping the NN, etc) but unfortunatelly I couldn’t make it work 😕, the model has a behavior pretty silly and it doesn’t seems to converge.
So I decided to stop the project here to keep working on new ones. Honestly I don’t like to left projects but this one taked much more time than others and I don’t want to lose my motivation, I want to keep moving and trying new ideas for projects.
The ML pipeline
Even though this week was the less successful one, I learned a lot about machine learning, mainly about the Tensorflow pipelines to create, save and load models during the project development and even in production.
The steps were pretty easy and are explained here. Basically, I exported the neural networks with their weights to a Git project using
save, then I commit and push them to Github and when I want to use them from any project, I just use the function
load that implements a fetch behind scenes (so you can load the model with a simple http request).
Saving the model
save function is part of the tensorflow.js API so with a simple line of code you can save the model wherever you want. In the browser using localstorage, in your filesystem and even in your own server throught internet by exposing an endpoint that recieves the model.
Loading the model
Once you saved the model in your own repository, the next step is to upload it by pushing the changes to the remote repository, for instance to Github. Then, you can just load the model directly to your project using the raw Github file.
Showing the result
I thought about logs, charts and other strategies to see the results of the training, but after investigate how could I show the results I realized that the best way to see if the machine is “intelligent” is by my own eyes. So I decided to serve the model directly to the game and tried to manage the fighters directly with the machine learning model.
Once the model is loaded into the game, the following code shows how the models recieve the states and perform actions in the environment.
I trained the model with around 100 fights of 300 epochs each one (~1min), and I didn’t see any kind of convergence, the behavior was pretty random.
Actually the best result that I had was a silly behavior from both fighters. Subzero only punchs and Kano only get closer to Subzero.
you can check it out in the deployed project here.
I couldn’t realize what was wrong in the trainning. So, as I said in the introduction, I decided to stop the project here and probably retake it in the future more prepared and with a cleaner mind.
This project was pretty funny and I learned a lot about Reinforcement learning, ML pipelines, ML training, ML integration and so on. So my first goal was archieved, to learn.
I’m happy with the results event though the fighters are a bit silly, I could create the whole pipeline and integration with Tensorflow.js taking a JS project that I didnt knew at all.
If you read all the series thank you very much for your attention!
Github repo: https://github.com/MCarlomagno/RLMortalKombat