This is a tutorial of training your own model in our Formula-RL environment. We assume that you have pre-installed Unity3D and ml-agents. If you haven’t install them already, please refer to the official documentation from Unity:
First thing we need to do is to open our project in the Unity. To open our project, you need to start Unity hub and select open on the upper right corner, and select our source project folder.
Once it loaded up the unity, you will probably see some scene popped up. We have severals scenes in our project, and for training your model specifically, we will select scene located under ./Assets/Karting/Scene/MountainTopTrainingScene.unity
Double click the scene file, if everything goes right, it will popup the scene we see right now, this is the training environment for your own model.
Now our training track is ready, we will introduce the kart model you will be using for training. Looking at the left where you have a list of hierarchy of objects. Under “====Agents====”, you’ll see a list of agents that are available to train. Of course you can add more or reduce the number of agents training in the scene, which I think it only affects the speed of training.
Make sure you selected all agents together, and on the right make sure you selected Mode to be Training. Here you can also tune your rewards or customize new reward script to your model.
We’re mostly ready for the Unity3D side, and now we can take a look at how we will use the ml-agent pacakge to train our model.
Open up your favorite terminal and make sure ml-agent is installed and working. To train our model, we need to make sure we have a path to the training configuration file. The training config file is used to specify the training algorithms and hyperparaters you can tune for the training process. Here we are using a training configuration that uses PPO algorithm, and you are free to modify it according to the instructions on ml-agent official documentation. We will also need to specify —run-id, which is a unique identifier for each training you run, you can startover from a new run-id, or you can resume from an existing run-id. Lastly and optionally, you will need to specify a —result-dir which we recommend output your model under ./Assets/results/
Hit enter and the ml-agents will start, it will wait until you hit the play button in the Unity3D. After you hit play button in Unity3D, information will be logged into terminal every certain number of steps.
Once you finished training, you can just kill the ml-agent process and it will save an .onnx model file to the directory you specified. And you can refer to our “how-to-run-your-model” demo using your own .onnx model.
