At the beginning of March 2024, I had never created a machine learning model. However, by the end of April, my AWS DeepRacer model ranked in the top 50 in the Middle East & Africa Region competition, earning me a $99 Amazon gift card. Here are some of the lessons I learned along the way.

Getting Started
If you don't have an AWS account yet, you'll need to create one. Then, search for DeepRacer and click on "Get Started" in the left menu.

Setting Up Your First Model and Choosing Training Algorithm
Give your model an exciting name, like 'super fast model.' On the next page, select the time trial option.

I recommend starting with the PPO training algorithm. While SAC can provide a more optimized model, it only works with a continuous action space, an option available on the next page.

As for hyperparameters, the default settings are typically sufficient and are designed to work well for most use cases. If your model begins to plateau after several iterations, it might be worth tweaking these settings.

Writing a Reward Function
Check out the provided reward function examples. I began with the "follow the center line" model and made a minor adjustment for the first training session.

Your First Training Session
After completing the first training session, evaluate your model on a track. Take note of its strengths and, more importantly, its weaknesses. Now, the fun part begins.

Rinse and Repeat
If you're satisfied with your model's performance, clone it to build on its existing knowledge. For the remaining training sessions, setting the maximum time under stop conditions to 60 to 120 minutes should suffice. If it's too short, the model won't have sufficient learning time; if it's too long, overfitting becomes a concern.

Unless your model's performance deteriorates after a training session, keep cloning your most recent model to build on its existing learning.

Enjoy the Experience!
DeepRacer provides a fun and competitive way to get started with machine learning. However, it can become expensive, so be sure to monitor your AWS account billing regularly or try and get AWS credits which helped me, and delete any models you no longer need to keep costs down. See you at the finish line.