Training Bittle robot dog to walk
Training Bittle robot dog to walk
Attempting to train the Bittle robot dog to walk with reinforcement learning.
YouTube video to help you
Access it on SmilingRobo Simulations here
Overview
The Bittle robot dog is a fun and interactive robot that can be trained to perform various tasks. In this tutorial, we will focus on training the Bittle robot dog to walk using reinforcement learning.
This exciting project, now available on our platform, demonstrates the application of reinforcement learning to train the Bittle robot dog using the TD3 (Twin Delayed Deep Deterministic Policy Gradients) algorithm within the Isaac Sim environment. The project creator has made significant strides in adapting TD3 to the Bittle robot and has shared the code, files, and insights gained so far.
The Bittle robot, a compact quadruped developed by Petoi, provides a fantastic platform for robotics enthusiasts and developers to experiment with AI-driven movements. This project focuses on enabling Bittle to learn how to walk using reinforcement learning.
Demo of Simulation
Key Features
-
Petoi Bittle URDF Integration: The project leverages the Bittle robot’s URDF (Unified Robot Description Format) to allow Isaac Sim to simulate the robot’s physical properties, which are crucial for training the RL model.
-
YouTube Video Walkthrough: A detailed video guide accompanies the project, offering insights and visual demonstrations to help users understand the setup and how to run the project in Isaac Sim.
-
TD3 Benchmarking and Modifications: The original TD3 algorithm has been modified and benchmarked specifically for use with Isaac Sim and the Bittle robot. The creator has shared their progress, acknowledging potential errors or redundancies in the code, making this project an excellent opportunity for contributors to improve and build upon the current state.
Running the Code
To run the project, you’ll need to use the Omni version of Python, which is compatible with Isaac Sim. The project includes two key files that must be placed in the Omniverse directory for seamless execution. These files and the overall structure are well-documented, and running the code is straightforward with the provided instructions.
-
Main Training File: TD3-Bittle-16-1.py – This is the core training file where the reinforcement learning process is initiated.
-
Modified TD3 Model: TD3_4.py – This is an updated version of the TD3 model, tailored for larger-scale simulations with Bittle.
-
Models Directory: Contains the best-performing TD3 model so far, offering a starting point for further training or experimentation.
-
Simulation Scenes: Files like 20-Bittles-very-long-6.usd and 20-Bittles-very-long.usd are used to control the simulation environment, ensuring that the Bittle robots don’t collide with one another during training. These scenes are crucial for the smooth functioning of the reinforcement learning process in Isaac Sim.
-
Visualization Tools: graph_reward.py helps visualize the rewards and performance of the exploration and exploitation agents in real time during the training process, providing users with insights into the learning progress.