第 11 課：機器人模擬 — MuJoCo & Isaac Gym

強化學習：從基礎到高級

第 3 部分：強化學習架構與實踐

亞洲開發網

簡介

Robotics is the most important application of RL - train agents in simulation, deploy to real robots. MuJoCo and NVIDIA Isaac Gym are the top 2 physics simulators.

---

1. MuJoCo 環境

import gymnasium as gym

# Standard robotics environments
envs = [
    "Ant-v4",        # 4-legged locomotion
    "Humanoid-v4",   # Bipedal walking
    "HalfCheetah-v4", # 2D running
    "Hopper-v4",     # 1-leg hopping
    "Walker2d-v4",   # 2D walking
]

for env_name in envs:
    env = gym.make(env_name)
    print(f"{env_name}: obs={env.observation_space.shape}, act={env.action_space.shape}")
    env.close()

---

2. Training with SAC

from stable_baselines3 import SAC

env = gym.make("Ant-v4")
model = SAC(
    "MlpPolicy", env,
    learning_rate=3e-4,
    buffer_size=1_000_000,
    batch_size=256,
    tau=0.005,
    gamma=0.99,
    verbose=1,
    tensorboard_log="./ant_tensorboard/",
)
model.learn(total_timesteps=1_000_000)
model.save("sac_ant")

---

3.NVIDIA Isaac 健身房

GPU-accelerated parallel training — 1000× faster:

# Isaac Gym runs thousands of environments in parallel on GPU
from isaacgym import gymapi, gymtorch

gym = gymapi.acquire_gym()
sim = gym.create_sim(0, 0, gymapi.SIM_PHYSX)

# Create 4096 parallel environments
num_envs = 4096
envs = []
for i in range(num_envs):
    env = gym.create_env(sim, lower, upper, num_per_row)
    envs.append(env)

---

4. 模擬到真實的傳輸

技術	描述
領域隨機化	Randomize physics params (mass, friction, etc.)
系統識別	校準模擬以匹配真實機器人
Curriculum Learning	簡單→困難的任務逐漸
師生	模擬培訓教師，精進學生

# Domain randomization example
def randomize_physics(env):
    env.gravity = np.random.uniform(-10.5, -9.5)
    env.friction = np.random.uniform(0.5, 1.5)
    env.mass_scale = np.random.uniform(0.8, 1.2)
    env.actuator_strength = np.random.uniform(0.9, 1.1)

---

總結

模擬器	GPU Accel	速度	現實主義	授權
穆喬科	❌ CPU	中	高	免費
Isaac Gym	✅ GPU	很快	高	免費
PyBullet	❌ CPU	中等	中等	开源
Brax	✅ 賈克斯	快	中等	開源