stable-baselines3

Stable Baselines3 (SB3) is a powerful, PyTorch-based reinforcement learning library that provides reliable, documented implementations of popular RL algorithms including PPO, SAC, DQN, TD3, DDPG, and A2C. This skill is designed for researchers and engineers working on sequential decision-making tasks, robotics, or complex environment simulation. It enables users to rapidly prototype agent training using a simplified, scikit-learn-like API while maintaining the flexibility required for deep learning research. The skill covers the entire RL lifecycle, from initial environment setup and policy selection to advanced training diagnostics and model persistence.

• Full implementation support for popular algorithms: PPO, A2C (general-purpose), SAC, TD3 (continuous control), DQN (discrete), and HER (goal-conditioned).

• Streamlined creation of custom Gymnasium-compatible environments with built-in validation tools to check observation and action space specifications.

• Advanced training features including vectorized environments (DummyVecEnv, SubprocVecEnv) to maximize CPU utilization for parallel simulation.

• Comprehensive callback management system for monitoring training metrics, checkpointing models, implementing early stopping based on reward thresholds, and custom training logic.

• Standardized model persistence for saving/loading agents, normalization statistics, and interacting with PyTorch state dictionaries.

• Evaluation utilities for quantifying model performance, measuring mean reward and standard deviation, and deterministic evaluation.

• Prioritize PPO/A2C for multi-processing needs and general stability, while selecting SAC/TD3 for sample-efficient continuous control applications.

• Use check_env() before initiating long training runs to ensure custom environments satisfy all Gymnasium constraints.

• When working with vectorized environments, note that the step() signature differs from single-env logic (returns 4-tuple); terminal observations are accessed via info dictionaries.

• The replay buffer is excluded during model.save() to minimize disk usage; ensure models are loaded correctly via the class method rather than an instance method.

• Set gradient_steps=-1 when using multiple environments with off-policy algorithms to maintain a balance between sample efficiency and wall-clock time.