pytorch-lightning

This PyTorch Lightning skill provides a comprehensive toolkit for AI engineers and researchers to streamline neural network development. It removes boilerplate code by standardizing your PyTorch implementation into LightningModules, ensuring consistency across complex deep learning experiments. The skill is designed for users who need to scale their models from local laptops to high-performance computing clusters without manual code modifications.

• Full support for LightningModule structures including training, validation, test, and prediction loops.

• Automated Trainer configurations for multi-GPU, TPU, and multi-node hardware acceleration.

• Built-in distributed training strategies including Distributed Data Parallel (DDP), Fully Sharded Data Parallel (FSDP), and DeepSpeed for large-scale models.

• Comprehensive data pipeline management using LightningDataModule for reusable and efficient dataset processing.

• Extensible callback system for automatic ModelCheckpoint, EarlyStopping, and custom training metrics.

• Seamless logging integration for W&B, TensorBoard, MLflow, Neptune, and Comet for real-time experiment tracking.

• Users should define models by overriding core methods like training_step and configure_optimizers to ensure compatibility with the Trainer.

• Leverage the self.log() utility within your classes to track metrics across all devices automatically.

• When working with models exceeding 500M parameters, the skill recommends using FSDP or DeepSpeed strategies.

• Use the provided scripts/template_lightning_module.py and scripts/template_datamodule.py as the primary starting point for new projects.

• This skill expects PyTorch to be installed in the environment and manages the lifecycle of training rather than the model architecture definition itself.