django-celery-expert

The django-celery-expert skill provides specialized engineering support for developers building robust background task systems within the Django framework. Designed for backend engineers managing asynchronous workflows, this plugin encapsulates Vinta Software's production-proven best practices for task orchestration, message broker management, and observability. It bridges the gap between basic implementation and production-grade reliability by enforcing strict patterns for idempotency, serializable task arguments, and secure transaction handling.

• Task Design Patterns: Expert advice on implementing task signatures, complex workflows using chains, groups, and chords, and maintaining state in long-running tasks.

• Broker and Worker Configuration: Detailed guidance on optimizing Redis, RabbitMQ, or SQS backends, tuning worker concurrency, and managing queue routing.

• Error Handling and Resilience: Advanced strategies for retry logic, exponential backoff, circuit breaking, handling dead letter queues, and ensuring atomicity with database transactions using transaction.on_commit.

• Periodic Tasks: Streamlined implementation of Celery Beat, including crontab scheduling, dynamic task generation, and robust timezone handling for scheduled events.

• Monitoring and Observability: Best practices for implementing Flower, Prometheus metrics, structured logging, and debugging stuck or failed tasks in distributed systems.

• Intended for developers working on Django applications with Celery; not intended for Django Q, Huey, RQ, or ML orchestration tools like Airflow.

• Input: Natural language requests regarding task implementation, configuration debugging, or architecture reviews.

• Output: Code snippets following Vinta best practices, configuration guidance, and architectural pattern recommendations.

• Practical Constraints: Always use IDs instead of model instances for serialization; ensure tasks are idempotent to support safe retries; implement timeouts on all long-running processes to prevent worker starvation.

• Integrates directly with the broader Django agent ecosystem to provide consistent backend development guidance.