ai-llm-engineering

This skill serves as a high-performance operational hub for LLM system architecture, evaluation, and production deployment. It is designed for AI engineers and DevOps practitioners tasked with building, scaling, and maintaining production-grade LLM applications. The tool provides a structured decision framework for choosing between RAG, fine-tuning, and agentic workflows, ensuring that systems meet modern production standards through rigorous validation and optimization.

• Orchestrates the full LLM engineering lifecycle, including data pipelines, training, fine-tuning via PEFT/LoRA, and deployment strategies using vLLM for 24x throughput.

• Implements advanced LLMOps practices such as automated drift detection with 18-second response windows, multi-layered security defenses, and AI-powered guardrails to mitigate hallucinations and bias.

• Provides cross-functional navigation to specialized skills covering RAG pipeline chunking, search tuning (BM25, HNSW, hybrid), prompt engineering CI/CD, and agentic orchestration (LangGraph, AutoGen, CrewAI).

• Utilizes comprehensive evaluation patterns integrating tools like LangSmith, Weights & Biases, and RAGAS to ensure metric-driven rollout gates and quality assurance.

• Includes decision matrices for stack selection, performance budgeting, and identifying anti-patterns such as context overload, data leakage, and inefficient retrieval.

• Ideal for building and troubleshooting RAG systems, deploying high-throughput inference services, and managing multi-agent orchestrations.

• Expected inputs include architectural requirements, model performance metrics, deployment constraints, and observability logs; outputs provide actionable configuration patterns, architectural blueprints, and troubleshooting checklists.

• Operational constraints include careful management of context windows, balancing latency against reasoning depth, and ensuring compliance with safety guardrails.

• Best practices emphasize hybrid architectures that combine retrieval-augmented generation with fine-tuned models to achieve optimal accuracy and cost-efficiency in complex production environments.