update-llm-model-list

This skill provides a systematic approach for maintaining the accuracy of the supported LLM models within the Agenta platform. By utilizing the litellm model_cost registry as the authoritative source of truth, this tool ensures that model dropdowns, cost metadata, and provider routing configurations in the playground remain consistent with real-world availability. It is designed for software engineers and LLMOps practitioners who need to add new model versions, remove deprecated entries, or verify the integrity of the model registry during routine maintenance or feature expansion.

• Performs comprehensive auditing of model keys against the litellm ecosystem to identify missing, incorrect, or outdated entries.

• Automates the identification of new, relevant models while filtering out noise such as embedding, audio, and legacy models.

• Validates provider prefix conventions, ensuring that anthropic, gemini, groq, and openai models correctly map to their respective routing and cost-calculation logic.

• Generates diagnostic reports to verify alignment with the Secrets API enums and unit test requirements.

• Facilitates the maintenance of the canonical model list located in sdk/agenta/sdk/assets.py.

• Run the provided Python scripts via uvx to perform automated checks without requiring local environment configuration.

• Use the diagnostic output to manually update model lists in the source code, ensuring that new models are correctly categorized and placed in the appropriate provider block.

• Always execute the corresponding pytest suites in sdk/oss/tests/pytest/unit/test_supported_llm_models.py to ensure that changes do not break existing model integrations.

• Ensure adherence to provider-specific prefix requirements (e.g., stripping prefixes for Claude but maintaining them for Gemini/Groq) to avoid runtime routing errors.

• Integration with the ruff linter is recommended post-edit to maintain code style and compliance with repository standards.