higress-auto-router

The Higress Auto Router skill provides a command-line interface to manage intelligent model routing rules for the Higress AI Gateway. This utility allows developers to optimize LLM usage by automatically switching between different models based on user input, ensuring that complex reasoning, coding tasks, creative writing, or math problems are handled by the most appropriate model providers. By utilizing the get-ai-gateway.sh script, users can maintain a centralized configuration for model switching without needing to restart the gateway container. It acts as an abstraction layer over the underlying Wasm plugin configuration, simplifying the lifecycle management of routing policies.

• Define routing rules using trigger phrases or custom regex patterns to map specific user intents to specialized models like Claude, GPT, or Qwen.

• Perform CRUD operations on routing rules, including adding new mappings, listing currently active configurations, and removing deprecated rules via rule IDs.

• Automatically trigger hot-reloads of the Higress AI Gateway configuration file, ensuring zero downtime for routing updates.

• Validate YAML syntax for the internal model-router configuration, preventing errors during dynamic configuration updates.

• Support for complex routing scenarios including multi-language trigger support (e.g., '深入思考' vs 'deep thinking').

• Ideal for AI infrastructure engineers and platform operators managing multi-model LLM deployments within a cloud-native API gateway environment.

• Requires a running Higress AI Gateway container and the get-ai-gateway.sh CLI tool to be accessible.

• Routing logic relies on the 'higress/auto' model identifier in user requests.

• Configuration is persisted within the gateway's internal file system at /data/wasmplugins/model-router.internal.yaml.

• Users should ensure appropriate model provider credentials are configured within the Higress console before defining specific model routes to prevent request failures.