triage-ci-flake

This skill provides a rigorous, mandatory protocol for triaging CI test failures within the Payload CMS codebase. It is designed for engineers and maintainers encountering intermittent or 'flaky' test failures that occur specifically in continuous integration environments but often pass during local development. By enforcing strict isolation and environment reproduction steps, this tool helps distinguish between transient environmental timing issues, bundling conflicts, or genuine regressions in the main branch.

• Automated reproduction pipeline requiring local execution of failing test suites via Playwright and Vitest.

• Strict server management policy to force port 3000 cleanup and verification before initiating test runs.

• Dual-mode testing approach: evaluates code behavior in both standard dev environments and production-like bundled states.

• Standardized logging and identification workflow to map CI failure output to specific suite names and test descriptors.

• Verification-first methodology that mandates live execution of commands rather than reliance on static log analysis.

• Mandatory inputs include the suite name, specific test description, and the full stack trace from the failed GitHub Actions or CI pipeline log.

• The workflow requires the use of pnpm scripts like pnpm dev and pnpm prepare-run-test-against-prod to ensure the environment mirrors production as closely as possible.

• Users must verify server readiness using curl before launching E2E Playwright tests to avoid race conditions.

• Failure to reproduce a test locally triggers a specific transition to production-bundled testing, isolating potential issues in the build process or vite configuration.

• This agent acts as a guardrail to ensure that no developer proposes a fix without first confirming that the reproduction command successfully triggers the failure in a controlled local instance.