Skip to content
FastTest Agent

Run Tests in CI

Run saved FastTest suites in CI and post results as GitHub PR comments. Works with GitHub Actions, GitLab CI, or any CI system.

Prerequisites

  • A saved test suite (create one with Vibe Shield)
  • Cloud auth completed — you’ll need your API key
  • A GitHub repository (for PR comments)

Set Up GitHub Integration

  1. Create a GitHub Personal Access Token

    Create a classic token with repo scope, or a fine-grained token with Contents: Read and Pull requests: Read and write permissions.

  2. Store the token via FastTest

    Set my GitHub token to ghp_xxxx

    The token is encrypted and stored per-organization.

  3. Test PR comments locally

    Run the login suite against PR https://github.com/myorg/myrepo/pull/42

    FastTest executes the suite and posts a comment on the PR with pass/fail results, regressions, and healed selectors.

Add to GitHub Actions

Add this workflow to .github/workflows/fasttest.yml:

name: FastTest
on:
  pull_request:
    branches: [main]


jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4


      - name: Start app
        run: |
          npm ci
          npm run build
          npm start &
          npx wait-on http://localhost:3000


      - name: Run FastTest
        run: |
          npx -y @fasttest-ai/qa-agent ci \
            --api-key ${{ secrets.FASTTEST_API_KEY }} \
            --suite-id ${{ vars.QA_SUITE_ID }} \
            --app-url http://localhost:3000 \
            --pr-url ${{ github.event.pull_request.html_url }}

Required secrets and variables

NameTypeDescription
FASTTEST_API_KEYSecretYour FastTest API key (from dashboard Settings)
QA_SUITE_IDVariableSuite ID to run (from dashboard or list tool)

If your test suite uses {{VAR_NAME}} placeholders, add the corresponding values as GitHub secrets. FastTest resolves them from environment variables automatically.

CI Runner Options

Terminal window
npx -y @fasttest-ai/qa-agent ci \
  --api-key <key> \
  --suite-id <id> \
  [--app-url <url>]         # Override suite's base URL
  [--pr-url <url>]          # Post results as PR comment
  [--environment <name>]    # Use environment config for URL + variables
  [--browser chromium]      # Browser engine (chromium/firefox/webkit)
  [--json]                  # Output JSON for custom integrations

Exit codes: 0 for all tests passed, 1 for any failure. CI pipelines auto-fail on test regressions.

What PR Comments Look Like

FastTest posts (and updates on re-run) a comment showing:

  • Pass/fail summary with duration
  • Regressions since last run
  • Healed selectors (auto-repaired without failing)
  • Flaky test retries
  • Link to full results in the dashboard

Next Steps