Getting started

Quickstart

The fastest way to score an OpenAPI spec is the CLI — no account required.

1

Run the CLI

Score any local spec in seconds. Works with both JSON and YAML.

Terminal
npx agenticscore score ./openapi.yaml
2

Read the output

You'll get an overall score, a letter grade, and a breakdown by category with actionable findings.

Output
AgenticScore — Pet Store API v1.0.0
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Overall score: 86 / 100  (B)

  Examples          53
  Semantics        100
  Intent           100
  Error Handling    83
  Parameters       100
  Pagination       100

Top findings:
  → 4/12 operations lack response examples
  → No RFC 9457 Problem Detail format detected
3

Use the API (optional)

For CI/CD integration or programmatic scoring, use the REST API. Paid API access is opening soon.

Terminal
curl -X POST https://api.agenticscore.dev/score \
  -H "x-api-key: ar_live_your_key" \
  -H "Content-Type: application/yaml" \
  --data-binary @openapi.yaml
Authentication

API Keys

All API requests require an API key passed in the x-api-key header. The CLI does not require a key.

Key format

Keys are prefixed by environment:

ar_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  # production
ar_test_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  # test / sandbox

Passing the key

HTTP header
x-api-key: ar_live_your_key
Keep your key secret. Anyone with it can consume your monthly quota. If it's ever exposed, rotate it at /keys/rotate.

Getting a key

Paid API access is opening soon. The free CLI works locally without a key.

Rotating a key

Visit /keys/rotate. You'll need your current key and the email address used at checkout. A new key is issued and emailed immediately; the old one stops working.

CI/CD

GitHub Action

Score your OpenAPI spec on every pull request. Requires a Pro plan API key.

1

Add your API key as a secret

In your GitHub repo go to Settings → Secrets and variables → Actions → New repository secret.

Secret name
AGENTICSCORE_API_KEY
2

Add the workflow

Create .github/workflows/agenticscore.yml in your repo.

.github/workflows/agenticscore.yml
name: AgenticScore

on:
  pull_request:
    paths:
      - 'openapi.yaml'   # adjust to your spec path

jobs:
  score:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write

    steps:
      - uses: actions/checkout@v4

      - uses: slaterhaus/agenticscore-action@v1
        with:
          api-key: ${{ secrets.AGENTICSCORE_API_KEY }}
          spec-path: openapi.yaml
          fail-below: 70   # optional
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Inputs

InputRequiredDescription
api-keyrequiredYour AgenticScore API key
spec-pathrequiredPath to your OpenAPI spec (JSON or YAML)
fail-belowoptionalFail the workflow if score drops below this threshold (0–100)

Outputs

OutputDescription
scoreOverall score (0–100)
gradeLetter grade (A–F)
passedtrue if the score met the fail-below threshold
CLI

Command Line Interface

The CLI runs entirely on your machine. No account, no API key, no data leaves your system.

Installation

Use via npx (no install needed) or install globally:

# Run without installing
npx agenticscore score ./openapi.yaml

# Install globally
npm install -g agenticscore
agenticscore score ./openapi.yaml

Commands

CommandDescription
score <file>Score a local OpenAPI spec file (JSON or YAML)

Flags

FlagDescription
--format textHuman-readable output (default)
--format jsonOutput the full report as JSON
--format markdownOutput as Markdown (useful for piping into PR comments)
--min-score <n>Exit with code 1 if score is below n (0–100). Use in CI to fail the build.

Examples

# Score a YAML spec
npx agenticscore score ./openapi.yaml

# Score a JSON spec
npx agenticscore score ./openapi.json

# Get machine-readable output
npx agenticscore score ./openapi.yaml --format json

# Pipe into jq
npx agenticscore score ./openapi.yaml --format json | jq '.overallScore'
API Reference

REST API

Base URL: https://api.agenticscore.dev

POST/score

Score an OpenAPI spec. Accepts the raw spec body as application/yaml or application/json.

Request

HeaderValue
x-api-keyYour API key required
Content-Typeapplication/yaml or application/json
Example
curl -X POST https://api.agenticscore.dev/score \
  -H "x-api-key: ar_live_your_key" \
  -H "Content-Type: application/yaml" \
  --data-binary @openapi.yaml

Response

200 OK — application/json
{
  "score": 86,
  "grade": "B",
  "categories": [
    {
      "category": "examples",
      "score": 53,
      "rules": [
        {
          "id": "examples-operations",
          "name": "Operation Examples",
          "score": 75,
          "findings": ["2/4 operations lack examples: POST /pets, DELETE /pets/{id}"]
        }
      ]
    }
  ],
  "topFindings": [
    "2/4 operations lack examples: POST /pets, DELETE /pets/{id}"
  ],
  "metadata": {
    "specTitle": "Pet Store API",
    "specVersion": "1.0.0",
    "operationCount": 4,
    "schemaCount": 5,
    "scoredAt": "2026-05-22T17:00:00.000Z"
  }
}

Response fields

FieldTypeDescription
scorenumber0–100 composite score
gradestringA / B / C / D / F
categoriesarrayPer-dimension scores and rule-level findings
topFindingsstring[]Most impactful issues, ready to surface in a PR comment or dashboard
metadata.specTitlestringTitle from the spec's info.title
metadata.scoredAtstringISO 8601 timestamp of when the score was computed

Error responses

StatusMeaning
401Missing or invalid API key
403Monthly quota exceeded
413Spec exceeds the 1 MB size limit
422Body is not a valid OpenAPI spec
Score Interpretation

How scoring works

AgenticScore evaluates six dimensions of AI agent readiness. Each dimension is weighted and contributes to the 0–100 composite score.

Grade thresholds

GradeScore rangeMeaning
A90–100Excellent agent readiness — minimal friction for LLM tool use
B75–89Good — a few gaps but agents can work with this spec reliably
C60–74Moderate — agents will struggle with edge cases and error recovery
D40–59Poor — significant gaps that cause agent failures in practice
F0–39Failing — spec is not usable for reliable agent tool use

Dimensions

01

Examples 24% weight

Do operations and schemas include request/response examples? LLMs learn from concrete examples more reliably than descriptions alone. Scored on the fraction of operations and schemas that include examples or example fields.

02

Semantic Clarity 31% weight

Are descriptions meaningful — not just "Gets a thing"? Covers operation descriptions, operation summaries, and schema descriptions. Descriptions shorter than 10 characters are treated as absent. Highest weighted dimension because agents depend on natural language to select the correct endpoint.

03

Intent Signals 16% weight

Are operationId values descriptive (e.g. createInvoice vs operation123)? Are operations grouped with tags that have descriptions? These signals help agents understand what an operation does before reading its full description.

04

Error Handling 15% weight

Are 4xx and 5xx responses documented? Agents must handle failures gracefully — without documented error responses they have no way to recover from or surface meaningful errors. Bonus points for using RFC 9457 Problem Detail format.

05

Parameter Documentation 8% weight

Are query and path parameters described? A parameter named limit is obvious; q or filter is not. Agents need descriptions to understand valid values, formats, and constraints.

06

Pagination 7% weight

Can agents paginate through list endpoints? Checked by detecting cursor, page, offset, limit, or next_page_token style parameters on operations that return arrays. Missing pagination means truncated data or infinite loops.