AI Crawler Policy Snapshot Comparison

POST /v1/crawler-policy

AI Crawler Policy Snapshot Comparison

Compare customer-supplied crawler permission maps with a saved baseline and required crawler list.

20 credits · USD $0.020

Input schema

{
  "type": "object",
  "properties": {
    "project_id": {
      "type": "string",
      "maxLength": 100,
      "x-maxUtf8Bytes": 100
    },
    "rule_version": {
      "type": "integer",
      "minimum": 1
    },
    "baseline_id": {
      "type": "string",
      "maxLength": 100,
      "x-maxUtf8Bytes": 100
    },
    "current": {
      "type": "object",
      "properties": {
        "crawlers": {
          "type": "object",
          "maxProperties": 20,
          "propertyNames": {
            "minLength": 1,
            "maxLength": 100
          },
          "additionalProperties": {
            "type": "boolean"
          }
        }
      },
      "required": [
        "crawlers"
      ],
      "additionalProperties": false
    }
  },
  "required": [
    "project_id",
    "rule_version",
    "baseline_id",
    "current"
  ]
}

Request

curl -X POST 'https://staging.callmint.dev/v1/crawler-policy' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'Idempotency-Key: unique-request-001' \
  -H 'Content-Type: application/json' \
  -d '{"project_id":"YOUR_PROJECT_ID","rule_version":1,"baseline_id":"YOUR_SNAPSHOT_ID","current":{"crawlers":{"GPTBot":true}}}'

Response result

{
  "project_id": "YOUR_PROJECT_ID",
  "kind": "crawler",
  "passed": true,
  "findings": [],
  "severity": "none",
  "rule_version": 1,
  "checked_at": "2026-11-03T00:00:00.000Z",
  "fingerprint": null,
  "duplicate": false,
  "changes": []
}

Limitations

20 credits. Customer-supplied normalized permissions only, at most 20 crawler names. Does not fetch or parse robots.txt, claim RFC 9309 compliance, grant legal permission or guarantee AI visibility. No background monitoring. Caller is responsible for obtaining accurate snapshots. External robots fetching remains disabled.

All enabled routes accept a JSON object of at most 60,000 UTF-8 bytes. String limits are also measured in UTF-8 bytes.

crawler-policy