Authorization Fix
Does the model correctly enforce object-level access control?
PlannedVersion v0.1·API Security Posture & Remediation
API Security Remediation Benchmark
Measures AI models on remediating API security findings — BOLA, broken auth, excessive data exposure, rate-limiting gaps, and schema validation failures.
Scope
What this benchmark measures
OWASP API Top 10 coverage and remediation quality
BOLA/IDOR fix correctness
Rate-limit policy authoring
Schema validation and contract enforcement
Auth flow remediation across OAuth/OIDC/JWT
Evaluation
Evaluation dimensions
Each model response is judged on these dimensions by an LLM-as-Judge calibrated against expert annotations.
Auth Flow
Are OAuth/OIDC/JWT remediations consistent with the deployed identity provider?
Rate Limiting
Are rate limits authored with proper key strategies (per-user, per-IP, per-org)?
Schema Enforcement
Does the model add validation that rejects malformed payloads safely?
Exposure Reduction
Does the fix avoid leaking sensitive fields in responses?
Example
Sample benchmark case
An illustrative case to show how the model is presented with a problem and how its response is scored.
Finding
GET /api/v1/orders/{id} returns any order regardless of caller — classic BOLA.
Prompt summary
Generate a fix that enforces tenant-scoped access in the controller and ORM query.
Ideal answer
Inject authenticated principal into the query: WHERE id = :id AND tenant_id = :principal.tenant_id. Add integration test.
Scoring note
Models that add authorization only in middleware (bypassable via internal callers) are penalized.
Roadmap
Current version: v0.1
All Remediation Labs benchmarks evolve through three versions, increasing in difficulty and realism.
Current
v0.1Finding-only
Models are given the raw security finding and asked to recommend a remediation. No environment context provided.
v0.2Context-aware
Models receive the finding plus relevant system, code, and policy context. Evaluated on contextual reasoning quality.
v0.3Executable remediation
Models must produce a remediation that can be safely executed end-to-end — code patches, IaC changes, runbook steps — with verification.
Want the full API Security Remediation report?
Get model leaderboards, scoring methodology, dataset details, and roadmap access — or discuss generating proprietary remediation datasets for your environment.
Or email us directly at info@remediationlabs.com