API and MCP

What you get

Public post-incident reports, written up one case at a time and reviewed. Each case names the symptoms as the engineers saw them, the trigger, the root cause, the components involved, the contributing factors, what was done, what the organisation changed afterwards, the lessons, and links to the original write-up and an archived copy. Texts are in English and, where the reviewers wrote it, Korean.

The same cases are open three ways: this site for people, an HTTP API for programs, and an MCP server for AI tools. The id of a case (IL-0001 and so on), its summary and its source are the same on all three. No key is needed; everything is a GET.

Search by a sentence. Describe what you see. Each result carries a relevance from 0 to 1, and the answer counts the results by root cause, component, symptom and factor.
Topic pages. Every root cause, component, symptom and contributing factor has a page with its cases and what those cases carry. A sentence can restrict the page to the cases it finds.
Vocabulary by meaning. A few words find the slug to filter by: "cert ran out" finds certificate-expiry.

Attach the MCP server

The server speaks Streamable HTTP at https://incidentlib-mcp.benchlet.com/mcp. In Claude Code, one command:

claude mcp add --transport http incidentlib https://incidentlib-mcp.benchlet.com/mcp

In Cursor, add it to .cursor/mcp.json; any client that takes a Streamable HTTP server by URL uses the same address.

{
  "mcpServers": {
    "incidentlib": { "url": "https://incidentlib-mcp.benchlet.com/mcp" }
  }
}

In Claude Desktop, open Settings, Connectors, and add a custom connector with the URL https://incidentlib-mcp.benchlet.com/mcp.

Then ask

Our pods stopped resolving DNS after CoreDNS restarted. Which public incidents look like this, and what caused them?

The assistant calls the tools like this and answers with case ids it can cite:

search_incidents(symptoms="pods cannot resolve dns after coredns restarts", limit=5)
→ results: [{ id, url, title, org, date, relevance, root_cause, symptoms, components, contributing, summary, snippet, source }, …]
  facets: { root_causes: [{ slug, en, ko, count, cases }], components: [...], symptoms: [...], factors: [...] }

get_incident(id="IL-0001")
→ the whole write-up: symptoms_text, trigger, root_cause, components, contributing, summary, detection, remediation, action_items, lessons, sources, similar_cases

The five tools

search_incidents: Use first, whenever someone describes a failure. A symptom sentence in; the matching cases with relevance, url and source, plus facets with case ids, out.
get_incident: Use with a case id to read the whole write-up, including the similar cases.
list_incidents: Use when the question is about a category: every case with a root cause, component, symptom, factor, domain or organisation. Filters are slugs and combine with AND.
get_topic: Use to understand one category: its cases and the other categories they carry, with counts. A sentence restricts it to matching cases.
vocab: Use to turn words into a slug, or to list every slug with its names.

Call the API

Base address https://incidentlib-api.benchlet.com. Answers are JSON with snake_case fields; every case and topic carries a url, the page a person would read.

Search by a sentence

curl 'https://incidentlib-api.benchlet.com/search?q=pods+cannot+resolve+dns+after+coredns+restarts&limit=3'

A real answer, shortened. relevance is how well the case answers the sentence; cases below the floor are left out, so an unrelated sentence returns few or none. facets are counted over the returned results.

(the api did not answer while this page was rendered)

One case, and the cases like it

curl 'https://incidentlib-api.benchlet.com/cases/IL-0001'

{
  "id": "IL-0001",
  "org": "Zalando",
  "date": "2019-01-07",
  "docs": [
    "https://github.com/kubernetes/dns/issues/159#issuecomment-336449931",
    "https://nodejs.org/docs/latest/api/dns.html#dns_implementation_considerations",
    "… 1 more"
  ],
  "title": {
    "en": "Cluster-wide DNS outage after CoreDNS ran out of memory",
    "ko": "CoreDNS 가 메모리 한도에 걸려 죽으면서 클러스터 전체 DNS 가 멈춘 장애"
  },
  "domain": [
    "kubernetes",
    "aws"
  ],
  "impact": {
    "scope": "whole cluster DNS; customer-facing product pages",
    "users": "unknown",
    "region": "unknown"
  },
  "review": {
    "written_by": "session",
    "reviewed_at": null,
    "reviewed_by": null
  },
  "lessons": {
    "en": "DNS in the cluster was not resilient enough: one application could take it down for everyo…",
    "ko": "클러스터 안 DNS 가 너무 약했어요. 애플리케이션 하나가 모두의 DNS 를 무너뜨릴 수 있었어요. 클러스터 안에서 도는 모니터링은 클러스터와 함께 죽어요."
  },
  "related": [
    "IL-0005"
  ],
  "sources": [
    {
      "url": "https://github.com/zalando-incubator/kubernetes-on-aws/blob/dev/docs/postmortems/jan-2019-…",
      "kind": "github",
      "license": "unknown",
      "archived": "http://web.archive.org/web/20261004005813/https://github.com/zalando-incubator/kubernetes-…"
    }
  ],
  "summary": {
    "en": "CoreDNS pods ran with a 100Mi memory limit set from old usage figures. A spike of DNS quer…",
    "ko": "CoreDNS 파드의 메모리 한도가 예전 사용량 기준으로 100Mi 였어요. DNS 조회가 몰리자 한도를 넘어 OOM 으로 죽었고, 부하가 계속되니 살아나자마자 …"
  },
  "trigger": {
    "en": "Calls from the aggregation layer to one downstream service timed out; clients retried, and…",
    "ko": "집계 계층이 뒤쪽 서비스 하나를 부르다 타임아웃이 났어요. 클라이언트들이 재시도하면서 요청이 몰렸고, 그 요청이 그대로 CoreDNS 로 가는 DNS 조회 폭증으…"
  },
  "severity": "unknown",
  "symptoms": [
    "high-error-rate",
    "dns-resolution-failures",
    "… 2 more"
  ],
  "timeline": {
    "detected_after": null,
    "resolved_after": null
  },
  "date_kind": "incident",
  "detection": {
    "en": "An on-call engineer from the aggregation layer phoned the Kubernetes on-call engineer; aut…",
    "ko": "집계 계층 당직자가 쿠버네티스 당직자에게 전화했어요. 자동 호출은 클러스터 안 모니터링에 기대고 있었는데 그게 죽어 있었어요."
  },
  "components": [
    "coredns",
    "memory-limit",
    "… 1 more"
  ],
  "root_cause": "capacity",
  "remediation": [
    "raised CoreDNS memory requests and limits from 100Mi to 2000Mi by hand"
  ],
  "action_items": [
    {
      "text": {
        "en": "Run CoreDNS behind a per-node dnsmasq to contain the blast radius",
        "ko": "노드마다 dnsmasq 를 두고 CoreDNS 를 그 뒤에 둬서 영향 범위를 줄이기"
      },
      "status": "planned"
    },
    {
      "text": {
        "en": "Alert from outside the clusters",
        "ko": "클러스터 바깥에서 경보 보내기"
      },
      "status": "planned"
    },
    "… 1 more"
  ],
  "contributing": [
    "limit-unknown",
    "no-caching",
    "… 2 more"
  ],
  "recurrence_of": null,
  "symptoms_text": {
    "en": "Product pages on the storefront returned errors for more than an hour. Every DNS lookup in…",
    "ko": "상품 페이지가 한 시간 넘게 오류를 냈어요. 클러스터 안의 DNS 조회가 전부 실패했고, 클러스터 안에 있던 모니터링도 같이 죽어서 아무도 자동으로 호출되지 않았…"
  },
  "updated_at": "2026-11-08T06:09:02.994198Z",
  "url": "https://incidentlib.benchlet.com/cases/IL-0001"
}

curl 'https://incidentlib-api.benchlet.com/cases/IL-0001/similar'

[
  {
    "id": "IL-0016",
    "title": {
      "en": "Prometheus on the live cluster killed by memory exhaustion",
      "ko": "메모리 소진으로 죽은 운영 클러스터의 Prometheus"
    },
    "org": "UK Ministry of Justice",
    "date": "2023-07-25",
    "severity": "sev3",
    "root_cause": "capacity",
    "url": "https://incidentlib.benchlet.com/cases/IL-0016",
    "shared_components": [
      "memory-limit"
    ]
  }
]

Cases by category

curl 'https://incidentlib-api.benchlet.com/cases?root_cause=capacity&component=coredns'

Filters: root_cause, component, symptom, factor, domain, org, limit (up to 200). Newest first.

Topic pages

curl 'https://incidentlib-api.benchlet.com/symptoms/dns-resolution-failures'

The same path works for /root-causes/…, /components/…, /symptoms/… and /factors/…; without a slug, each lists every value with its case count. For a symptom, related.root_causes is "what usually causes this".

{
  "kind": "symptom",
  "slug": "dns-resolution-failures",
  "en": "DNS resolution failures",
  "ko": "DNS 조회 실패",
  "url": "https://incidentlib.benchlet.com/symptoms/dns-resolution-failures",
  "count": 2,
  "first_date": "2019-01-07",
  "last_date": "2024-12-11",
  "cases": [
    {
      "id": "IL-0005",
      "title": {
        "en": "Telemetry rollout overloaded the control planes and broke service discovery",
        "ko": "텔레메트리 서비스 배포가 컨트롤 플레인을 과부하시켜 서비스 디스커버리를 끊은 장애"
      },
      "org": "OpenAI",
      "date": "2024-12-11",
      "severity": "sev1",
      "root_cause": "config-change",
      "url": "https://incidentlib.benchlet.com/cases/IL-0005"
    },
    {
      "id": "IL-0001",
      "title": {
        "en": "Cluster-wide DNS outage after CoreDNS ran out of memory",
        "ko": "CoreDNS 가 메모리 한도에 걸려 죽으면서 클러스터 전체 DNS 가 멈춘 장애"
      },
      "org": "Zalando",
      "date": "2019-01-07",
      "severity": "unknown",
      "root_cause": "capacity",
      "url": "https://incidentlib.benchlet.com/cases/IL-0001"
    }
  ],
  "related": {
    "root_causes": [
      {
        "slug": "capacity",
        "en": "Capacity exhaustion",
        "ko": "용량 한계",
        "count": 1,
        "cases": [
          "IL-0001"
        ],
        "url": "https://incidentlib.benchlet.com/root-causes/capacity"
      },
      {
        "slug": "config-change",
        "en": "Configuration change",
        "ko": "설정 변경",
        "count": 1,
        "cases": [
          "IL-0005"
        ],
        "url": "https://incidentlib.benchlet.com/root-causes/config-change"
      }
    ],
    "components": [
      {
        "slug": "dns-resolver",
        "en": "Client-side DNS resolver and cache",
        "ko": "클라이언트 쪽 DNS 리졸버와 캐시",
        "count": 2,
        "cases": [
          "IL-0005",
          "IL-0001"
        ],
        "url": "https://incidentlib.benchlet.com/components/dns-resolver"
      },
      {
        "slug": "coredns",
        "en": "CoreDNS",
        "ko": "CoreDNS",
        "count": 1,
        "cases": [
          "IL-0001"
        ],
        "url": "https://incidentlib.benchlet.com/components/coredns"
      },
      "… 3 more"
    ],
    "factors": [
      {
        "slug": "failure-delayed-by-cache",
        "en": "Cache-masked failure",
        "ko": "캐시로 가려진 장애",
        "count": 1,
        "cases": [
          "IL-0005"
        ],
        "url": "https://incidentlib.benchlet.com/factors/failure-delayed-by-cache"
      },
      {
        "slug": "limit-unknown",
        "en": "Unknown limit",
        "ko": "한계 미인지",
        "count": 1,
        "cases": [
          "IL-0001"
        ],
        "url": "https://incidentlib.benchlet.com/factors/limit-unknown"
      },
      "… 5 more"
    ]
  }
}

With q, the page keeps only the cases the sentence finds, ordered by relevance, and counts the related values over them:

curl 'https://incidentlib-api.benchlet.com/root-causes/certificate-expiry?q=dns+lookups+fail'

Vocabulary

curl 'https://incidentlib-api.benchlet.com/vocab?q=cert+ran+out&kind=root_causes'

(the api did not answer while this page was rendered)

Without q, /vocab returns every slug with its English and Korean name, per kind.

Fields of a case

`id`	IL-0001
`url`	page on this site
`title`	{en, ko}
`org`	Zalando
`date`	2019-01-07
`severity`	sev1 … sev4, unknown
`root_cause`	slug
`components`	slug[]
`symptoms`	slug[]
`contributing`	slug[]
`domain`	slug[]
`symptoms_text`	{en, ko}
`trigger`	{en, ko}
`summary`	{en, ko}
`detection`	{en, ko}
`remediation`	string[]
`action_items`	[{text: {en, ko}, status}]
`lessons`	{en, ko}
`sources`	[{url, archived, kind, license}]
`impact`	{scope, users, region}
`timeline`	{detected_after, resolved_after}
`recurrence_of`	id or null
`related`	id[]

Machine-readable description: OpenAPI (JSON). Language: add lang=ko to search and case calls to get the Korean text where it exists. Fair use: when one address sends more than 60 requests a minute, the entrance answers 503 until the minute passes.

Where the data comes from

Each case is written from a public post-incident report: an engineering blog, a status page, a talk, a paper or a repository. The write-up is a summary in our own words with the facts the report states; a fact the report does not state is left empty, never guessed. Every source link has an archived copy where one could be made.

Vocabulary values (slugs) come from one ontology: root causes, components, symptoms, contributing factors and domains. Relations between a case and its values make the graph on the home page.

Licence: the write-ups, the vocabulary and the API answers are CC BY 4.0; the original reports keep their own licence, named on each source. To report a mistake or propose a case, open an issue at github.com/Team-DogFoot/incidentlib/issues.

Changes

2026-11-08 Public surface defined: search with facets, topic pages for root causes, components, symptoms and factors (restrictable by a sentence), vocabulary lookup by meaning, five MCP tools, this page, llms.txt.
2026-11-08 Search moved to OpenSearch with a Korean analyzer, multilingual embeddings and a re-ranking model; every result carries a relevance.
2026-11-08 First 25 cases approved (20 Kubernetes, 5 GPU and AI infrastructure), ontology version 4, English and Korean written together.