Incident Library?

API · MCP 한국어

Incident Library?

API · MCP 한국어

Component ?

Training data loader

학습 데이터 로더 · training-dataloader

1 case · 2025

Root cause ?Capacity exhaustion

Contributing factor ?Unverified assumption Overlapping failures Unrepresentative staging

Symptom ?Throughput degradation

Cases, by severity

IL-0129 Throughput of the 384-GPU SmolLM3 pretraining run collapsed because shared WekaFS storage evicted dataset shards to S3, then kept dropping on local NVMe because the nanotron dataloader built an index that grew with the step countHugging Face · 2025
Root cause Capacity exhaustion

View on the graph