ParseBench-2 · Metric comparison
We applied the same controlled errors and benign variations to 100 real pages per dimension, and scored them with the ParseBench-2 metrics and with about 90 metrics from other benchmarks. A metric follows the severity policy when it charges enough for an error and nothing for a benign variation. Pick a dimension, then an error, and see which metrics notice.
Each point is one metric. Right means it charges enough for errors; up means it leaves benign variations alone. The ideal metric sits in the top-right corner: it charges every error enough and penalizes no benign variation. Click a point to compare it with ParseBench-2 below.
Metrics ranked by the summary. Click one to select it.
Choose one perturbation and see how many points each metric takes off for it, against what the policy asks.
Mean score drop in points over the pages each perturbation changes. Click a row to open it above.