Benchmark route · under review
Five methods, four routes, one protocol. No method is told how many regions an image contains.
| Route | Metric | LISA | GSVA | GLaMM | Sa2VA | Detecture |
|---|---|---|---|---|---|---|
| RWTD | mIoU | 0.7541 | 0.3706 | 0.6225 | 0.5452 | 0.7843 |
| ARI | 0.4413 | 0.4395 | 0.4499 | 0.4569 | 0.5993 | |
| RWTD-COCO | mIoU | 0.6837 | 0.4099 | 0.6014 | 0.2617 | 0.8460 |
| ARI | 0.3364 | 0.5001 | 0.4222 | 0.2362 | 0.7351 | |
| TextureADE | mIoU | 0.5752 | 0.2010 | 0.3840 | 0.3092 | 0.7222 |
| ARI | 0.3869 | 0.2402 | 0.1851 | 0.1970 | 0.6863 | |
| CSTD | mIoU | 0.8394 | 0.6578 | 0.7577 | 0.7510 | 0.8642 |
| ARI | 0.6208 | 0.7656 | 0.6177 | 0.7155 | 0.7742 |
| Route | Metric | LISA | GSVA | GLaMM | Sa2VA | Detecture |
|---|---|---|---|---|---|---|
| RWTD | Cov. | 0.5419 | 0.4230 | 0.4201 | 0.3987 | 0.7946 |
| B-F | 0.0000 | 0.1179 | 0.0000 | 0.1025 | 0.4252 | |
| RWTD-COCO | Cov. | 0.4817 | 0.4247 | 0.3558 | 0.2398 | 0.8536 |
| B-F | 0.0000 | 0.1575 | 0.0000 | 0.1036 | 0.4226 | |
| TextureADE | Cov. | 0.3155 | 0.2092 | 0.1674 | 0.1793 | 0.7479 |
| B-F | 0.0000 | 0.1795 | 0.0000 | 0.0516 | 0.5038 | |
| CSTD | Cov. | 0.5094 | 0.7145 | 0.4844 | 0.5958 | 0.8835 |
| B-F | 0.0000 | 0.3976 | 0.0000 | 0.2871 | 0.6499 |
Ground truth is exactly 2 on every image, marked on each bar.
GSVA is the one baseline that always returns a partition; it simply returns the wrong number of parts. Its high CSTD ARI (0.7656, close to Detecture's 0.7742) is the reward for that, since over-fragmentation still agrees with the ground truth on most pixel pairs.
CSTD's released region maps are the stitching masks fed into the generator, not annotations of what came out. Where the generator invented a third material or drifted from the mask, the ground truth silently stops being true. All 10,000 images were screened and ranked, 1,296 candidates were reviewed by eye, and 274 passed — an acceptance rate of 21%. The top 256 are what these numbers are computed on. Anyone rerunning CSTD off the shelf will get different numbers.
It enters only as an exactly-two-region control, not as evidence about real scenes. The real-world evidence is RWTD, RWTD-COCO and TextureADE.
One protocol, unchanged across all five methods and all four routes: no ground-truth region count in the prompt, no inverse-mask completion, no truncation of proposals to a known K, and no dropping of images where a method returns nothing. Each baseline runs under its own pinned environment, the same ones that reproduce the RWTD, RWTD-COCO and TextureADE numbers.