Emergence Meter
See how metrics create the illusion of emergence
1
Observe Emergence
"Emergent abilities" are capabilities that seem to appear suddenly as models get larger. But are they real, or are they measurement artifacts?
Try this: Select a task below and watch how the performance curve changes when you toggle between binary (exact match) and continuous (partial credit) metrics.
Computing 234 + 567 = 801
Source: Schaeffer et al. (2023) "Are Emergent Abilities of LLMs a Mirage?"
Binary: Answer is either completely right or completely wrong (creates sharp "emergence")
2
Adjust Parameters
Complete previous steps to unlock this step.
3
Complete Challenges
Complete previous steps to unlock this step.