Understanding time · controlled experiment
The model uses timing information to select the right interval
Correct timing substantially improved interval selection. The improvement also held when the text stayed identical, supporting the conclusion that the model uses timing rather than relying only on textual clues.
Experimental criteria met · three training runs
Research question
Does the model use actual timing information to select the correct interval between events, or does it guess from clues in the text?
Method
The model was trained in three runs with different random initial settings. In each run, correct timing was compared with randomly mixed timing and timing set to zero. An additional test kept the text identical while changing the times and the correct answer. Success criteria were set before training.
Results
On the main test with time intervals outside the training conditions, correct interval selection averaged 92.75% with real timing and 29.42% with randomly mixed timing: a difference of 63.33 percentage points. For pairs with identical text, both answers were correct in 75.33% of groups with real timing, 2.33% with mixed timing and 0% with zeroed timing. All three runs met the predefined experimental criteria.
Limitations
The experiment tests selection of a time interval from given options. It does not establish a ready-to-use language model that can answer any time-related question. An earlier experiment also left reliable matching of repeated intervals to events unresolved.
About this report
These are technical research reports, not peer-reviewed publications. Each report identifies its source, measurement conditions and limitations. The evidence section explains what can be downloaded or reproduced.
Technical notes and source records
Experiment: EXP011g. A seed is a setting that makes random training initialisation reproducible; the runs used 42, 43 and 44. OOD denotes testing outside training conditions. The primary metric is pair_acc; group_both records joint success on paired examples. Detailed measurements are in the linked tables.
The file names and SHA-256 fingerprints identify the exact source versions used to prepare this report.
- EXP011g-placebo-gates.md
SHA-256: 8986d1842c8084791b3137047669833b67deeae00004f50919b6c0baaaa5371e