Can LLMs Uncover Hidden Physical Laws through Free Exploration?
LLMs get no hints in worlds where physics is altered, and must notice anomalies alone. One of twelve models reached a 16.7% discovery rate; the other eleven scored 0%.
The claim
Almost no LLM notices altered physics on its own and discovers the changed law.
The problem
Scientific discovery often starts with an unexpected observation.
The model must notice anomalies without being told what to investigate.
The paper asks whether an LLM can find the changed law alone.
How it works
LongChang builds simulated worlds where one or more physical laws differ.
Explorer LLMs manipulate objects and build tools to test their own questions freely.
Reviewer agents generate scenes, judge tool designs, and evaluate the reports.
Exploration records then guide how each scene evolves further.
The whole cycle runs without any human intervention.
Figures
What changes
GPT-6-Astra reaches a 16.7% discovery rate among the twelve tested models.
The other eleven models score 0%.
Reports still leave anomalies uninvestigated and laws only partly identified.
Limits
Twelve models carry the whole evaluation.
The task gives no hint about what to investigate first.
Altered worlds may differ from real experimental research settings.
Source · arXiv