Prompt “engineering”Perhaps the most curious thing about these AI discoveries is how easy the prompting is. The model won’t even try: it’ll just respond something like “as a language model, I can’t solve such a hard problem”. In the long run, AI discoveries will naturally become part of the training data. Because of this, I expect that even if AI capabilities stalled out, the pace of AI discoveries will accelerate. In fact, if there truly is an intelligence overhang in frontier models, tuning models to make them more self-confident will likely make them more intelligent by default.