At a typical AI conference, there's usually much discussion of "the future of software", but it usually ends with agents producing traditional deterministic code in increasingly convoluted ways. Agents are contained to customer support popups on a website, or made multiplayer in the 67th clone of Conductor/Slack. Not very futuristic.
Returning to AIE to keynote the System One aka "Jev Day" track, Diogo Almeida of TypeSafe AI talked about the MISSION of Jev — not the 100 clones of the API, but the point of why it was necessary to establish the entire category of System One models, as a reaction against the entire category of Instruct-tuned LLMs that became ChatGPT.
In his words, "Where is all the automation?!?" Where is the GDP per capita impact of AI? Why is everyone working longer hours despite having Millennium-Prize-solving AI?
The simple answer is we overtuned for RLHF, and unsurprisingly tuning for more human feedback only serves to keep the human in the critical path