Large action models

The integration of large language models (LLMs) into robotics and automated infrastructure systems is transforming how they interact with humans and perceive an interact with the world around them. LLMs allow these computational systems to understand and respond to human commands. Going forward, a new generation of Large Action Models (LAMs) like Google's RT-1 and Microsoft's ChatGPT for Robotics translate these capabilities to the physical world, learning by observing human actions, and generating novel sequences of actions. Like LLMs, LAMs allow machines to perform complex tasks by mimicking human actions within software applications.

As Large Action Models become more widely used in urban robotic and autonomous systems, they will profoundly expand the range of possible applications from transportation and logistics to building construction, operation, and maintenance. For climate adaptation, this will enhance cities capacity to rebuild, repair, and operate systems essential to urban resilience.