Large action models

Large language models (LLMs) are revolutionizing robotics and automated infrastructure systems by enabling more sophisticated interactions between machines, humans, and the physical environment. These models process and understand natural language commands, allowing humans to communicate with automated systems more intuitively. Building on this foundation, a new generation of Large Action Models (LAMs) is emerging, exemplified by Google's RT-1 and Microsoft's ChatGPT for Robotics. While LLMs focus on understanding and generating language, LAMs specifically translate language understanding into physical actions by learning from human demonstrations and generating novel sequences of movements and operations.

Like LLMs process text to understand language patterns, LAMs process visual and sensor data to understand and replicate human actions in the physical world. As these models become more widely integrated into urban robotics and autonomous systems, they will expand the possibilities for automation across sectors including transportation, logistics, construction, and infrastructure maintenance. This capability is particularly valuable for climate adaptation, as cities face increasing pressure to rapidly modify and maintain their infrastructure in response to extreme weather events and changing environmental conditions. LAMs can enhance urban resilience by enabling robots to autonomously perform complex repair tasks, adapt infrastructure systems, and respond to climate-related emergencies with greater speed and efficiency than traditional methods.