In robotics, natural language is an accessible interface for guiding robots, potentially empowering individuals with limited training to direct behaviors, express preferences, and offer feedback. Recent studies have underscored the inherent capabilities of large language models (LLMs), pre-trained on extensive internet data, in addressing various robotics tasks. These tasks range from devising action sequences based…

Numerous challenges underlying human-robot interaction exist. One such challenge is enabling robots to display human-like expressive behaviors. Traditional rule-based methods need more scalability in new social contexts, while the need for extensive, specific datasets limits data-driven approaches. This limitation becomes pronounced as the variety of social interactions a robot might encounter increases, creating a demand…

Since it enables humans to teach robots any skill, imitation learning via human-provided demonstrations is a promising approach for creating generalist robots. Lane-following in mobile robots, basic pick-and-place manipulation, and more delicate manipulations like spreading pizza sauce or inserting a battery may all be taught to robots through direct behavior cloning. However, rather than merely…

Robotics is currently exploring how to enhance complex control tasks, such as manipulating objects or handling deformable materials. This research niche is crucial as it promises to bridge the gap between current robotic capabilities and the nuanced dexterity found in human actions. A central challenge in this area is developing models that can accurately indicate…

The problem of achieving superior performance in robotic task planning has been addressed by researchers from Tsinghua University, Shanghai Artificial Intelligence Laboratory, and Shanghai Qi Zhi Institute by introducing Vision-Language Planning (VILA). VILA integrates vision and language understanding, using GPT-4V to encode profound semantic knowledge and solve complex planning problems, even in zero-shot scenarios. This…

The team of researchers from NYU and Meta aimed to address the challenge of robotic manipulation learning in domestic environments by introducing DobbE, a highly adaptable system capable of learning and adapting from user demonstrations. The experiments demonstrated the system’s efficiency while highlighting the unique challenges in real-world settings. The study recognizes recent strides in…

In this paper, researchers have introduced a NeRF-based mapping method called H2-Mapping, aimed at addressing the need for high-quality, dense maps in real-time applications, such as robotics, AR/VR, and digital twins. The key problem they tackle is the efficient generation of detailed maps in real-time, particularly on edge computers with limited computational power. They highlight…

With the increase in the popularity and use cases of Artificial Intelligence, Imitation learning (IL) has shown to be a successful technique for teaching neural network-based visuomotor strategies to perform intricate manipulation tasks. The problem of building robots that can do a wide variety of manipulation tasks has long plagued the robotics community. Robots face…

Teaching robots complicated manipulation skills through observation of human demonstrations has shown promising results. Providing extensive manipulation demonstrations is time-consuming and labor costly, making it challenging to scale up this paradigm to real-world long-horizon operations. However, not all facets of a task are created equal. A new study by NVIDIA and Georgia Institute of Technology…

In many domains that involve machine learning, a widely successful paradigm for learning task-specific models is to first pre-train a general-purpose model from an existing diverse prior dataset and then adapt the model with a small addition of task-specific data. This paradigm is attractive to real-world robot learning since collecting data on a robot is…