Training Robotic fine Motor Skills Using AI

Researchers have developed a new technique to train robotic hands to perform complex dexterous tasks using artificial intelligence (AI). The system, called Eureka, allows robotic hands to learn very sophisticated fine motor skills such as pen-spinning better than human experts.

Eureka was created by researchers from NVIDIA, the same team behind Voyager, an AI agent that explores Minecraft worlds. Eureka uses the language model GPT-4 to automatically generate reward functions that train robotic hands in a physics simulator.

There are four key components to Eureka:

1. GPT-4, which generates reward functions in Python based on a prompt and simulation environment code.

2. Isaac Gym, the GPU-accelerated simulator, which trains reinforcement learning policies using the rewards.

3. A process called Evolutionary Search, in which Eureka samples multiple reward functions from GPT-4 and iteratively improves them through trial and error.

4. Another process called Reward Reflection during which performance feedback about the rewards is provided back to GPT-4 to iterate and generate better versions.

In tests across a variety of simulation tasks, Eureka outperformed experts on 83% of tasks, with 52% higher scores on average. It achieved complex maneuvers, like pen-spinning, not feasible manually and improved over multiple iterations even above human-level.

Eureka demonstrates that coding language models combined with evolutionary algorithms can autonomously design training systems competitive or superior to human experts. This approach may enable reinforcement learning agents to perform complex dexterous manipulation skills not previously feasible.

Sources:

Eureka | Human-Level Reward Design via Coding Large Language Models (eureka-research.github.io)