Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Recently I have given My OpenClaw A real robot arm to play with. The result just destroyed my neural network.
The AI assistant was able to program the arm, use it to see and touch objects slowly, and train another AI model to pick up and place real objects. And they say AGI is still a few years away! (Just kidding, maybe I am).
The results have convinced me that we can be close to robotics. Training and controlling the robots used requires a lot of skill. Today’s AI models can make it easy.
“AI-driven modeling is very exciting because it can bridge the gap between conventional engineering methods, which are reliable but inconsistent, and today’s models, which are stable but not reliable,” says Ken Goldberg, a robotics expert at UC Berkeley who is evaluating the method.
I bought a pre-built arm called a LeRobot 101. It’s part of an open source project from HuggingFace that makes it affordable to start building and testing robotics.
LeRobot comes with two arms: a control arm that a person uses to handle and shoot, and a follower arm with a camera that captures the action. You can train an AI model using the phone’s control arm and have the model learn how to move the tracker by following what it sees on the camera.
Building with OpenClaw
Before using OpenClaw, I spent several hours trying to connect and control the robot, at one point almost breaking the motors by using the wrong settings, which caused them to overheat.
Then, with the help of OpenClaw and Codex, I was able to call a simple program that closed the claw barrier when it saw a red ball. In the terminal, Codex went through the tricky task of fixing the connections to the robot. Then, with my help, it arranged the location of its points. It also wrote a Python script that used several libraries to detect and catch the ball in question. Vibe-coding isn’t perfect, and visualization can cause bugs especially when working with different devices, but the results were impressive.
