Two-Arm Table Setter

Two simulated SO-101 robot arms set a dinner table from a plain-language command. They see with a camera, understand with a vision-language model, hand items to each other, and fix their own mistakes — all on an ordinary Intel laptop with OpenVINO. Built for the AI Infra Summit Hackathon, Intel online track.

10 random tables (harder settings: positions, sizes, colours), the command read by Qwen3-VL each time, a result card per table checked against simulator truth; then a recovery demo and the OpenVINO benchmark. Played at 2× speed.

30/30random tables set correctly

29/30harder tables: ±4 cm starts, sizes ±5–10%, colours

95%trained ACT policy, mug placed within 1.5 cm on unseen tables

2.5×faster policy with OpenVINO, task success unchanged

How it works

Observe

An overhead colour + depth camera finds every object — 0.5 mm average error.

Understand

Qwen3-VL-4B (INT4, OpenVINO) reads the command and the picture: "the red thing" is the mug.

Plan

Which arm, a hand-off for cutlery, a set-down swap for round items, what goes first.

Act

Two SO-101 arms; a LeRobot ACT policy, trained on a laptop CPU, handles the mug.

Look again

After every item, anything knocked out of place is put back first.

Why it matters

Service robots in homes, hospitals, hotels and restaurants need instructions anyone can give, two-handed handling, and the ability to check their own work. This runs on an ordinary Intel laptop, and its skills are taught from demonstrations the robot records itself — no hand-collected data.

Intel optimisation, measured on this laptop

Intel Core i7-1165G7 with Iris Xe iGPU, 16 GB, OpenVINO 2026.3. Full table: bench/RESULTS.md

ACT policy (CPU)	Latency	Throughput	Mug placed (10 tables)
PyTorch FP32	4.34 ms	230 calls/s	9/10
OpenVINO FP32	1.73 ms	578 calls/s	9/10
OpenVINO FP16	1.88 ms	532 calls/s	9/10
OpenVINO INT8 (NNCF)	3.68 ms	272 calls/s	9/10
OpenVINO INT8 weights	3.53 ms	283 calls/s	10/10

Qwen3-VL-4B INT4	CPU	Iris Xe iGPU	Correct
320 px picture	3.8 s	4.0 s	5/5
480 px picture (used)	4.3 s	4.2 s	5/5
960 px picture	14.9 s	7.5 s	5/5

Run it yourself

Every scene is seeded; every number here comes from one command. python brain/run.py --seed 3 "set the table"

Code on GitHub Setup guide Build log — including what failed

Honest limits: tested on an 11th-gen Core i7, not the Core Ultra the brief prefers (no NPU); most skills are scripted, one is a trained policy; the drawer and pouring from the example task are not included. MIT licence.