Two-Arm Table Setter

Two simulated SO-101 robot arms set a dinner table from a plain-language command. They see with a camera, understand with a vision-language model, hand items to each other, and fix their own mistakes — all on an ordinary Intel laptop with OpenVINO. Measured end to end on an Intel Core Ultra 7 155H, across its CPU, Arc iGPU and AI Boost NPU. Built for the AI Infra Summit Hackathon, Intel online track.

10 random tables (harder settings: positions, sizes, colours), the command read by Qwen3-VL each time, a result card per table checked against simulator truth; then a recovery demo and the OpenVINO benchmark. Played at 2× speed.

30/30random tables set correctly

29/30harder tables: ±4 cm starts, sizes ±5–10%, colours

95%trained ACT policy, mug placed within 1.5 cm on unseen tables

2.9×faster policy with OpenVINO on the Core Ultra CPU, task success unchanged

0.90 mswhole policy on the AI Boost NPU (FP32); 10/10 mugs placed at FP16

How it works

Observe

An overhead colour + depth camera finds every object — 0.5 mm average error.

Understand

Qwen3-VL-4B (INT4, OpenVINO) reads the command and the picture: "the red thing" is the mug.

Plan

Which arm, a hand-off for cutlery, a set-down swap for round items, what goes first.

Act

Two SO-101 arms; a LeRobot ACT policy, trained on a laptop CPU, handles the mug.

Look again

After every item, anything knocked out of place is put back first.

Why it matters

Service robots in homes, hospitals, hotels and restaurants need instructions anyone can give, two-handed handling, and the ability to check their own work. This runs on an ordinary Intel laptop, and its skills are taught from demonstrations the robot records itself — no hand-collected data.

Intel optimisation, measured on a Core Ultra

Intel Core Ultra 7 155H — CPU, Arc iGPU, AI Boost NPU — Windows 11, OpenVINO 2026.3.1. Raw logs: bench/core_ultra/. Full table: bench/RESULTS.md

ACT policy	CPU	Arc iGPU	NPU	Mug placed (10 tables)
PyTorch FP32	2.07 ms	–	–	9/10
OpenVINO FP32	0.83 ms	1.49 ms	0.90 ms	9/10
OpenVINO FP16	0.81 ms	1.52 ms	1.06 ms	10/10 on iGPU and NPU
OpenVINO INT8 (NNCF)	0.72 ms	1.42 ms	1.10 ms	9/10
OpenVINO INT8 weights	0.74 ms	1.93 ms	1.04 ms	10/10

Qwen3-VL-4B INT4	CPU	Arc iGPU	NPU	Correct
320 px picture	3.0 s	1.1 s	won't compile	5/5
480 px picture (used)	3.6 s	1.3 s	won't compile	5/5
960 px picture	10.9 s	2.8 s	won't compile	5/5

An honest negative result: OpenVINO 2026.3's NPU compiler aborts on this model's vision tower, so the language model detects an NPU request and falls back to the iGPU. The ACT policy runs on the NPU without trouble.

Run it yourself

Every scene is seeded; every number here comes from one command. python brain/run.py --seed 3 "set the table"

Code on GitHub Setup guide Build log — including what failed

Honest limits: the NPU runs the policy but cannot compile the vision-language model (OpenVINO 2026.3); most skills are scripted, one is a trained policy; the drawer and pouring from the example task are not included. MIT licence.