ratm · inverse reinforcement learning

Comparing Human and Autonomous Drone Flight Behavior

Bellow is 6-DOF quadrotor simulator with motor-level dynamics and a Betaflight-style flight controller, rendered in WebGL and running entirely in this page — plug in a RadioMaster Pocket transmitter (USB joystick mode) or use the keyboard, then chase lap times through the gates. After that, explore inverse reward learning on the Race Against the Machine drone-racing telemetry, recovering the objective a human pilot optimizes and how it differs from an autonomous controller's from 500 Hz flight logs,. Source code is on GitHub.

Watch the walkthrough

A short video tour of this page — the ideas behind it, followed by a live flight through the simulator below.

input: …
Settings

Flight

Mode
Wind

Recording

Stream to
Video

Transmitter

No calibration is normally needed — remap only if your model uses a different channel order.

Controls
WS
throttle
AD
yaw
pitch / roll
Enter
arm
R
reset
M
angle / acro
C
camera
P
autopilot
Flying the track

Arm with throttle low, then throttle up. Fly the glowing gate in order — a light beacon marks the next gate and the page keeps per-track lap times. The tracks are the real gate layouts from the two datasets analyzed below: the TII RATM arena (4 gates, two straights with 180° turns and a 1.6→4.4 m altitude weave, gate headings taken from the human pilots' actual flight direction through each gate) and the UZH AlphaPilot figure-8 (10 gates, 50×20 m, flat and wave variants), coordinates lifted directly from the flight logs. Angle mode self-levels; Acro is real rate-mode flying with Betaflight-style expo — switch between them under Settings → Flight, along with the wind level.

Flying with an RC transmitter

RadioMaster / any RC transmitter: put the radio in USB joystick mode (EdgeTX: plug in USB → “USB Joystick (HID)”), then press a button or flip a switch once — browsers deliberately hide gamepads until a button is pressed, so moving the sticks alone won't wake it. (If nothing happens, map a switch to a button channel in the EdgeTX USB joystick setup.) It is read via the browser Gamepad API, so it works the same on this deployed site as it does locally. RadioMaster/EdgeTX devices use the browser layout observed for these radios. Firefox's standard-gamepad mapping exposes throttle as analog trigger/button 6 rather than an axis; the simulator handles that automatically. Other unmapped browsers fall back to the MoBI simulator's raw AETR channel order. No calibration is normally needed. If your model uses a different channel order, remap with Settings → Transmitter → Calibrate sticks (remembered only for that exact controller), or restore the built-in mapping with Use controller defaults. The arm switch is auto-detected: flip any spare switch once and that switch arms/disarms from then on. The input: readout in the control bar shows the live source the sim is receiving. With a radio connected, A arms, D disarms, and Space resets; after a crash, disarming respawns you. The Gamepad API needs a secure context — HTTPS, or localhost during development. Linux Firefox note: Firefox only exposes a gamepad after a button press, and RC radios in classic joystick mode emit none — use a Chromium-based browser to fly with a transmitter.

Recording flights for offline RL

Hit ● REC and every armed span becomes a logged episode — 100 Hz state (position, velocity, attitude, body rates, motor speeds), your stick inputs, the motor commands the flight controller issued, and a per-step reward (gate progress + pass bonus − crash penalty − control effort, the same shape as the task reward in the research below) — plus webm video of the canvas. It streams as one NDJSON session to the destination you pick under Settings → Recording: browser download, or local disk / S3 via the same local bridge the EEG page uses (python -m eeg_lab.serve). Each episode also carries what a learner needs but a screen recording loses: the gate layout it was flown on, attitude and body rates converted into the research code's frame, and the rate setpoints the controller derived from your sticks — in Angle mode the sticks are tilt commands, so the setpoints are the only action channel that means the same thing in both modes. python -m ml.drone_rl.browser_data converts recordings into per-episode (obs, action, reward, done) datasets for offline RL, and python -m src.browser in ragtm/ converts them into dataset-format flights, so your own laps join the RATM pilots as demonstrations for the agents below.

What objective does a racing pilot optimize?

The RATM dataset (TII) has 18 human-piloted and 18 autonomous flights on the same tracks — 500 Hz state, RC-stick, and gate telemetry, six of the human flights flown by a world champion. Inverse RL recovers a reward function from each cohort; the UZH eye-gaze dataset (21 pilots, a different quad, a simulator) tests whether what's recovered is real or an artifact.

36RATM flights · 500 Hz · human + autonomous
21UZH pilots · 208 runs · 435 min
10.9 m/schampion average — ordinary pilots: 4.4–5.2
3independent IRL methods, one conclusion
2 MPPO steps flown in an independent physics sim
The racing lines, in space and time
every RATM flight, rendered from the 500 Hz telemetry — comets fly each trajectory in real time; color by cohort, speed, or stick activity
t = 0.0 s RATE TRACK COLOR SHOW

What each objective weighs — linear reward weights
logistic reward over 7 interpretable features, expert vs. perturbed counterfactuals · RATM, 50 Hz
smoothness (control rate) −0.51 −0.16
control effort +0.17 −0.14
speed −0.01 +0.17
jerk +0.16 +0.17
heading alignment +0.01 −0.01
Human pilots Autonomous controller

The headline split: humans are smoothness-dominated (control rate −0.51), the machine is speed-dominated (+0.17). Replicated on UZH with disjoint pilots and different hardware — control rate again dominant and negative (−1.53).

Does the recovered reward survive a different drone?
human-vs-autonomous discrimination accuracy, RATM-trained · evaluated in-domain and on the UZH AlphaPilot sim (0.75 kg real quad → 3.2 kg simulated, disjoint pilots)
Feature-space reward
7 physics features
0.977 0.974
State-space reward
raw 20-d state
chance 0.50 0.913 0.502
In-domain Cross-dataset

Feature-space rewards transfer essentially losslessly (0.977 → 0.974); the same reward learned on raw state collapses to coin-flip. "Smooth sticks" is a property of pilots, not of one drone.

Three findings

champion

The champion flies like the machine

Split out the world champion's six flights and their recovered reward scores autonomous flying (+0.18) above ordinary human flying (−0.14). Deconfounded on UZH — 21 pilots, identical tracks — skill still tracks objective space: per-pilot r = −0.44 ± 0.10 between lap time and human-objective score.

pixels

Pixels beat mocap when data is scarce

Behavior cloning from frozen V-JEPA 2 latents (FPV video only) gets 2.5× lower error than privileged mocap state with a single training flight; the lines cross only at ~8–12 flights. Vision pretraining is worth about seven flights of telemetry.