The end-of-line test, and the robot that kept driving
A factory doesn't ship a product because each part passed its own test. At the end of the line, one last test
checks the assembled thing: every sensor, every motor, every way it can stop. Our robot now has one: a single
script, make eol-run, that drives the robot through 33 checks and prints PASS or FAIL. It found two real
problems on its first two runs, in a simulator we thought we knew well.
One script, the robot's own sensors
The rule that shaped it: the end-of-line test may only use what the real robot has. Every other report in this devlog compares the robot against Gazebo's ground truth, which a real lawn doesn't have. This one uses the wheel encoders, the IMU, the GNSS, the lidar and the battery, so the script that passes in the simulator is the same script that will run on the real robot, unchanged.
It drives through the *_raw command topics like any mission, so every command goes through the safety gate: the
test exercises the whole chain, not just the motors.
| Step | What it does | Pass if |
|---|---|---|
| 1 sensors | Listens for 5 s, robot still | Every topic at ≥ 90% of its rate; gravity 9.81 m/s²; RTK fixed; a sane 7S battery |
| 2 wheels | Forward, pivot left, pivot right | Each wheel at the speed the command implies, with the right sign; IMU yaw rate agrees |
| 3 blade | Soft start, then off | 90% speed in ≤ 4 s; below 10% in ≤ 2 s |
| 4 STOP | Presses STOP at 0.3 m/s with the blade on, while the driver keeps commanding motion | Standstill ≤ 1 s; nothing moves while latched; release leaves it stopped |
| 5 watchdog | Drives with the blade on, then the driver goes silent | Drive stops ≤ 1 s, blade ≤ 3 s |
| 6 obstacle | Drives at a box and keeps asking to go on | The safety gate stops it ≥ 10 cm short |
| 7 manual | Lift, tilt, bumper, interlock connectors, hung microcontroller | A person does these and signs the build record |
Step 2 is the one that will earn its keep on hardware: a swapped motor connector or an IMU mounted backwards looks fine on a bench and turns the robot in circles on a lawn. Checking the sign of every wheel against the gyro catches both in three seconds.
eol-1: the robot that kept driving
The first run passed 25 of 27 automatic checks. The two failures were step 5: when the driver went silent, the robot didn't stop. It never stopped. It carried on at 0.3 m/s until the lidar saw the box and the safety gate stopped it 0.69 m short.
Gazebo's drive plugin keeps executing the last command it received, and so do many real motor controllers. Our safety gate passed commands through but never asked how old they were. So a mission that crashed, hung or lost its connection mid-stripe would have left the robot driving, blade spinning, in a straight line, with only the lidar between it and the fence.
The fix is a rule: commands are a heartbeat. A driver that wants motion has to keep saying so. If the gate hears nothing for 0.5 s (the limit our safety analysis already named for the motor controller) it sends zero, for the drive and the blade separately. A zero command needs no watchdog, so a parked robot isn't flooding the log. One driver, the old square-drive experiment, sent its blade command only once and relied on it sticking; it now repeats it every tick.
eol-2: the blade that twisted the robot
With the watchdog, step 5 passed (drive stopped in 0.51 s). Step 6 now failed in a new way: the robot saw nothing ahead but the fence 8 m away, and the box was not in front of it. The bag's ground-truth heading told the story:
| Sim time | What happened | Heading (ground truth) |
|---|---|---|
| 29 s | driving straight, blade on | 0.1° |
| 31 s | STOP pressed: blade cut | 7.9° |
| 47 s | driver silent: watchdog cuts the blade | 15.9° |
Each blade stop twisted the whole robot by 8°. The simulated blade motor had unlimited torque: told to stop, it stopped the blade within one 1 ms physics step. The blade disc stores 0.0081 kg·m² × 300 rad/s of angular momentum; dumping that in a millisecond means a reaction of about 2,400 N·m on the chassis, which overpowers the tyres. After two stops the robot pointed 16° off and drove right past the box.
We met this physics before, on the other side: the instant blade start that rotated the square by 6.7°. That finding was real physics, but with an unrealistically strong motor. A real blade motor has a torque limit and takes most of a second to stop. The simulated one now does too: torque control, capped at ±3 N·m.
| Run | Watchdog: drive stops | Blade stop (commanded) | Twist per blade stop | Result |
|---|---|---|---|---|
| eol-1 | never | 0.01 s | 8° | FAIL |
| eol-2 (+ watchdog) | 0.51 s | 0.01 s | 8° | FAIL (missed the box) |
| eol-3 (+ torque-limited blade) | 0.54 s | 0.75 s | < 0.1° | PASS |
The blade stop time is now 0.75 s instead of 0.01 s. The old number was never a believable one to test against; the last post already said the sim blade had no inertia. Also honest: with the motor capped, an instant start no longer twists the simulated robot much either. We keep the soft start (it's kinder to the motor and the battery), but its simulator evidence was partly an artefact.
The 2 s blade-stop limit is our target, not a measurement. Gate 3 measures the real motor, and the limit gets set from that, in the verification page, never quietly in code.
eol-4: the pivots don't cancel
eol-3 left one loose end: pivoting left for 2 s then right for 2 s should end where it started, but the robot ended 6.5° off. The IMU-rate checks passed (within 15%), and nothing else depends on the pivots being symmetric. Rather than chase it, the test now uses the robot's own gyro to turn back to the start heading before step 6, so the box is always in front of it. The report records how far off it was.
eol-4: the gyro said −6.1° after the pivots, the robot turned back, and every automatic check passed again (blade stop 0.74 s, watchdog 0.52 s, 19 cm short of the box).
Did the changes break mowing?
A watchdog that cuts a silent driver could also cut a quiet one: a mission that sends a command only when it changes would now stop every 0.5 s. So the same scripted test-patch mission as last time (ui-test-3: STOP press, release, go home, resume) ran again, with all four reports:
| Test patch | ui-test-3 (before) | ui-test-4 (watchdog + torque-limited blade) |
|---|---|---|
| Coverage of the reachable area | 97.3% | 96.9% |
| Worst cross-track | 8.4 cm | 8.2 cm |
| Localization | 2.2 cm RMS | 2.4 cm RMS |
| STOP: standstill / blade below 10% | 0.22 s / 0.02 s | 0.04 s / 0.76 s |
| Dockings | first try | 2 of 2, first try |
| Watchdog firings during the mission | — | 0 |
The mission already sends a command every tick, so the watchdog never fired. The STOP blade time is now the believable 0.76 s.
The build manual grew
- End-of-line test: what it checks, how to run it on a real robot
(
blade:=false lidar:=falseon the H2 build: no blade, no lidar yet), and the build-record entry it writes. - H2: brain transplant and H3: blade on: drafts, written before the hardware exists, from the build guide and the safety checklist. The order and the pass criteria are fixed; the physical how-to gets rewritten from the real build.