Things that move

A post in the lawn is an obstacle: stop, go round it. A child is not, even standing still: stop, blade off, and wait until they've gone. The camera will one day say "that's a child". Until then the lidar can still tell one thing apart: something that moves. This post teaches the robot to see that, and to treat anything that moves as a person.

The rule

Decided at the start of this phase ("better not even bump into anyone"):

Something that moves within 5 m: blade off, slow down, and the mission pauses.
Within 2 m of the robot's body: full stop.
Both hold until it has been gone for 30 s. If it's still there after five minutes, the robot gives up and waits for the owner.

A thing that has moved stays "a thing that moves", even if it then stands still. A child who walks up and sits down in front of the robot is still a child. And if it walks right up beside the robot, where the lidar can't see the ground next to the deck, the robot remembers it's there.

How the lidar sees motion

Each scan, the points that stick up out of the grass are grouped into clumps on a 15 cm grid and followed from scan to scan, in map coordinates, so the robot's own driving is taken out. A clump becomes a mover when:

it's been seen in 5 of the last 7 scans (not a flicker of grass),
it has moved at least 0.4 m, and more than half its own size plus 0.3 m. What the lidar sees of a still bench is the side facing it, and that side's middle shifts as the robot drives round it, but never out of the bench's own footprint,
and it is standing where the lidar saw empty space before.

The third rule turned out to be the important one. All the ones before it let walls through.

The house wall that walked

The first runs had a simulated person walk across the lawn ahead of the robot. They were seen. But so was the house.

The robot mows rows that run close by the house's south wall. From there, the lidar sees the wall at a grazing angle, and at a grazing angle a wall comes back as a scatter of separate fragments, not one line. As the robot drives, the fragments slide along the wall, split and join up again. To the tracker, a fragment that slid 0.5 m is a thing that moved 0.5 m. The robot stopped for its own house, waited 30 s, set off, and stopped again.

So, rule 3: a person walks into space the lidar's beams used to pass straight through. A wall was never empty. The robot now keeps a map of where its beams have passed, at person height, in the last minute. A clump only counts if at least 30% of it stands on space that was empty before.

That helped less than expected, and finding out why needed better tools than a ten-minute sim per try. So we recorded every lidar scan of a run (1,701 of them) with the robot's position, and replayed them through the tracking offline in seconds. The replay showed three ways a wall still looked like "empty space before":

Beams graze the wall. A beam that hits the wall 6 m away passes within a few centimetres of the wall for the last metre or two. Between the wall's sparse returns, that marked the wall's own cells "empty". Now a cell that held a return, or is within 30 cm of one, in the last minute is never marked empty.
The pose is only so good. A heading 1–2° off puts a wall 5 m away 10–20 cm onto the lawn in front of it, and that lawn really was empty. The 30 cm margin covers that too.
The newest pose isn't the scan's pose. The safety check used the robot's latest position, not its position when the scan was taken. In a pivot at 1 rad/s, a tenth of a second late is 6° off. A tree trunk 3 m away jumped 30 cm sideways and became a "mover" that held the robot for five minutes. Now each scan is placed with the pose at its own timestamp, from a short history.

In the replay that took the false movers from 13 to 0. Then the live runs found one more:

The robot forgets. "Something was here" lasted a minute. The robot mows rows away from the house for longer than that, and every time it came back, the beams grazing the wall marked it empty again. A false mover at the house corner on every pass. Now anything that has returned the lidar's beams 30 cm or more above the ground (no grass is that tall) in 10 scans is structure for the rest of the mow: walls, fence posts, tree trunks. A person walking across a spot is seen there in 2–4 scans and doesn't count.

One more safety net: if a "mover" ends up standing on what was already structure before it moved, it's let go. It was a wall or a trunk that a mover passed. A child who sits down sits where the lawn was empty, and stays a child.

Five more, from the live runs

The replay tool made each of these an hour's work rather than a night's:

Bumps and the hose. On the bumpy lawn, the tops of bumps came and went as the robot rocked, and a few became "movers". So did the 4 cm garden hose in the obstacle course. Now nothing under 25 cm tall can be a mover. It's still an obstacle; it just can't be a person. A cat is taller than that.
Beams over the fence. "Empty space" was carved by every beam, at every height. A beam passing over a fence or a flowerbed from far away said "the fence's cells are empty". Now only beams at leg height (15–60 cm) count: everyone who moves fills that band, and grass is below it.
The heading jumps. The robot's global position estimate sometimes swings its heading by up to 0.4 rad (23°) in one step on a turn. One scan placed with a heading that wrong puts the whole house a metre over. Now a scan taken while the pose is jumping isn't used for tracking at all; the stops still use it. The jumps themselves are a bug in the position estimate, and fixing that is next.
A dog that was never let go. The robot remembers a mover that walks beside the deck, where the lidar can't see the ground. A second, stale track of the same dog was remembered 0.45 m out, where the lidar sees perfectly well, and held the robot after the dog had run off. Now only a mover within 35 cm of the body is remembered, and a remembered one is dropped as soon as a live track turns up where it was.
No return isn't nothing. A beam with no return was taken as "empty all the way out to 10 m". Beams grazing a thin fence 10 m off came back empty and said the fence wasn't there. Now a beam with no return clears only 6 m.

The wall seen edge-on

The last one was the house again. In the back yard the robot mows the 1 m gap between the house and the fence. From there, the lidar sits almost exactly in the plane of the house's north wall. A beam that hits a wall edge-on can land anywhere along it: one scan at 2 m, the next at 4 m. To the tracker that's something walking along the wall at several metres a second, and it was standing on cells the robot had never seen before, so nothing said "structure".

Replaying with the simulator's true positions gave the same false movers, so this wasn't a position error. It's geometry. The fix is a classic lidar check: if the returns either side of a point, in the same ring, lie along that point's own beam, the surface is seen edge-on. Those returns are left out of the tracking (they still count as obstacles for stopping). A person or a dog gives returns side by side across the beams, never along them. In the replay: 9 false movers to 0, and the real person, dog and child were still picked up at the same moment as before.

One last surprise came from the dog. With the edge-on fix in, it was suddenly seen at 1.9 m instead of 4. The replay showed the tracker had been blind for 1.2 s, exactly while the dog ran in: the "pose is jumping" check had fired. The robot's position gets nudged about 5 cm each time a GPS fix comes in, and between two pose updates 1/30 s apart, 5 cm reads as 1.5 m/s. Now a jump is judged by its size (10 cm or 3° more than the robot could have moved), not its rate. The dog was back to being seen at 4 m.

The test person walked through the fence

The simulated person, dog and child follow scripts laid out from wherever the robot is when they start: "walk across 3.5 m ahead, from 5 m on its left to 5 m on its right". One run started that with the robot beside the north fence, so the person started 3.6 m outside the yard and walked through the fence. Another sent them through the tree. The scripts now stay inside the fences and out of the house.

Results

Three scenarios on the open patch, 4 cm grass, the robot mowing at full speed when the mover appears, with the final code:

Scenario	Seen at (from the body)	Closest	What the robot did	Result
A person walks across 3.5 m ahead	2.70 m	2.40 m	blade off, paused, waited, finished the mow	PASS
A dog runs at the robot from the side at 3 m/s	4.08 m	0.44 m (the robot stopped; the dog came to it)	blade off, full stop, waited, finished	PASS
A child walks up and sits 4 m ahead for 45 s	4.12 m	2.46 m	blade off, paused for the whole sit, finished	PASS

And the runs where nothing moves, to count false alarms:

Run	Coverage	False stops per 100 m²	"Something moves" alarms
Obstacle course (pole, box, child, ball, hedgehog, hose)	7 of 7 PASS	–	0
Back yard, full mow	98.1%	0.0	1: a fence 4.4 m away, for 12 s

That last one is honest bad news. At that moment the robot's global position estimate had its heading 7.4° wrong, and 7.4° at 4.9 m moves a fence 60 cm. The same run then failed to dock on three tries. The heading estimate wobbling on some turns has turned up all through this post, and it's what we fix next, before the camera.

Not covered yet

A person standing still from the start. To the lidar alone, that's a post: the robot stops for it like any obstacle, but doesn't know to wait 30 s after it leaves. Recognising a person is the camera's job, next.
Real lidar noise. The simulated lidar has no motion smear and tidy noise. The margins here (30 cm, 10 scans, 30%) are starting points for the real lawn.
Bumpy-lawn grass. On one bump, 7 cm grass sometimes reads 10 cm and stops the robot. It did before this post too; it's on the list.
Next: the heading wobble, then the camera (5.3). Labelled pictures of what the robot sees, and an honest number for how fast a Pi 5 can do it.