Verification and acceptance tests
Verification = the simulator's reports, run on the real robot. Every behaviour in the devlog was proven in the simulator with a script that reads a recorded run and prints PASS or FAIL. The same scripts, with the same thresholds, are the robot's acceptance tests. Where real hardware needs a looser limit, the limit is written here with the reason, never quietly changed in code.
On the real robot there is no simulator "ground truth"; RTK GNSS (2–3 cm) plays that role, so thresholds tighter than about 3 cm can't be checked outdoors. Those are marked bench.
Gate 1: sensors (H1)
| Test | Script | Pass if |
|---|---|---|
| GNSS static | gnss_static_report |
RTK FIXED within 5 min of a cold start and ≥ 95% of the time after; ±3 cm per axis (95% of fixes) over ≥ 10 min |
| IMU and odometry noise | sensor_report |
Measured noise within 50% of the model's; heading drift ≤ 1° over the run (bench, on a turntable) |
| Data rate | sensor_report |
≥ 90% of each sensor's nominal rate; pairs aligned within 11 ms |
Gate 2: driving, blade off (H2)
| Test | Script | Pass if |
|---|---|---|
| Drive a square | drive_report |
Speed within 5% on straights; path error ≤ 10 cm; closes within 30 cm |
| Localization | localization_report |
Error ≤ 5 cm RMS, never above 15 cm; yaw ≤ 2° RMS |
| Mow a lawn (blade disconnected) | coverage_report |
≥ 95% of the reachable area covered; no keep-out incursions; cross-track ≤ 15 cm while mowing |
| Energy | power_report |
Never below 10% state of charge; returns to the dock on its own |
| Docking | power_report / dock events |
10 out of 10 docks, retries allowed |
| Obstacles | obstacle_report |
Stops ≥ 10 cm from every obstacle on the course (boxes, a hose, a soft toy) |
| Buttons | ui_report |
STOP: standstill ≤ 1.0 s; no motion while latched; release leaves it paused; "go home" docks and waits for GO |
Gate 3: blade on
Only after gate 2 passes, and with the end-of-line test passing with the blade on. The steps are in H3: blade on. Every item is a recorded run, analysed by a script, and kept with the build record.
| Test | Pass if |
|---|---|
| STOP button | Blade and drive stop; blade stop time measured and recorded; blade does not restart on release |
| Lift either side | Blade stops within the measured stop time; no automatic restart |
| Tilt past the limit | Blade stops |
| Kill the ROS software | Blade stops (watchdog) |
| Hang the microcontroller | Blade stops (hardware watchdog) |
| Unplug each interlock connector in turn | Blade stops (fail-safe wiring) |
| RTK fix lost | The robot stops inside the geofence margin |
| Bumper hit | Drive and blade stop |
| 10 supervised mowing runs | No safety event. Only then any unattended mowing |
If any gate-3 test fails
Disconnect the blade motor, tape the connector, and record the failure in the defect log. Don't retest until the cause is understood and fixed; a "passed on the second try" is a fail.