A conveyor sorter turns one image into a physical decision, so model accuracy is only one part of the system. Lighting, motion blur, camera timing, memory pressure, uncertain objects and actuator latency all matter. A credible ESP32-S3 prototype should be able to say “unknown,” measure its end-to-end timing and fail to a safe reject path.
The Espressif camera driver supports ESP32-S3 and multiple sensor/output formats; choose a frame size and pixel format that fit the actual PSRAM, bandwidth and inference path. Benchmark capture, conversion, preprocessing and model time together. A frame that is already stale when inference ends can send a gate command to the wrong object.
Collect images across shifts, material condition, color, dirt, folds, orientation, speed and lighting variation. Split training and evaluation by collection session or physical item, not by near-duplicate frames from the same clip. Include visually similar classes and objects that should be rejected. Keep class definitions operational: “recyclable” may depend on local rules and material composition, not appearance alone.
Crop or resize deterministically, document normalization and ensure training preprocessing matches firmware exactly. Measure per-class precision and recall, confusion matrix, unknown/reject rate and performance under expected line speeds. A single overall accuracy can hide a dangerous or expensive class-specific failure.
ESP-DL uses its own supported model format and quantization flow; a generic int8 file is not automatically deployable. Select operators and quantization supported by the chosen ESP32-S3 toolchain, prepare representative calibration data and compare quantized outputs with the reference model. Profile peak memory, latency, frame drops and temperature on the target board. If a model does not fit, reduce input size or model complexity and then repeat the accuracy evaluation.
Use a confidence threshold validated on held-out line data, but do not interpret softmax confidence as a universal probability of correctness. Add an unknown class or reject decision when scores are close, the image is poor, the frame is stale or the object is outside the training distribution. Require a tracking ID and prediction timestamp before actuating. If confidence is low or timing is missed, route to a safe reject bin or stop according to the risk analysis.
| Failure mode | Detection signal | Safe response |
|---|---|---|
| Blur or bad exposure | Image-quality gate or low classifier margin | Reject/recapture; do not guess |
| Unknown material | Out-of-distribution review and low confidence | Unknown bin and sample for labeling |
| Inference too slow | Frame age exceeds conveyor deadline | Reject item or pause feed |
| Gate jam or missed object | Position sensor disagrees with command | Stop actuator and alert operator |
Keep motor power and safety interlocks outside the model's authority. Firmware should enforce pulse duration, cooldown, item spacing, emergency stop and sensor-confirmed gate position. Test reset during actuation, watchdog reboot, full buffer and lost tracking. Record item ID, model version, class, score, decision, gate command and observed outcome so quality teams can find systematic errors.
Store derived counts and short-lived diagnostic frames only when needed. Restrict image access, define retention and avoid capturing workers or nearby spaces unnecessarily. A dashboard should show throughput, reject rate, confidence distribution, inference latency, missed triggers, jams and model version. Aggregate data can guide maintenance and retraining without turning every frame into a cloud upload.
An ESP32-S3 sorter is a systems project: camera pipeline, representative dataset, compatible quantization, calibrated reject behavior and guarded mechanics. Measure the entire item-to-actuator deadline and make unknown outcomes safe. Edge AI is valuable when the small model's limits are visible and the rest of the line is designed around them.