embed_dev_93 OP
3 days ago
Been running TensorFlow Lite models on Pi 4 for about 18 months doing anomaly detection on a factory floor sensor array, and I finally swapped in a Pi 5 last week. The headline difference is real — the Cortex-A76 cores make inference noticeably snappier for int8-quantized models even without a hardware accelerator attached. My 200KB temperature-anomaly model dropped from roughly 38ms to 16ms latency per inference, which matters a lot when you're polling 12 sensors simultaneously. One thing nobody talks about enough: the Pi 5's PCIe 2.0 lane (exposed via the FPC connector) means you can mount the official Hailo-8L M.2 HAT and push into proper edge AI territory — we're talking 13 TOPS for around $70 extra. That completely changes the economics of on-device computer vision at this price point. Power draw is higher though, so if you're on battery-backed deployment, budget your regulator before you design the enclosure.

manu_k_iot
2 days ago
The PCIe point is huge and I can confirm from production. I tested the Hailo-8L HAT on Pi 5 running a YOLOv8n object detection model — compiled to Hailo's HEF format via their Dataflow Compiler — and landed at 45+ FPS at 640×640 resolution with the full board pulling under 8W total. The compilation step takes a while on first run but the runtime is clean once you have the HEF file. Here's the minimal inference loop I'm using in production right now:

from hailo_platform import HEF, VDevice, HailoSchedulingAlgorithm

params = VDevice.create_params()
params.scheduling_algorithm = HailoSchedulingAlgorithm.ROUND_ROBIN

with VDevice(params=params) as vdevice:
hef = HEF("yolov8n.hef")
infer_model = vdevice.create_infer_model(hef.get_hef_path())
infer_model.set_batch_size(1)
with infer_model.configure() as cfg_model:
bindings = cfg_model.create_bindings()
bindings.input().set_buffer(input_data)
cfg_model.run([bindings], timeout_ms=1000)
output = bindings.output().get_buffer()
Pro tip: memory-map your input buffer from a V4L2 camera capture instead of copying frames — saves roughly 3ms per frame at 1080p. Small change, real gain.

sara_liu_ml
2 days ago
Worth adding on the CPU-only side: if you stay without the Hailo HAT, switch your TFLite runtime delegate to XNNPACK with num_threads=4 and enable ARM NEON explicitly during build. I benchmarked MobileNetV3-Small at 8.2ms average with that config, versus 22ms with the stock delegate — it's a free win that most tutorials skip. Also, the Pi 5 ships with LPDDR4X at 4267 MT/s — memory bandwidth is actually a bigger bottleneck than raw compute for most vision models at this scale, so try to keep your model's activation tensors in L2 cache by tiling your inference window appropriately. For factory floor IoT with anything heavier than a shallow MLP, the Pi 5 upgrade is worth it.

rv_hardware
1 day ago
One practical gotcha that no review mentions: the Pi 5 runs genuinely hot under sustained inference load. My unit hits 80°C without active cooling when running continuous CV inference at 30 FPS — the official active cooler brings it down to a stable 55°C, but if you're enclosing this in a cabinet, plan your thermals from day one or you'll hit thermal throttling mid-shift and wonder why latency spiked. Also, the UART console moved to different pin numbering versus Pi 4, which tripped me up badly during the first headless deployment. Double-check your boot config before you seal the enclosure.