It’s been a long time…
Introduction
This project is a smart-car vision perception and motion control system based on STM32 + Raspberry Pi. It uses a layered upper/lower-computer architecture and combines technologies including computer vision (OpenCV), embedded systems, frontend/backend visualization, and AI object detection. It can handle lane recognition, closed-loop control, stranger detection, and upload statistics.
Features
The Rasp smart car, battle-tested.
Next.js Visualization Dashboard
React frontend, built with the Next.js framework.
The inspection car can be controlled with the keyboard arrow keys.
Suspicious-person detection review screen:
OpenCV
A dual-lane-line cruising algorithm implemented with the OpenCV framework.
Architecture
The whole project mainly consists of four parts, all open-sourced on GitHub:
- RaspVisionCar: Python | Upper-computer source code. Runs on the Raspberry Pi. Mainly includes the Flask web dashboard, SerialIO communication with the lower computer, and OpenCV dual-lane-line recognition.
- RaspVisionCar-STM32: C | Lower-computer source code. Runs on the STM32. Implements motor speed and steering control with a PID + feedforward controller. The serial communication is specially optimized with a circular data buffer. Also includes IMU attitude calculation.
- roboinspector: TypeScript | A frontend big-screen dashboard built with Next.js, displaying YOLO-layer results in real time.
- roboinspector-api: Python | The frontend dashboard’s gateway API. Processes the camera stream from the Raspberry Pi upper computer. Includes YOLOv8 + Bot Tracker object detection and PostgreSQL database records.
Development Process
STM32
This project started in September this year, and the overall software and hardware design was completed in November. I did not refer to any existing car projects. For the hardware, I bought off-the-shelf modules and assembled them myself in a rather improvised way. The software part was all designed by me.
I continued development in March and added YOLO object detection and the Next.js layer.
At the same time, this was also my first transition from software into hardware development. After trying it out, I became even more convinced that I should stay on the CS path rather than EE. I felt that EE is a fairly closed industry, and for opportunistic builders, there are not as many opportunities as in CS.
For example, in CS, there are definitely lightweight, low-resource opportunities for individuals, while EE is resource-heavy.
Also, EE in China leans more toward manufacturing and consumer electronics, and the compensation is questionable. The whole stack of C, analog circuits, digital circuits, FPGA, microcontrollers, and Linux may change relatively slowly, but the return is not very high either. Personally, I am not especially interested in it.
Hardware
For the chassis, I directly bought the R3X from Wheeltec. It has one caster wheel in the front and two drive wheels in the back. It came with two Wheeltec MG513X motors and a TB6612 driver board.
The car uses an STM32F103 as the lower computer, mainly for motor control, including feedforward and PID feedback controllers.
I connected the Raspberry Pi to the STM32 through DAPLink, used OpenOCD on the Raspberry Pi, and started a GDB server. This enabled wireless flashing and wireless debugging over Wi-Fi. In testing, however, the Raspberry Pi must be on the same LAN as the PC. If I use a regular home router, the connection is quite stable. But when I use my iPhone 11 personal hotspot, the connection occasionally, very occasionally, acts up.
For the IMU, I chose the ICM42688, although many people are still using the MPU6050.
Somewhere around the middle of the project, it seems I entered the HardFault_Handler interrupt because of SysTick.
I referred to keysking’s tutorial to implement circular-buffer reception for serial commands.
The web dashboard runs on the upper computer. It provides an MJPG camera video stream and can control forward/backward/left/right movement, as well as forward speed and steering speed.
The Raspberry Pi runs the OpenCV algorithm and a web interface.
-480¥