AI & Hardware · 2026-10-17 · EN

How to Build Your Own AI on an ESP32

Yes, you can run artificial intelligence directly on an ESP32. The realistic approach is TinyML: a tiny trained model performing one specific task locally. It is not the same as running a full-size conversational language model.

What can it do?

An ESP32 can classify sensor signals, recognize a small set of gestures or sound patterns, and support lightweight image inference with suitable hardware. It is an excellent platform for embedded decisions that work without Wi-Fi.

1. Pick the right board

Start with an ESP32-S3 development board, preferably with PSRAM. This chip is better suited to many embedded AI workloads than an original ESP32. A USB data cable and a PC are enough to begin with sample models.

2. Understand the workflow

Collect examples: record sensor measurements with labels, such as “still”, “tilt left”, and “tilt right”.
Train on your computer: use a small model built in a desktop machine-learning environment.
Quantize and convert: prepare a compact model, using the tooling required by the selected inference framework. ESP-DL uses .espdl models.
Run inference on the device: deploy the model to the ESP32 and classify incoming data without relying on a server.

Training usually happens on the computer; the microcontroller performs inference.

3. Set up ESP-IDF and ESP-DL

Install Espressif's ESP-IDF, then open a compatible example from ESP-DL. For an ESP32-S3 example project, the usual build-and-flash commands look like this:

idf.py set-target esp32s3
idf.py build
idf.py -p PORT flash monitor

Replace PORT with your serial device. These commands require a complete, compatible ESP-IDF project and prepared model. They do not train an AI by themselves.

4. Build a first project

Try a gesture-controlled LED. Gather labeled accelerometer samples for two gestures plus a resting class. Train a very small classifier on your PC, deploy its optimized form, and switch a GPIO output when the intended gesture is detected. Test with people and movements not present in the training set: real-world generalization is harder than a successful training run.

5. Know the limits

Running a large language model or solving arbitrary physics problems locally is generally beyond a standard ESP32. For a narrow task such as simple classification, however, a tiny on-device model is practical, private, and responsive.