Micropython and ESP32 custom firmware
Overview
I have an upcoming project that needs wifi, cheep and serial. Normally I would just pick up an Arduino.
However recently I noticed the ESP32 SoC.
There has been a project to port Python 3 to Micro's, you can find out more about MicroPython here.
So now you have your ESP32 SoC and are thinking how to get MicroPython on to it.
Wait? Python on a microcontroller?? :D Yep!
You have two choices :
- Install the precompiled firmware binary.
- Build your own firmware binary
This post will cover both options, but more so will focus on building your own firmware binary.
Requirements
If you only want to flash prebuilt images you just need the "Kernel Options" and "esptool" sections of the requirements.
If you want to build your own firmware you will need all of the requirements section.
Instructions to just flash a precompiled binary will be at the bottom of this guide.
Kernel Options
You will need at least the following kernel options set:
- CONFIG_USB_ACM
- CONFIG_USB_SERIAL_CP210X
esptool
esptool is used to write/read images and settings from the ESP32 SoC.
It will be needed for any write/read/erase operation
Installing esptool
esptool is a pip package, the best way to install this is :
pip install esptool
For Gentoo Linux you may need to use the following syntax :
pip install esptool --user
ToolChain
You will need a copy of the xtensa-esp32 toolchain.
At the time of writing this guide the current version was 1.22.0-61
ESP IDF
This is the Espressif dev framework
The stable/supported version of the ESP IDF is set via git commit hash tags, more on that later.
Buildtools
Your operating system will need various tools to support building the firmware.
If you are using Gentoo, your set.
However if you are not using Gentoo take a look over the following page and the refrence section in this post.
- Toolchain prerequisites
- Python 3.x
- git
Micropython-esp32
This code is managed in the MicroPython-esp32 github account.
Building your own firmware
Unpack xtensa-esp32-elf
This guide makes use of 64bit binary and library files, if you have a 32bit system your will need to change to to the 32bit package.
Download the xtensa-esp32-elf tarball.
At the time of this guide the latest version was 1.22.0-61.
Unpack this into /opt.
tar -xvf xtensa-esp32-elf-linux64-1.22.0-61-gab8375a-5.2.0.tar.gz -C /opt
You can unpack this anywhere really. Next we need to ensure the bin folder is part of your systems path.
You can add a bash alias, add a file to profile.d or export when you need it.
For now lets just export it :
export PATH=$PATH:/opt/xtensa-esp32-elf/bin
I prefer to add a file into "/etc/profile.d" to make this more permanent, however refer to your disto's advice.
ESP IDF
This code is managed via github, follow the steps to fetch the code. For now do not worry about what commit is pulled.
I like to unpack this into a directory under my home folder, however you can select any location.
git clone --recursive https://github.com/espressif/esp-idf.git
Finial directory "~/repos/esp-idf"
Micropython
Download a current release of Micropython. At the time of writing this was version 1.11
You will need to checkout the latest code from the MicroPython githib account.
Follow the steps below to fetch the code.
I like to unpack this into a directory under my home folder, however you can select any location.
tar -xvf tar -xvf micropython-1.11.tar.gz
Finial directory is "~/repos/micropython-esp32/esp32"
Initial set up of MicroPython-ESP32
We need to set up two things before we can start :
- Build cross compiler support
- A makefile to suit out environment
Building cross compiler support
Run the following commands to build cross compiler support.
cd micropython-1.11
^ micropython-1.11, should be the directory that git checked out in the above step.
make -C mpy-cross
That is all that is needed for now.
Craft a makefile
We need to override some Makefile settings to suit our build environment.
The base minimum you need to set is :
- ESPIDF location
- Serial port device of the ESP32
For example I create a makefile in "~/repos/micropython/ports/esp32" that contains the following :
ESPIDF = /home/brendan/repos/esp-idf
PORT = /dev/ttyACM0
include Makefile
You may need to update "port" to reflect your device.
This can be ttyACM0 or ttyUSB0, best to check dmesg for the exac device name.
ESPIDF, is where we did the git checkout of the esp-idf. PORT, is the serial device of the ESP32 when plugged in. include, just pulls in the global Makefile.
Fix modules in MicroPython
I noticed there where some broken symlinks in the modules directory for esp32.
While this will not prevent you from compiling the firmware, it will leave out some cool functions.
For me the broken modules where "urequests.py" and "upysh.py"
$ ls -la
total 28
drwxr-xr-x 2 brendan brendan 4096 Jul 13 21:35 .
drwxr-xr-x 4 brendan brendan 4096 Jul 13 21:41 ..
-rw-r--r-- 1 brendan brendan 173 Jul 13 21:35 apa106.py
-rw-r--r-- 1 brendan brendan 173 Jul 13 21:35 _boot.py
lrwxrwxrwx 1 brendan brendan 28 Jul 13 21:35 dht.py -> ../../esp8266/modules/dht.py
lrwxrwxrwx 1 brendan brendan 32 Jul 13 21:35 ds18x20.py -> ../../esp8266/modules/ds18x20.py
-rw-r--r-- 1 brendan brendan 1034 Jul 13 21:35 flashbdev.py
-rw-r--r-- 1 brendan brendan 961 Jul 13 21:35 inisetup.py
-rw-r--r-- 1 brendan brendan 884 Jul 13 21:35 neopixel.py
lrwxrwxrwx 1 brendan brendan 32 Jul 13 21:35 onewire.py -> ../../esp8266/modules/onewire.py
lrwxrwxrwx 1 brendan brendan 19 Jul 13 21:35 upip.py -> ../../tools/upip.py
lrwxrwxrwx 1 brendan brendan 28 Jul 13 21:35 upip_utarfile.py -> ../../tools/upip_utarfile.py
lrwxrwxrwx 1 brendan brendan 39 Jul 13 21:35 upysh.py -> ../../../micropython-lib/upysh/upysh.py
lrwxrwxrwx 1 brendan brendan 47 Jul 13 21:35 urequests.py -> ../../../micropython-lib/urequests/urequests.py
Run the following to fix the broken symlinks :
cd ~/repos/micropython-1.11/ports/esp32/modules
rm upysh.py && rm urequests.py
wget https://raw.githubusercontent.com/micropython/micropython-lib/master/upysh/upysh.py
wget https://raw.githubusercontent.com/micropython/micropython-lib/master/urequests/urequests.py
Check for supported ESP IDF git tag
Next we need to check if the checkout of the ESP IDF is a supported tag version for MicroPyython-ESP32.
Change into the following directory :
cd ~/repos/micropython-1.11/ports/esp32
Then run :
make
You might see something like this (assuming you have the set up right :
Current git hash: 58df1d93bc17c74499d58e05390af9c309192a5c
Supported git hash: 5c88c5996dbde6208e3bec05abc21ff6cd822d26
This means the tag version of ESP-IDF is not a supported tag version for MicroPython-ESP32.
If you see the same tag versions it means you have a supported git tag version already.
To fix this run the following commands :
cd ~/repos/esp-idf
git reset --hard 5c88c5996dbde6208e3bec05abc21ff6cd822d26
then run :
git submodule update --init --recursive
Substitute the hash tag above with the one that is displayed in the output from the make command above.
Compile and install firmware
Now we can flash the stock firmware we just built.
Run the following commands :
cd ~/repos/micropython-1.11/ports/esp32
Note, ensure you have plugged in the ESP32 device and no other serial applications are currently using the device.
make
make erase
make deploy
Check it!
If you got no errors from the above steps you can connect to the serial port of the ESP32.
There are many ways to do this, I prefer minicom however anything that can access a serial device should work.
Connection settings are "118000,8-n-1".
Once connected you should see the python ">>>" prompt. If you do not hit reset on the board or "Ctrl-D".
The boot sequence looks like this :
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
ets Jun 8 2016 00:22:57
rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0008,len:8
load:0x3fff0010,len:3408
load:0x40078000,len:9488
load:0x40080000,len:252
entry 0x40080034
I (2277) cpu_start: Pro cpu up.
I (2277) cpu_start: Single core mode
I (2278) heap_alloc_caps: Initializing. RAM available for dynamic allocation:
I (2320) heap_alloc_caps: At 3FFAE2A0 len 00001D60 (7 KiB): DRAM
I (2381) heap_alloc_caps: At 3FFD5328 len 0000ACD8 (43 KiB): DRAM
I (2444) heap_alloc_caps: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (2508) heap_alloc_caps: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (2573) heap_alloc_caps: At 400924E4 len 0000DB1C (54 KiB): IRAM
I (2635) cpu_start: Pro cpu start user code
I (2794) cpu_start: Starting scheduler on PRO CPU.
MicroPython v1.11 on 2019-06-26; ESP32 module with ESP32
Type "help()" for more information.
>>>
Your output may look diffrent or less verbose.
.... Yes MicroPython is currently single core only for now.
I did try enable the second "app" core but there are still quite a few bugs.
Making it run your own code.
OK, so having a python shell on your micro is cool, but pretty useless really.
You are going to want to run your own python code on boot.
To do this your going to need to edit and create a few files.
boot.py
This is the file that will get called on boot.
I do not like to put much in here apart from importing other scripts/modules.
Do note if your code enters a loop you will never see the REPL prompt.
This is not really a problem once your code fully works.
Change into the following directory :
cd ~/repos/micropython-1.11/ports/esp32/modules
Create a file called "boot.py".
For now, as an example put the following code into "boot.py" :
import my_code
Next, create a file in the same directory called "my_code.py", the contents of the file should be :
print('esp32 ftw')
Save the files. As You can see this is a pretty useless example, we are just printing one line. You can import as many modules as you like inside of "boot.py".
Next change to the following directory :
cd ~/repos/micropython-1.11/ports/esp32
Next follow the normal flashing steps :
make
make erase
make deploy
That's it!, by now you should be able to compile your own firmware, include your own python modules and run them at boot time.
Flashing pre built firmware
If you just want to mess around with MicroPython via a REPL prompt you can easily flash a pre made binary firmware.
This also may be handy to just test out a new device if you are having issues with the above steps. After all electronic parts can be faulty!
You will need a working install of esptool, as listed in the prerequisites and the Kernel support, nothing else is needed.
Download a prebuild firmware.
At the time of writing this the current version was "esp32-20170714-v1.9.1-219-g3580284e".
Plug in the ESp32 bord and run the following commands :
esptool.py -p /dev/ttyACM0 erase_flash
esptool.py -p /dev/ttyACM0 --baud 460800 write_flash --flash_size=detect 0 esp32-20170711-v1.9.1-219-g3580284e.bin
Where : "/dev/ttyACM0" is the serial device of my ESP32.
Where : "esp32-20170711-v1.9.1-219-g3580284e.bin" is the downloaded binary MicroPython firmware.
After this you can connect via minicom or another application that an access the serial port. As descibed in section "Check it!" above.