Brendan Horan Avatar

Overview

This post aims to introduce two things.

  • Firstly Earthly as a tool to help me build software.
  • Secondly a github repo of some earth files I have created. This post will have a focus on the MicroPython earth file.

Earth files are definitions of commands and build environments that the earthly tool will execute.

Whats earthly ?

Earthy is a build tool that acts on top of Docker containers.
It aims to make building software as reproducible as possible.
By defining the build process in an earth file, Earthly then ensures that the build environment and build steps are identical no matter where they are running.
This means my build environment for my MicroPython projects is always consistent and portable across my machines. It also allows me to share my build environments with others.

Template repository

I wanted a central place to store and share my earth files, so I created the earthly-recipes git repository.
So far this only contains two templates:

  • Earthfile-MicroPython
  • Earthfile-lattice-ulx3s-fpga

This post will only cover the usage of Earthfile-MicroPython.

Earthfile-MicroPython

This earth file allows you to build, flash and erase ESP32 devices that support MicroPython.
This removes the need for your dev machine to have the ESP/IDF/MicroPython build tools and frame work installed, its all done in docker container.
The earth file contains a variable called ESP_IDF_VERSION, that defines the version of the ESP IDF framework that the earth file will build MicroPython on top of.
At the time of this blog post that version is 4.2.
The earth file supports three build targets:

  • firmware
  • flash
  • erase

You must run the firmware target before any other targets
The firmware target builds a MicroPython firmware binary and any tooling to flash the devices. It pulls in main.py and the entire modules directory from the current working directory and outputs build artifacts to the artifacts directory, you do not need to create the artifacts directory as earthly will take care of that for you.
Example of running this target:

$ earthly +firmware
           buildkitd | Starting buildkit daemon as a docker container (earthly-buildkitd)...
      buildkitd-pull | Pulling buildkitd image...
      buildkitd-pull | ...Done
           buildkitd | ...Done
            python:3 | --> Load metadata linux/amd64
             ongoing | python:3 (5 seconds ago)
               +base | --> FROM python:3
             context | --> local context .
               +base | [          ] resolve docker.io/library/python:3@sha256:e6654afa815122b13242fc9ff513e2d14b00548ba6eaf4d3b03f2f261d85272d ... 0%              [██████████] resolve docker.io/library/python:3@sha256:e6654afa815122b13242fc9ff513e2d14b00548ba6eaf4d3b03f2f261d85272d ... 100%
             context | transferred 49 file(s) for context . (439 kB, 2884 file/dir stats)
             ongoing |
             +base | --> RUN apt update && DEBIAN_FRONTEND=noninteractive apt install -y cmake
<<<<<<<<< SNIP >>>>>>>>>
              output | --> exporting outputs
              output | [██████████] copying files ... 100%
================================ SUCCESS [main] ================================
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/.bin_timestamp as local artifacts/.bin_timestamp
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/CMakeCache.txt as local artifacts/CMakeCache.txt
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/CMakeFiles as local artifacts/CMakeFiles
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/Makefile as local artifacts/Makefile
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/bootloader.bin as local artifacts/bootloader.bin
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/bootloader.elf as local artifacts/bootloader.elf
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/bootloader.map as local artifacts/bootloader.map
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/cmake_install.cmake as local artifacts/cmake_install.cmake
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/compile_commands.json as local artifacts/compile_commands.json
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/config as local artifacts/config
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/config.env as local artifacts/config.env
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/esp-idf as local artifacts/esp-idf
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/kconfigs.in as local artifacts/kconfigs.in
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/kconfigs_projbuild.in as local artifacts/kconfigs_projbuild.in
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/project_description.json as local artifacts/project_description.json
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/project_elf_src.c as local artifacts/project_elf_src.c
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/partition-table.bin as local artifacts/partition-table.bin
           +firmware | Artifact github.com/brendanhoran/python-clock:master+firmware/micropython.bin as local artifacts/micropython.bin
               +base | Artifact github.com/brendanhoran/python-clock:master+base/esptool.py as local artifacts/esptool.py

The flash and erase targets take an additional command line argument to define the serial port that the ESP32 is connected to.
These two targets run in local mode, so your host machine will execute the esptool.py command, thus you must have a working Python3 interpreter installed and in your path.

The flash target flashes the following two binary's that are build artifacts from the firmware target:

  • artifacts/bootloader.bin
  • artifacts/micropython.bin

The flash target runs the build artifact artifacts/esptool.py to flash the above to binary's to the ESP32 device. You need to specify the serial port with the --build-arg of SERIAL_PORT=.
Example of running this target with the esp32 on serial port /dev/ttyUSB0:

$ earthly --build-arg SERIAL_PORT=/dev/ttyUSB0 +flash

           buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
            python:3 | --> Load metadata linux/amd64
               +base | --> FROM python:3
              +flash *local* | SERIAL_PORT=/dev/ttyUSB0
              +flash *local* | --> RUN artifacts/esptool.py -p $SERIAL_PORT -b 460800 --before default_reset --after hard_reset --chip esp32 write_flash --flash_mode dio --flash_size detect --flash_freq 40m 0x1000 artifacts/bootloader.bin 0x8000 artifacts/partition-table.bin 0x10000 artifacts/micropython.bin
<<<<<<<<< SNIP >>>>>>>>>
              +flash *local* | Wrote 1483760 bytes (980289 compressed) at 0x00010000 in 23.9 seconds (effective 497.5 kbit/s)...
              +flash *local* | Hash of data verified.
              +flash *local* | Leaving...
              +flash *local* | Hard resetting via RTS pin...
              output | --> exporting outputs
              output | [██████████] copying files ... 100%
================================ SUCCESS [main] ================================
               +base | Artifact github.com/brendanhoran/python-clock:master+base/esptool.py as local artifacts/esptool.py

The erase target will just erase the esp32 device. This target also needs the --build-arg of SERIAL_PORT.
To erase the flash run the target like this:

$ earthly --build-arg SERIAL_PORT=/dev/ttyUSB0 +erase

If you want to see how this fits into an actual MicroPython project, I use it for my Nixie clock project.