Pico Dual-core MIDI Visualisations

I started this project as I had just bought two new toys from The Pi-Hut:

• Waveshare RGB Full-colour 16×10 LED Matrix Panel
• Waveshare 4-Digit 8-Segment Display Module

My initial thinking was to re-run the Pico Unicorn MIDI Visualiser and the Arduino MIDI 7 Segment Controller with these new devices and a Pico.  But whilst playing around, I thought it might be interesting to try to use them together.  This describes how I got on and why I ended up with a dual-threaded application using both processing cores of the Raspberry Pi Pico.

Warning! I strongly recommend using old or second hand equipment for your experiments.  I am not responsible for any damage to expensive instruments!

These are the key tutorials for the main concepts used in this project:

• Raspberry Pi Pico Python SDK – Chapter 3.5 Multicore
• Circuit Digest: Dual Core Programming on the Raspberry Pi Pico using Micropython
• Electrosoftcloud: Multithreaded script on Raspberry Pi Pico and Micropython
• Instructables: Dual Cores and Interrupts on Pi Pico

If you are new to microcontrollers, see the Getting Started pages.

Parts list

• Raspberry Pi Pico
• Waveshare RGB Full-colour 16×10 LED Matrix Panel (e.g. here from The Pi Hut)
• Waveshare 4-Digit 8-Segment Display Module (e.g. here from The Pi Hut)
• 3.3V suitable MIDI IN interface (e.g. one of the DIY MIDI Interfaces)
• IO expander (e.g. this quad expander from Pimoroni) or jumper wires

The Circuit

I’ve used my Pimoroni “quad expander” to allow me to use both Waveshare modules and my Raspberry Pi Pico MIDI “pack” Interface at the same time.  The devices are using the following GPIO pins:

• RGB Matrix: VBUS, GND, GP6
• 8SEG LED: VSYS, GND, GP9, GP10, GP11
• MIDI module: 3V3_OUT, GND, GP0, GP1

The whole thing is powered from the Pico’s USB port.

The Code

Waveshare provide a sample Micropython module for each of their modules.  You can get them from the “Resources” tab on their wiki pages:

• Waveshare RGB Full-colour 16×10 LED Matrix Panel
• Waveshare 4-Digit 8-Segment Display Module

In order to check that everything is working, you can just run each of these directly on your Pico and it will perform a demo of the module.  But each also provides a class interface that allows you to use them in your own code.  There was just one minor issue with this however.  The two filenames as provided (expanded from the 7-Zip file containing all the example code) both have “-” (dash) in the filenames, which don’t get on well with the python “import” system, so I had to rename them without dashes.

I also used my SimpleMIDIDecoder from before too.

So in additional to the code I’m about to describe, I have the following three python files on my Micropython system:

• SimpleMIDIDecoder.py
• PicoRGBLED.py
• Pico8SEGLED.py

Required functionality

I want the code to do the following:

• Decode MIDI received over the serial port and act on NoteOn and NoteOff messages.
• Use the LED matrix as a 10×12 display showing which notes are being played.  There is one row per octave and twelve notes in each row.  This covers MIDI notes 0 to 119 (C “-1” up to B7 depending on how you start counting).
• Use the 8-segment display (7-segments plus dot) to show the hex value of the first two bytes of the last MIDI message received – i.e. the MIDI command, channel and first byte of data.

The general structure is as follows.

Initialise all the hardware
Show some test patterns to ensure everything is working
Set up the MIDI callbacks:
  On NoteOn:
    Update the value for the 8-segment display
    Update which LEDs in the matrix are on

  On NoteOff:
    Update the value for the 8-segment display
    Update which LEDS in the matrix are off

Main loop:
  IF serial port data:
    Send off to the MIDI decoder
  Update the 8-segment display
  Update the LED matrix

There are a couple of constraints that mean it isn’t quite as straightforward as this:

• The 8 segment display will only show one digit at a time, so to get a four-digit display needs the microcontroller to continually output all four digits in sequence to create a “persistence of vision” illusion of having all four digits displayed.
• The LED matrix uses the WS2812 programmable LED protocol (the same as Adafruit’s “NeoPixels”) which requires a string of numbers to be sent to the matrix according to a relatively strict real-time protocol.
• All the above has to happen whilst ensuring that the serial port is read (and the associated MIDI messages decoded) without losing messages.

Attempt 1: JFDI

The first attempt pretty much just implemented the above, but it just wasn’t regular enough to allow everything to happen.  The 8 segment display appeared faded or flickering and loses the persistence of vision effect when not updated regularly enough.  So I need a way to service the 8 segment LED, ideally with the same period between updates, without having to worry about where the different calls come in the rest of the code.

Attempt 2: Timer Interrupts

The usual way to deal with something that has to be updated regularly is to use a Timer interrupt.  This is what I would usually do on an Arduino.  The Raspberry Pi Pico Micropython environment allows this as follows:

def timerCallback(timer):
  segScan()

ledTimer = machine.Timer()
ledTimer.init(mode=machine.Timer.PERIODIC, period=20, callback=timerCallback)

This will call the timerCallback() function every 20mS.  Of course this means that when this timer routine is running the processor on the Pico can’t do anything else, but it also means that no matter what else the processor is actually doing, every 20 mS it will get interrupted and call my segScan() function.

This sort of works.  The problem is that this really does have to be something like around every 20mS to get a good, non-flickering four digit 8 segment display.  But at that speed there is a very good chance of interrupting the LED matrix while it is attempting to send the numbers associated with the 160 LEDs using that strict-timing WS2812 protocol.  This results in “glitches in the matrix” (as it were) where spurious LEDs might get lit up instead.

Winding down the periodicity of the 8 segment update reduces the chances of a glitch (but doesn’t remove it), but then the four digits fade and flicker.  Another approach is to disable interrupts around the LED matrix update to temporarily stop the timer from triggering.  This sort of works, but eventually everything just seemed to completely lock up.

Ultimately it just didn’t seem possible (in Micropython – I’m sure it would be fine in raw C) to do all this on the same processor.

Recall though, I’m not really a python person, (as anyone who has followed along on my blog will know), so there may be other tricks I’m missing.  Do let me know the comments if you have a better way!

Attempt 3: Multi-threading

The Raspberry Pi Pico uses the RP2040 which is a dual-core 32-bit ARM processor.  So actually, my sub £4 microcontroller sort of has two processors “built in”.  So I decided that it would be great if I could get the second core to worry about the 8 segment display and leave the first core handling the MIDI and LED matrix.

The problem is that most of the example code already “out there” for multi-core operation on the Raspberry Pi Pico is written for the C environment.  As I say, I’m more of a C person than python, but I’ve not really looking into getting the Pico’s C environment up and running yet. I’ve dabbled with the Arduino IDE Pico support, but consider that still very “early days” in usability.  Anyway, the Pico is one of my excuses for learning python…

The examples that do exist are pretty simplistic though.  In fact the official Pico Micropython “multicore” example is really only saying “run this simple code once on the second core, then stop” whilst ignoring the original core entirely so isn’t actually a multi-core application at all.  This is the entirety of the information in the current official Raspberry Pi Pico Micropython SDK documentation:

That is the entire “chapter 3.5” on the topic.

The tutorials I’ve linked to at the start provide some more complex examples, but I just could not get it to work at all.  No matter what I did the whole thing would eventually completely lock up.

I happened to see in one of the examples mention of the “garbage collector” so I added a call to gc.collect() in the second thread and amazingly it all suddenly started working fine! I don’t really know why this fixes things – if it means there is a bug in my code, the Waveshare “driver” code, or something weird with the Micropython environment.  If you know, do let me know in the comments!

So here is the code to start and run the 8 segment code on the second core.

counter = 0
def ledThread():
  global counter
  while True:
    segScan()
    counter+=1
    if (counter > 100):
      gc.collect()

_thread.start_new_thread(ledThread, ())

The result is that now both displays update very nicely and I don’t think I’m losing any MIDI messages (although I’ve not attempted to verify it – this is just a bit of fun after all).

Note I’ve not worried about any thread-safety issues with the shared “LEDValue” that is passed between the two threads.  For what I’m doing it’s not really necessary – one writes, one reads. If it gets read part-way through an update, I don’t really mind.  But for more complex data exchanges some additional protection will be needed, but that is the subject of a whole new post.

Despite this success, however, one of the down-sides of multi-core programming is that it breaks the standard Thonny “update – run on Pico – break out of the code – update again” cycle for development.  Here are some of the issues you’ll face attempting to develop multi-core code using Thonny/Micropython on the Pico (or at least, that I found):

• There is a good chance you lose the serial connection once your multi-core code is running. This usually needs a reset to get back.
• Breaking out to REPL only seems to happen on one core – the other one is still running, so if you update and run again you will get a “OSError: core1 in use” when you attempt to start a new thread again, and will need to reset or power off/on the Pico.
• If you try to print from both cores, the messages will get mixed up if they happen to try to print at the same time.
• If there is a syntax problem, you might end up with the following cryptic message rather than nice and useful run-time diagnostic messages.

STDERR:
Traceback (most recent call last):
File "<stdin>", line 1
SyntaxError: invalid syntax

So multi-core (multi-thread) programming in Micropython is possible, but it isn’t particularly “nice” at the moment.  But in my case it did seem the only way to be able to do what I wanted.

Find it on GitHub here.  Recall this requires the three additional files mention at the start too.

Closing Thoughts

The video shows a PC playing the MIDI file from my Lo-Fi Orchestra – John Miles Music to the PC’s internal synth (everything plays on the default piano sound as I always remove program and control change messages from anything for the Lo-Fi Orchestra) and to the Pico.  There is a slight delay in the PC’s internal synth responding and the Pico updating, but generally it appears to be working. I can’t guarantee at this stage that no MIDI messages are being lost, but it seems convincing enough for a “gimmick”.  I wouldn’t use it for diagnosing MIDI systems however 🙂

This is just a first go with the two Waveshare devices too – in the end, in this post, there were somewhat overshadowed by the technical details of the approach! I have other plans for both of them too.

Having now taken the multi-core approach this once, I can certainly think about other cases where this could be worth thinking about some more.  But at the back of my mind, I’m still wondering quite how a multi-core Micropython approach really compares to Timer-interrupt based Arduino C code.

The real power I guess will come either from the Pico’s C environment or if multi-core programming on Micropython gets more “beginner friendly”.

All of the above comes with the proviso that I’m am not really a python person, so if you can tell me better ways to do any of it (or why I need garbage collection in the second thread) do leave me a comment!

Kevin