Raspberry Pi Pico Synth_Dexed? – Part 3

The story so far…

• In Part 1 I work out how to build Synth_Dexed using the Pico SDK and get some sounds coming out.
• In Part 2 I take a detailed look at the performance with a diversion into the workings of the pico_audio library and floating point maths on the pico, on the way.

This post describes how I’ve set things up for some further development and the decisions I’ve made to get to the point where it can receive MIDI and actually be somewhat playable within the limitations of 10 note polyphony, a 24000 sample rate, and a single voice only on MIDI channel 1!

Update: By overclocking the Pico to 250MHz I can do 16 note polyphony at 24000 or 8 note polyphony at 48000!

Warning! I strongly recommend using old or second hand equipment for your experiments.  I am not responsible for any damage to expensive instruments!

If you are new to microcontrollers, see the Getting Started pages.

Optimised Dexed->getSamples

I left things in part 2 noting that Dexed itself is essentially a fully integer-implemented synth engine, so why did I need the floating point calculations. I concluded it is all due to the filter that has been added, which is based on the LP filter code from https://obxd.wordpress.com/ which was added in Dexed, but wasn’t in the original “music synthesizer for Android“.

So I’ve decided not to bother with it. If I feel like it is really missing out, then I have stumbled across the following which looks promising: https://beammyselfintothefuture.wordpress.com/2015/02/16/simple-c-code-for-resonant-lpf-hpf-filters-and-high-low-shelving-eqs/

So, here is my integer-only version of Dexed->getSamples.

void Dexed::getSamples(int16_t* buffer, uint16_t n_samples)
{
  if (refreshVoice)
  {
    for (uint8_t i = 0; i < max_notes; i++)
    {
      if ( voices[i].live )
        voices[i].dx7_note->update(data, voices[i].midi_note, voices[i].velocity, voices[i].porta, &controllers);
    }
    lfo.reset(data + 137);
    refreshVoice = false;
  }

  for (uint16_t i = 0; i < n_samples; ++i)
  {
    buffer[i] = 0;
  }

  for (uint16_t i = 0; i < n_samples; i += _N_)
  {
    AlignedBuf<int32_t, _N_> audiobuf;

    for (uint8_t j = 0; j < _N_; ++j)
    {
      audiobuf.get()[j] = 0;
    }

    int32_t lfovalue = lfo.getsample();
    int32_t lfodelay = lfo.getdelay();

    for (uint8_t note = 0; note < max_notes; note++)
    {
      if (voices[note].live)
      {
        voices[note].dx7_note->compute(audiobuf.get(), lfovalue, lfodelay, &controllers);

        for (uint8_t j = 0; j < _N_; ++j)
        {
          int16_t tmp = audiobuf.get()[j] >> 16;
          buffer[i + j] += tmp;
          audiobuf.get()[j] = 0;
        }
      }
    }
  }
}

With this in place, I appear to be able to comfortably cope with 8-note polyphony. At least with my test chords.

Debug Output

Now before I go too far, I want a simple way to get some output out of the device. The Pico Getting Started documentation gives an example of how to get some standard output (stdio) working. There are two options for this output (see chapter 4 “Saying “Hello World” in C”):

• Using the built-in USB serial port.
• Outputting to the UART serial port.

To use USB requires building in TinyUSB, but I’m planning on using that later. It also adds quite a lot of overhead apparently, so the default is to output to the serial port via GP0 (TX) and GP1 (RX). All that is required is to find a way to connect this up to a computer or terminal device.

There are several options: some kind of 3V3 supporting USB<->Serial converter – there are several, based on the CH240 of FTDI devices for example, although not many of the cheap ones are 3V3 compatible (don’t use a 5V board it could damage the Pico!); or using a native Raspberry Pi development environment, then simply using GPIO directly to connect the Pico to the Pi’s UART.

It is also possible to use the picoprobe firmware running on another Pico I believe, but I haven’t tried that. It wasn’t totally clear to me if that supports the USB to serial link, although it is strongly implied. The official Raspberry Pi Debug Probe definitely does however, but I haven’t got one of those at the moment.

I initially opted to use another Pico as a serial to USB gateway by running Circuitpython and the following script on boot by saving it as code.py:

import board
import busio
import digitalio

uart = busio.UART(tx=board.GP0, rx=board.GP1, baudrate=115200, timeout=0.1)

while True:
    readbytes = uart.read()
    if readbytes != None:
        print (''.join([chr(b) for b in readbytes]))

Now this just needs connected to the Pico running PicoDexed as follows:

PicoDexed        Debug Pico
   GP0    <---->    GP1
   GND    <---->    GND

As this is running Circuitpython it means I also get the CIRCUITPY virtual drive appear and mounted too which isn’t ideal but not really a big issue.

Then I had a rummage in my Pico drawer looking for a neater solution and found a Waveshare RP2040-One that I’d forgotten I had! This is perfect as it has a USB plug at one end (via a shaped PCB) and GPIO at the other, including pins connected to UART 0.

I dropped Micropython onto the board this time, with the following script.

import time
from machine import UART, Pin

# Use one of the GPIO as a GND pin for the serial
gndpin = Pin(11, Pin.OUT)
gndpin.value(0)

print ("Initialising UART 0 on pins gnd=11, tx=12, rx=13...")
uart = UART(0, baudrate=115200, tx=Pin(12), rx=Pin(13))
print ("Ready")

while True:
    # Read raw data version
    rxdata = bytes()
    while uart.any() > 0:
        rxdata += uart.read(1)
        time.sleep_ms(10)

    if rxdata:
        print(rxdata.decode('utf-8'))

To keep the connections simple, I used GPIO 11 as an additional GND pin as there is only one on the board and it isn’t so convenient.

PicoDexed        RP2040-One
   GP0    <---->    GP13
   GND    <---->    GP11

To ensure the code can output text just needs something like the following:

#include <stdio.h>

void main () {
  stdio_init_all();
  printf("PicoDexed...");
}

Then with both devices connected to my virtual Ubuntu Linux machine, I can run minicom (once installed – it isn’t installed by default):

$ sudo minicom -b 115200 -D /dev/ttyACM0

Here is the output.

Welcome to minicom 2.8

OPTIONS: I18n 
Port /dev/ttyACM0, 13:27:28
Press CTRL-A Z for help on special keys

PicoDexed...

Connecting PIO I2S audio
Copying mono to mono at 24000 Hz

Note, to exit minicom use CTRL-A then X.

Alternatively I could use PuTTY on Windows on the COM port associated with the “debugging” Pico.

At some point I’ll probably need to set up proper SWD debugging, but this should do for the time being.

I might also need to switch UARTs if I want to use UART 0 for MIDI, but apparently there are some defines that can be changed in the CMakeLists.txt file:

target_compile_definitions(picodexed PRIVATE
  PICO_DEFAULT_UART=0
  PICO_DEFAULT_UART_TX_PIN=0
  PICO_DEFAULT_UART_RX_PIN=1
)

PicoDexed design

It is time to start thinking seriously if I can turn this into something interesting or not, so borrowing from some of the design principles encapsulated in MiniDexed, I’ve now got a project that looks as follows:

• main.cpp -> Basic IO, initialisation and main update loop.
• picodexed.cpp -> The core synthesizer wrapper with the following key interface:
  • CPicoDexed::Init -> perform all the required synthesizer initialisation.
  • CPicoDexed::Process -> perform a single “tick” of the synthesizer functions, including updating the sample buffers from Dexed.
• mididevice.cpp, serialmidi.cpp, usbmidi.cpp -> placeholder classes that will eventually support MIDI message passing and a serial and USB MIDI interface. This borrows heavily from the way it is done in MiniDexed. These classes will support the following interface:
  • CSerialMIDI::Init -> Initialise the hardware (same for USB).
  • CSerialMIDI::Process -> poll the hardware (same for USB).
  • CMIDIDevice::MIDIMessageHandler -> will be called by the lower-level devices when a MIDI message is ready to be processed. Once parsed, it will trigger calls into the PicoDexed main synthesizer to update its state.
• soundevice.cpp -> Encapsulating the interface to the pico_audio library to use I2S audio, with the following key interface:
  • CSoundDevice::Init -> Set the sample rate and I2S interface pins.
  • CSoundDevice::Update -> Fill the sample buffer using the provided callback, which will be a call to the Dexed->getSamples code above.
• config.h -> contains some system-wide configuration items, such as sample rate and polyphony.

PicoDexed will include the functions required to control the synthesizer. Examples include keydown and keyup functions for when MIDI NoteOn and NoteOff messages, and so on. It also includes a means to set the MIDI channel and to load a voice.

I don’t know yet if the MIDI handling will be interrupt driven or polled. I need to read up on how the Pico SDK handles USB and serial data, but I suspect a polled interface should be fine for my purposes as long as it doesn’t hold up the sample calculations, buffer filling, and sample playing.

With my first pass of this code, there is no external interface – it is still playing a test chord only. But at least most of the structure is now in place to hook it up to MIDI.

The “to do” list so far:

• Ideally find a way to better manage the Synth_Dexed changes. I should submit a PR to Holger, the creator of Synth_Dexed and discuss some conditional compilation steps.
• Hook up USB MIDI so that the Pico can act as a MIDI device and play the synth that way.
• Hook up serial MIDI too.
• Implement volume. Without the filter there is currently no volume changing.
• Implement some basic voice and bank loading.
• Connect up some more core MIDI functionality for program change, BANKSEL, channel volume, master volume, and so on.
• Think about how best to utilise the second core – in theory it should be possible to expand it to 16-note polyphony by using both cores. Or an alternative might be two instances of Synth_Dexed running, so making a second tone generator.

MIDI/Serial Handling

Rather than jump into USB, I’ve opted to get serial MIDI working first. The serial port handling I’ve implemented in serialmidi.cpp borrows heavily from the “advanced” UART example: https://github.com/raspberrypi/pico-examples/tree/master/uart/uart_advanced

It is interrupt driven and shares a simple circular buffer with the main Read function based on the implementation described here: https://embedjournal.com/implementing-circular-buffer-embedded-c/.

The basic design of the serial MIDI interface is as follows:

Interrupt Handler:
  Empty the serial hardware of data writing it to the circular buffer

Init function:
  Initialise the UART as per the uart_advanced example
  Install the interrupt handler and enable interrupts

Process function:
  Call the MIDI device data parser to piece together any MIDI messages
  Call the MIDI device msg handler to handle any complete MIDI messages

Read function:
  Read the next byte out of the circular buffer

There is a common MIDI device that the serial MIDI device inherits from (and that I plan to also use with USB MIDI support when I get that far). This has the following essential functionality:

MIDIParser:
  Read a byte from the transport interface (e.g. the serial MIDI Read)
  IF starting a new message THEN
    Initialise MIDI msg structures
    IF a single byte message THEN
      Fill in MIDI msg structures for single-byte message
      return TRUE
    IF there is a valid Running Status byte stored THEN
      IF this now completes a valid two-byte msg THEN
        Fill in MIDI msg structures for a two-byte message
        return TRUE
  Otherwise process as a two or three byte message
  IF message now complete THEN
    Fill in MIDI msg structures
    return TRUE
  return FALSE

MIDI Message Parser:
  IF MIDI msg already processed THEN return
  IF on correct channel or OMNI THEN
    Based on received MIDI command:
      Extract parameters
      Call appropriate picoDexed function
  Mark MIDI msg as processed.

I had a fun bug where in the serial handling, I was writing to a byte one-out in the circular buffer which meant that the MIDI handling largely worked, but only when using a controller with ActiveSensing – basically the reception of the extra byte “pushed through” the previous message. But it was a bit unresponsive, and occasionally a note of a chord would sound after the others.

I spent the better part of a day instrumenting the code, attempting to work out where the delays might be coming from. Eventually I got so fed up with the active sensing reception clouding my analysis (and triggering my scope when I didn’t want it to) that I filtered it out in the serial interrupt routine – so as early as I could.

This the made the delay a whole pile worse! That was the point I realised it was continually essentially one message behind. As a consequence I had another look at the buffer handling and that was when I realised the mistake.

Multicore support

My initial thought on the above problem was that it was a performance issue – that the MIDI handling wasn’t responsive enough. So I pushed ahead and moved all the synthesis code over to the second core. This is something I wanted to do anyway as I always had the plan of splitting the functionality across the two cores.

To enable multicore support requires including pico_multicore in the list of libaries in the CMakeLists.txt file and then it should largely be a case of doing the following:

#include "pico/multicore.h"

void core1_entry (void)
{
  // stuff to do to initialise core 1

  while (1)
  {
    // Stuff to do repeatedly on core 1
  }
}

// Rest of "normal" (core 0) initialisation code
multicore_launch_core1 (core1_entry);

The question is where to enable this. Eventually I settled on implementing this in picoDexed itself to split out the ProcessSound function over to the second core. This required the following:

• PicoDexed::Init – initialise multi-core support and start the second core running.
• PicoDexed::Process – no longer calls ProcessSound.
• PicoDexed::core1_entry – now calls ProcessSound in an infinite loop.

In order to ensure that I don’t get Dexed into an inconsistent state, I’ve protected the calls into Dexed from the Dexed_Adaptor with spinlocks (mirroring what was happing in MiniDexed) as shown in the following extract:

class CDexedAdapter : public Dexed
{
public:
    CDexedAdapter (uint8_t maxnotes, int rate)
    : Dexed (maxnotes, rate)
    {
        spinlock_num = spin_lock_claim_unused(true);
        spinlock = spin_lock_init(spinlock_num);
    }

    void getSamples (int16_t* buffer, uint16_t n_samples)
    {
        spin_lock_unsafe_blocking(spinlock);
        Dexed::getSamples (buffer, n_samples);
        spin_unlock_unsafe(spinlock);
    }

private:
    int spinlock_num;
    spin_lock_t *spinlock;
}

Spinlocks are described in chapter 4.1.19 of the RPi C/C++ SDK and are part of the hardware_sync library.

In order to ensure that the spin_locks are not held too long, and to allow things like keyup/down events to be registered in a timely manner and not hold up core 0 whilst core 1 is calculating samples, I’ve now reduced the sample buffer to 64 bytes.

As core 1 is essentially free-running calculating samples now, I figured it wouldn’t make much difference how many samples are calculated in each “chunk” but going to 64 from 256 gives four times the number of break points in the cycle where other events can be processed between the spin_locks.

Once consequence of running multi-core seems to be that I can now push the polyphony up to 10 simultaneous notes without any artefacts.

If I can find a way to keep some of the sound generation on core 0 too, I might be able to increase that even further, although getting 6 additional sound engines running to get up to the magic 16 note polyphony might be stretching things still. The trick will be finding a way to trigger and mix samples from the sound generators in the two cores, as all of that current happens within Dexed itself.

Overclocking the Pico

There have been a number of experiments already in seeing how far a Pico can be pushed. There is a standard API call to set the system clock: set_sys_clock_khz(), although not all values can be accurately configured. But general wisdom seems to be that running the Pico at 250MHz isn’t a big deal…

Of course, at this point it is running outside of the “normal” spec, so the long term effects may well reduce the life of the Pico…

But by doing this, the Pico is now running twice as fast and so can now easily cope with 16 note polyphony at a sample rate of 24000, or up the sample rate to 48000 and stick with 8 note polyphony.

It might even raise the possibility of running two tone generators, one on each core! It really does open up a wide range of possibilities!

Closing Thoughts

I’m really pleased with the progress so far. I was starting to think there wouldn’t be a usable combination possible, but 10-note polyphony at a sample rate of 24000 isn’t too bad for a 133MHz CPU with no FPU.

I think my basic design goal would be for something usable with a MIDI controller. I’m not looking to implement a UI like there is with MiniDexed as part of this build. But I do need a bit more MIDI functionality first and I would like to find a way to squeeze some sample calculations out of core 0 when it isn’t handling MIDI.

I also want to get USB MIDI up and running too. I’m not sure if I want to push for both device and host USB support though. I’ll see how complicated it all is!

In the video, I’ve used my Raspberry Pi Pico MIDI Proto Expander. It just needs the addition of the Pimoroni audio board and it is ready to go!

Of course the key question is: would I recommend this to anyone? Answer: no! No way – get yourself a Raspberry Pi Zero V2 and run a full-blown set of 8 DX7s using MiniDexed 🙂

Still for me, this is a bit of fun a really good excuse to do something that’s been on my “to do” list for ages – start getting to grips with the Raspberry Pi Pico C/C++ SDK and the RP2040.

Kevin