Arduino Touchscreen PWM Sound Output

Somewhat inspired by Daphne Oram, the thing I really wanted to do with my touchscreen was to use it to draw a waveform.  I now have all the bits and pieces I need to do this, so in this project I’m combining the touch screen with PWM sound output and the “drawing” idea from my slider potentiometers.

Warning! I strongly recommend using old or second hand equipment for your experiments.  I am not responsible for any damage to expensive instruments!

These are the key Arduino tutorials for the main concepts used in this project:

• Arduino PWM Sound Output – Part 2
• Arduino MIDI Slider PWM Waveform Generator

If you are new to Arduino, see the Getting Started pages.

Parts list

• Arduino Uno
• ILI9488 Touchscreen display and shield
• Ready-Made MIDI Modules and MIDI source
• Optional: Arduino PWM Output Filter Circuit – Part 2
• Amplification and speaker

The Circuit

This is largely plugging together previous modules, so there is no “circuit” to describe as such.  I’ve made use of my Arduino PWM Output Filter Circuit shield and we have to note that due to the pins used by the display I can’t use PWM output on pin 9, so I’ve also used the techniques described in Arduino PWM Sound Output – Part 2 for sound output on pin 3.

I also need a MIDI input to the Arduino’s RX pin and for demonstration purposes I’ve also hooked up my cheap oscilloscope to the post-filter output.

The Code

The aim of this project is to be able to draw a waveform on the touchscreen and have it played out as the source for some PWM synthesis on the frequency required by the MIDI note being played.

Much of the code for this project is taken from the fore-mentioned previous projects.  The main sticking point for me was having to work out how to create PWM sound on pin 3 rather than pin 9, hence the follow-up to my PWM blog post with the details for how to configure and use Timer 2 (full details here).

The basic “MIDI wave generation” is taken from Arduino MIDI Slider PWM Waveform Generator, although it is actually greatly simplified, being able to lose the multiplexer code and the interpolation to create the wavetable.  Instead, I’ve taken the graphics routines from Arduino Touchscreen MIDI Controller and build the wavetable directly from the touched points on the display.

My initial trial used a direct 1:1 mapping between display coordinates and the wavetable.  This worked fine, but mean most of the display was left unused, there being just a smaller 256×256 square in the centre.

In the end I’ve allowed for the ability to scale the wavetable to the display.  But as the display is 320×480, the ideal scaling factors are 1 on the Y-axis and 1.5 on the X-axis.  To allow for a fractional scaling, I set all the display parameters using 12.4 fixed point arithmetic.  The values are all 16-bit values with 12-bits for the integer component and 4 bits for a fraction.  This means all values are 16 times larger than the actual display coordinates they map on to.  To support this, I’ve created two macros for easy conversion.

Here is the relevant code and values I’ve used to scale the display.

#define fp2d(x) ((uint16_t)((x)>>4)) // div by 16
#define d2fp(x) ((uint16_t)((x)<<4)) // mul by 16
#define WTX_MIN (48*16UL)
#define WTX_SCALE 24UL // (1.5*16UL)
#define WTX_SIZE (384*16UL)
#define WTY_MIN (32*16UL)
#define WTY_SCALE (1*16UL)
#define WTY_SIZE (256*16UL)
#define BORDER 4
void displayWaveTable () {
  gfx->fillRect(fp2d(WTX_MIN)-BORDER, fp2d(WTY_MIN)-BORDER, fp2d(WTX_SIZE)+BORDER*2, fp2d(WTY_SIZE)+BORDER*2, RED);
  gfx->fillRect(fp2d(WTX_MIN), fp2d(WTY_MIN), fp2d(WTX_SIZE), fp2d(WTY_SIZE), BLACK);
  for (int i=0; i<WTSIZE; i++) {
    updateWaveTable (i);
  }
}

void updateWaveTable (int idx) {
  int drawval = 255-wavetable[idx];
  gfx->drawFastVLine(fp2d(WTX_MIN+idx*WTX_SCALE), fp2d(WTY_MIN), fp2d(WTY_SIZE), BLACK);
  gfx->drawPixel(fp2d(WTX_MIN+idx*WTX_SCALE), fp2d(WTY_MIN+drawval*WTY_SCALE), GREEN);
}

void setWaveTable (int xc, int yc) {
  xc = d2fp(xc);
  yc = d2fp(yc);
  if ((xc < WTX_MIN) || (xc >= WTX_MIN+WTX_SIZE) || (yc < WTY_MIN) || (yc >= WTY_MIN+WTY_SIZE)) {
    return;
  }

  wavetable[(xc-WTX_MIN)/WTX_SCALE] = 255-((yc-WTY_MIN)/WTY_SCALE);
  updateWaveTable((xc-WTX_MIN)/WTX_SCALE);
}

The displayWaveTable() function is called on start-up and clears and redisplays the entire table according to its current settings.

The updateWaveTable() function just clears and then re-draws the pixels corresponding to one specific entry in the wavetable, so to draw the whole thing this needs to be called 256 times.  Both of these need to convert from the 12.4 format back to actual display coordinates.

The setWaveTable function is designed to be called when a touch event is detected.  It takes display coordinates corresponding to the touch point, checks them, and converts them into values to update the wavetable.  The X-coordinate is scaled and turned into a wavetable index,  and the Y-coordinate is scaled and turned into the value within the wavetable.  Then updateWaveTable is called to re-draw the display for that specific point.

As with the previous touch projects, there are two functions to initialise and scan the display – gfxSetup() and gfxLoop().  The latter is responsible for calling the setWaveTable () function when a touch is detected.

The PWM and MIDI handling is exactly the same as for the Arduino MIDI Slider PWM Waveform Generator – but I’ve had to use the code from Arduino PWM Sound Output – Part 2 to use Timer 2 and pin 3 as the PWM output.

Find it on GitHub here.

Closing Thoughts

I’ve been wanting to try this for some time.  You can hear the results in the video.  There is one quirk, which I actually quite like.  When drawing waveforms you get the odd missed pixel, especially if you draw quickly.  This means that an otherwise fairly smooth waveform has the odd out-of-step portion dotted around.  If you draw really quickly you can almost get two waveforms superimposed on the display!

The upshot of this is that almost any hand-drawn waveform has a real grittiness to it.  There are lots of additional harmonics due to edges and odd entries in the wavetable and with the PWM signal this is really audible in the output. This makes the type of sounds you can generate really quite complex.

This would be a great thing to use as an oscillator source for some fancy further processing with filters and so on.

This was mostly inspired by Daphne Oram’s Oramics.  She had to draw sounds using paint on film back in the 1960s.  Now it is possible to do that digitally with a hand-full of off-the-shelf components!

It would be really interesting to use this to control some of the other parameters of a synthesizer somehow.  Drawing envelopes is the obvious one. I wonder how many I could get on the screen at once.

Kevin