Category Archives: V-Drums

V-Drum Explorer: MIDI interface

If this is the first blog post about V-Drum Explorer you’ve read, see the first post in this series for the background. In this post we’ll look at the MIDI interface I use in V-Drum Explorer to send and receive configuration information.

MIDI basics

(Apologies to anyone who really knows their stuff about MIDI – this section is the result of bits of reading and experience working with my drum kit. Please leave corrections in comments!)

MIDI is a protocol for communications between musical instruments, computers, synthesizers and the like. There are many different ways of connecting devices together, including dedicated MIDI cables, USB, TCP/IP and Bluetooth, but applications shouldn’t need to care much about this. They should only need to worry about the messages defined by the protocol.

MIDI is one-directional, in that APIs expose both input ports and output ports which are effectively independent even if they’re connected between the same pair of devices. Messages are sent on output ports and received on input ports.

Most electronic instruments (of all kinds, not just drums) support at least basic MIDI functionality. There are several simple messages with standard representations, including:

  • Channel Voice messages (such as “note on” and “note off”)
  • Channel Mode messages (such as “all sounds off”)
  • System Exclusive messages which are manufacturer-specific

The channel voice and channel mode messages apply to a specific MIDI channel. It’s possible for multiple devices to be daisy-chained together so that they can be controlled over a single port, but using different channels. Each of these messages has a fixed size.

Note: one irritating problem with the protocol is that I haven’t found a way of detecting which channel a particular device is on. This is why channel-specific operations in V-Drum Explorer (such as recording instrument sounds) need a user interface element. If there’s something I’ve missed here, I’d love to hear about it.

Only a few operations in V-Drum Explorer need to simulate or react to the channel messages. Most of the time we’re interested in the system exclusive messages which allow the configuration data to be fetched and set.

System Exclusive Messages

System Exclusive Messages – often abbreviated to SysEx – are manufacturer-specific messages for additional functionality. The size of the data isn’t fixed, but is indicated by a byte of 0xF0 indicating the start of the data, and another byte of 0xF7 indicating the end of the data. Each byte within the data has to have the top bit cleared.

The first byte of data is conventionally the manufacturer ID as assigned by the MIDI Manufacturers Association, which allows a device to ignore messages it wouldn’t properly understand. For example, all Roland SysEx messages have a first byte of 0x41.

Protocol design note: MIDI is a relatively old protocol, dating back to 1983. I’d hope that these days we’d allow more than one byte for a manufacturer ID, and use a length-prefix instead of restricting every data byte to 7 useful bits. Oh well.

SysEx messages are divided into realtime and non-realtime messages. There are also universal SysEx messages which are non-manufacturer-specific, using “fake” manufacturer IDs of 0x7E (for non-realtime) and 0x7F (for realtime).

For the V-Drum Explorer, we’re interested in four SysEx messages:

  • We send Identity Request (universal, non-realtime)
  • We receive Identity Reply (universal, non-realtime)
  • We send Data Request (Roland, non-realtime)
  • We send and receive Data Set (Roland, non-realtime)

The identity request and identity reply messages are used to discover information about the connected devices. V-Drum Explorer uses them to check whether you have any supported devices connected (currently just the TD-17 and TD-50) and present the appropriate schema for the device.

The Data Request and Data Set messages are more interesting from the perspective of V-Drum Explorer: they’re the messages used to communicate the configuration data.

You can think of the configuration data as a bank of memory in the module. (We’ll look at the layout next time.) The Data Request message indicates “Give me X bytes starting at address Y.” The Data Set message indicates “Here are some bytes starting at address X.” Interestingly, Data Set is used both to respond to Data Request and to set new data in the module. When loading data from the device, I send a Data Request and wait for a corresponding Data Set. When copying data to the device, I just send a Data Set message. The device also sends Data Set messages when the configuration data is modified on the module itself (if a particular setting is turned on).

There are a few tricky aspects to this though, all effectively related to the protocol being two independent streams rather than being request/response in the way that HTTP is, for example:

  • Can the code rely on all the data from a single Data Request being returned in a single Data Set? (Answer: no in theory, but in practice if you ask for small enough chunks, it’s fine.)
  • How long should the code wait for after sending an Identity Request until it can assume it’s seen all the Identity Reply messages it’s going to?
  • How long should the code wait for after sending a Data Request until it can assume there’s something wrong if it doesn’t get a reply?
  • How long should the code wait between sending Data Set messages when copying data to the device?
  • What should we do with messages we didn’t initiate?

V-Drum Explorer MIDI code

In the V-Drum Explorer MIDI code, I’ve modeled Identity Request / Identity Reply using an event for “I’ve seen this device”, and I just leave half a second for all devices to respond. For Data Request / Data Set things are a little trickier. For better or worse, I expose an asynchronous API along the lines of

async Task<byte[]> RequestDataAsync(int address, int size)

I keep a collection of “address/size pairs I’m waiting to see data about”, and when I receive a Data Set message I see whether I’m expecting it, I complete the corresponding task, with a buffer of unexpected messages for diagnostic purposes.

The current implementation uses the Sanford.Multimedia.Midi library, which at least assembles multiple packets into SysEx messages for me. I’m trying to move to managed-midi though, as that’s more cross-platform. (In particular, it should make a Xamarin client possible, as well as running a CLI on a Raspberry Pi.) The managed-midi library is quite low-level, so I’ll need to perform the reassembly myself – but that shouldn’t be too hard.

One thing I’m really hoping is that I can reimplement the VDrumExplorer.Midi project with almost no changes to VDrumExplorer.Wpf.

Next time…

In the next post, I’ll go into more detail about the layout of the configuration data, and the annoyance of 7-bit addressing.

V-Drum Explorer: Introduction

This is the first in what I expect to be quite a long series of blog posts, meandering over a fairly wide range of topics as they crop up. There’s nothing particularly technical in this introductory post. It’s really just a starting point.

V-Drums

In July 2019, inspired/encouraged by a friend name Alice, I bought an electronic drum kit. It was a Roland TD-17KV; I now have the Roland TD-17KVX because I’m a sucker for upgrades.

Both of these are kits using the TD-17 as the module. That’s the part of an electronic drum kit that turns electrical signals from the triggers on the pads into sound and/or MIDI messages. The module is sometimes known as the “brain” of the kit. There’s quite a lot of terminology involved in electronic drum kits, and I suspect I’ve barely scratched the surface in the few months since I’ve started.

The TD-17 series is just one option within a suite of products called Roland V-Drums. The TD-17 is currently the most recent part of this suite. It’s a mid-tier product; currently the high-end kit is the TD-50 and the entry level is the TD-1. When I was choosing between the TD-1 and thd TD-17 (with the TD-50 never being an option) I was basically drawn to the greater flexibility of the TD-17 module over the TD-1. I didn’t know how feasible it would be to configure it programmatically, but it at least felt like something I’d have fun investigating.

Configuring a TD-17

In order to understand why I was even considering coding for this, it’s worth having a look at what you can configure on the TD-17 module.

There are a few top-level things you rarely need to configure, such as which triggers you have attached to the module. Most of the configuration happens within a kit – and there are lots of kits. Now this is already slightly confusing because right at the start I talk about buying an “electronic drum kit”. The term is effectively overloaded, unfortunately, but most of the time in this blog series I’ll be using the term “kit” in the sense it’s used within the module.

The TD-17 supports 100 kits. There are 50 presets, and 50 user-defined kits. There’s no real difference between the presets and user-defined kits in terms of what you can do with them – you can think of it as “There are 100 kits available, but Roland got bored after creating 50 of them and left the other 50 in a default configuration.” A single kit is selected at a time, and this controls everything about how the drums sound.

At the top level of a kit, you can configure:

  • Its name and volume
  • The instruments (this is going to be another overloaded term)
  • Its MIDI configuration (which MIDI instruments are triggered)
  • Its ambience (simulations for being in a concert hall, or arena etc)
  • Its MultiFX (sound effects such as reverb)

There are 20 “instruments” to configure – one for the kick pedal, two for most other triggers (snare, toms, crash cymbals, hi-hat, and a general purpose aux), and three for the ride cymbal. There are two instruments for most triggers because you can configure the head and the edge/rim separately – as an extreme example, you could set the crash cymbal to sound like a gong when you hit the rim, and a snare drum when you hit the head. You probably wouldn’t want to, but you could.

For each of these instruments, you can set:

  • The left/right pan (primarily for playing with headphones, so everything sounds like it’s in the right place)
  • Equalizer settings (low/medium/high frequency response)
  • The instrument (this is the overloading…) – for example “13 inch maple tom”
  • The “sub-instrument” which is a second instrument layered under the first (e.g. the first preset kit has a beech kick with a sub-instrument of a deep shell kick) and how the two instruments are layered together
  • How much the ambience and sound effects affect this particular instrument

Each main/sub instrument then has its own extra tweaks in terms of tuning/size, muffling, and snare buzz – depending on the instrument.

Of course, you can just play the TD-17 with the preset kits, never going into any of this. But I saw possibilities… the user interface is designed very well considering the physical constraints, but I tend to think that complex configuration like this is more easily managed on a computer.

V-Drum Explorer

And so the idea of the V-Drum Explorer was born. Of course, it’s open source – it’s part of my demo code GitHub repository. There’s even documentation with screenshots so you can get the general idea, and a Windows installer downloadable from the GitHub releases page.

From the very start, I imagined a user interface with a tree view on the left and details on the right. It’s possible that by thinking of that from that early on, I’ve missed out on much better UI options.

This blog series is partially a diary of what I’ve done and the challenges I’ve faced, and partly a set of thoughts about how this affects professional development.

I expect to blog about:

  • The MIDI protocol used to communicate with the TD-17
  • The configuration data model
  • The challenges of 7-bit addressing
  • Enabling Hi-DPI for .NET Core 3 WinForms apps
  • Initial code vs schema considerations
  • The transition to a full schema
  • Displaying data more usefully: the logical tree
  • Working with real users: remote diagnostics
  • Immutabilty and cyclic references
  • Performance: WPF vs WinForms
  • Performance: field instances
  • Code signing
  • Windows installers
  • (Maybe) Xamarin and BLE MIDI
  • Moving back from schema JSON to code for ViewModels

That’s likely to take a very long time, of course. I hope it proves as interesting to read about as it has been to implement.