V-Drum Explorer: Memory and 7-bit addressing

So, just to recap, I’m writing an explorer for my Roland V-Drums set (currently a TD-17, but with a TD-27 upgrade on the way, excitingly). This involves copying configuration data from the module (the main bit of electronics involved) into the application, displaying with it, editing it, then copying it back again so that I can use the changes I’ve made. I use MIDI System Exclusive (SysEx) messages to request data from the module.

(Since I last wrote about all of this, MIDI 2.0 has been ratified, which has more sensible 2-way communication. However, I don’t think the V-Drums line supports MIDI 2.0 yet, and even if it does I expect we’ll need to wait a while for drivers, then managed APIs exposing them.)

In fact, most of the time that I’m working on the V-Drum Explorer I don’t have it connected to the drum kit: it’s much easier (and quicker) to load the data from a file. Once it’s in memory, it really doesn’t matter where the data came from. I’ll go into the file formats I’m using in another post.

This post is about how that configuration data is organized, and particularly about the 7-bit addressing it uses.

Download the docs!

If you’d like to know about all of this in more detail, it’s well worth downloading some of the reference documentation Roland very helpfully provides. The TD-17 is the simplest of the modules we’re talking about, so I’d suggest downloading the TD-17 MIDI implementation so you can go at least one level deeper than this blog post, if you’re interested. If you think you’re likely to want to do that, I’d suggest doing so before reading any further. The important bit starts on page 5 – the “Parameter Address Map” which is the bulk of the document.

Configuration as memory

The configuration data in the module isn’t stored in any kind of file system, or with separate SysEx messages for different kinds of data. Instead, it’s modeled as if the module contains a big chunk of memory, and different areas of that memory have different effects on the module. I don’t know what the implementation is like within the module itself, of course; this is just the interface presented over MIDI.

As the simplest possible example, address 0 in the memory (on all three of the modules I have documentation for) represents “the currently selected kit”. It accepts values between 0 and 99 inclusive, to represent kits 1 to 100 inclusive. (As a reminder, a kit is a configuration of all the pads, allowing you to switch the whole module between (say) a rock feel or something more electro-funky.) So as an example, if my module is currently on kit 11 (“Studio / Live room”), and I asked the module to give me the content of address 0, it would return 10. If instead I set the value of address 0 to 30, the module would display kit 31 (“More cowbell / pop”) and all the sounds would change accordingly.

The documentation describes a number of containers – that’s my own term for it, but it seems reasonable. Each container is named, and has its own description in terms of its content at different offsets. For example, address 0 belongs in the [Current] container, which is documented very simply:

[Current]
+-----------------------------------------------------+
| Offset Address | Description                        |
|----------------+------------------------------------+
|          00 00 | 0aaa aaaa | Drum Kit Number (0-99) |
|                |                             1-100  |
+----------------+------------------------------------+
| 00 00 00 01    | Total Size                         |
+----------------+------------------------------------+

The 0aaa aaaa shows that 7 bits of the value are used. Due to MIDI’s inherent 7-bit nature, each address can only store 7 bits. Whenever a larger number is required, it’s stored across multiple addresses, typically using only the bottom four bits of each 7 bit value.

The content of each container is broadly broken into three types of data – again, all the terminology is mine:

  • Primitive fields: strings, numbers etc
  • Fields which are other containers (often repeated, e.g. Kit 1 to Kit 100)
  • Overlaid containers (fields directly in this container, interpreted according to a different container)

I’ll talk about overlaid containers at length another time, as they’re tricky.

So basically you end up with a natural tree structure. So far, so good… except for 7-bit addressing.

7-bit addressing

I entirely understand why the values in memory are 7-bit values. That’s inherent in MIDI. But Roland also chose to use a 7-bit address space, which makes the code much more complex than it needs to be.

All addresses and offsets are documented using hex as if it were entirely normal – but the top bit of every byte of an address is always clear. So address 00 7F is followed directly by address 01 00 – even if they’re within the same container. Now this does mean that the values in the MIDI request messages are exactly the documented addresses: the top bit of each byte in the request message has to be clear, and that drops out naturally from this. But it makes everything else hard to reason about. I’ve been bitten multiple times by code which looks like it should be okay, but it’s either skipping some data when it shouldn’t, or it’s not skipping addresses when it should. By contrast, it would have been really simple (IMO) to document everything with a contiguous address space, and just specify that when requesting data, the address is specified in seven-bit chunks (so bits 27-21 in the first request byte, then 20-14, then 13-7, then 6-0).

I’ve tried to isolate this into a ModuleAddress struct, but the details have still leaked out into a few more places. Over the past few days I’ve tried to be more rigorous about this with a DisplayValue (7-bit) and a separate LogicalValue (contiguous), but it’s still leaking more than I want it to. I don’t think I can fix it without a more serious rewrite – which I probably want to attempt reasonably soon anyway.

You might wonder why I don’t just model everything using the logical contiguous address space, removing all the gaps entirely. The problem is that all the schema information is basically copied from the documentation – and that refers to the 7-bit addressing scheme. I really want the schema to match the documentation, so I can’t move away from that entirely. Another thing that makes it tricky is that a lot of the time I deal in offsets rather than addresses. For example, the “Kit Unit Common 1” part of a “Kit” container is always at offset 00 20 00 relative to the start of the container. That’s not too bad on its own, but I also need to express the “gap between offsets” which is a sort of offset in its own right (maybe). For example, “Kit Unit Common 2” is at offset 00 21 00 within a kit, so in the schema when I describe the “Kit Unit Common” repeated field, I describe it as having an initial offset of 00 20 00, with a gap of 00 01 00. That sounds fine – until you’ve got a repeated field which is large enough to have a gap in the middle, so you need to model that by making the offset have a gap as well. (I’m reminded of calendar arithmetic, which has similar weirdnesses.)

The lesson I’ve learned from this is that when there’s hairiness like this, it’s worth abstracting it away really thoroughly. I wish I’d stopped at the first abstraction leak and thought, “maybe I need to redesign rather than live with this”.

Conclusion

Even without 7-bit addressing, there would have been plenty of challenging choices in the design of the V-Drum Explorer, particularly in field and container representation. More details of those choices will come in future posts – but at least they feel inherently tricky… the kind of thing software engineers are expected to have to think about.

7-bit addressing feels like it was a choice made to make one small use case (MIDI request/response messages) simple, but made everything else trickier. I’d be fascinated to know the module code is a lot more complicated because of this as well, or whether so much is effectively hard-coded (because it needs to actually do stuff rather than just display data on a screen) that it doesn’t make much difference.

Next time: using data vs code to represent differences between modules.

7 thoughts on “V-Drum Explorer: Memory and 7-bit addressing”

  1. Hey I’m in the market for an E-Drum set. Curious if you ‘gig’ with your TD-17. I want to have an E-Drum set I can ‘gig’ with, that doesn’t sound ‘cheap’. Just curious. If I get the TD-27 I assume I’d be very interested in your work described in this blog.

    Thanks

    Like

  2. Hi, I’ve been gigging with TD-20, TD-30 and now TD-50. They all sound great. TD-17 and TD-27 are also great, but with fewer options for tweaking, not none, but fewer. Lots of people do it. I think many have concerns about the optics of the kits, they (esp TD-17) are small and toy-ish looking. Hook them up to a decent PA, you’ll need some wattage and 15″ bass speakers, and you’ll be blown away by the sound. Also, Roland builds their stuff for battle. For 10 years I’ve not had Roland fail on me a single time due to equipment failure.

    Now I’m trying to make OnSong set the click tempo via Midi, and even if the midi communication seems to work, I can’t get the tempo to change when I bring up a new song. Any hints are highly appreciated.

    Like

    1. Having sorted out the schema for the TD-27, I’d say it’s very close to the TD-50 in terms of tweaking – it’s much closer to the TD-50 than it is to the TD-17. To be honest, I suspect I’d never personally exhaust the tweakability of even the TD-17, but it’s nice to know it’s there. I don’t care much about the looks of it – but I do care about being able to get it all in my shed (home office) which would be tricky with a TD-50 or an acoustic kit. I haven’t hooked it up to a PA and I’m unlikely ever to do so, but for personal use the PM-100 is more than enough.

      In terms of setting the click tempo, it looks like that isn’t even settable via the SysEx messages I use, which is a little odd – sorry not to have any more information.

      Like

      1. I took a look in the TD-27 manual, and it’s identical to the TD-50, right down to the addresses for tempo handling. It’s on page 25 in the Midi implementation manual for your module. I’m trying something like this:

        F0 Exclusive Status
        41 Roland’s manufacturer code
        10 Device ID
        00 4 lines of TD-50 ID, this will be different for you
        00
        00
        24
        12 DT1
        2D Kit Tempo Switch
        01 Tempo on
        00 4 lines of actual tempo data
        00 4-bits, I think this means the tempo is 127
        01
        04
        7A Checksum, not remotely sure I have this one right
        7F EOX

        In my editor it does, of course, look like this:

        F0 41 10 00 00 00 24 12 2D…etc with no comments and such.

        I can see the lamp in the midi adapter blinking when I test it, but nothing changes on the module.

        Thought I’d see if I can provoke the programmer in you to take a look, but completely understand if you don’t ;-)

        Like

        1. That’s the tempo for the kit rather than the click. It’s used for things like multi-fx, where you can set the delay to a beat rather than a fixed number of milliseconds. I can set the kit tempo (I believe, anyway – I haven’t tried it but I’d certainly expect it to work) but that won’t affect the click tempo :(

          Like

          1. Aha. Here I am, actually trusting advice I got from Roland’s support in Scandinavia. Goes to show ya. Thanks for that. I’ll keep looking.

            Like

Leave a Reply to ToreL Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s