Synchronizing activities with music

It is often desired to synchronize actions or "output" to an audio stream; for example, to show individual phrases of the lyrics as subtitles, or to synchronize lighting effects with the progression of the music. Our range of programmable audio players supports this through an extension of the MP3 file format, called the ID3 tag.

So what's an ID3 tag?

The MP3 file format lacks a file header that gives information on the entire file; it only has frame headers for chunks of audio with a duration of approximately 26 milliseconds. Data such as the total duration of the track can only be determined by parsing the file from the beginning to the end, and artist/author or copyright information cannot be stored at all.

An ID3 tag is a general purpose appendage that contains such textual information. Although ID3 tags can be added to almost any multimedia file, it is in practice used for MP3 files almost exclusively. An ID3 tag is inside the same file as the MP3 audio data. Various software packages exist that add or edit ID3 tags for MP3 files.

There are two versions of the ID3 tag in existence today. Version 1 provided only a fixed set of fields (and with a fixed size). In version 2, each ID3 tag holds one or more variable-length chunks of information, called frames. Every frame holds a particular item: the title, the album name, the artist and/or band, the composer, the lyrics, the preferred volume and tone presets, etc. Version 2 of the ID3 tag is still evolving and the earlier revisions (2.0, 2.1 and 2.2) of the tag are now obsolete. Our audion players (models H0420 and H0440) support versions 2.x of the ID3 tag.

The ID3 tag, as of version 2, supports a frame for synchronized lyrics, originally intended for subtitling songs or Karoke. In the synchronized lyrics frame (abbreviated to SYLT), the text (of what is presumed to be the lyrics) is split into lines and every line gets a precision time-stamp. This allows a player to display the line at precisely the right moment.

The audio player passes the text of the time-stamped line to an event function in the script, and it provides the script writer all the needed tools to analyze the line. As such, a script writer may come up with a protocol where special text in the lines cause activities such as sending commands over the serial port or toggling I/O lines (the audio players have at least 8 general purpose I/O pins, and various other peripherals, such as a standard RS-232 port).

Reacting on time-stamped events

A script controls how the programmable audio player functions and how it responds to events. To react on SYLT data, the script must contain a function calld @synch(). When that function is present, the audio player will invoke it for every line of text and at the appropriate time. The line of text itself is in the parameter. For purposes of Karaoke or subtitling, all that is needed is to show this text. So on a H0420 (which has a connector for an LCD) the @synch() function could be as simple as the snippet below.

@synch(const event[])
    print event

Instead of printing text, I want the next example to turn on and off four LEDs that I have branched onto the top four I/O pins —the pins numbered 12 to 15. To this end, the script assumes a simple protocol where each line holds a series of codes like "+13" or "-14". The operator ("+" or "-") tells whether to turn an I/O on or off, and the number following the operator gives the I/O pin to act upon.

As an example, the line

+13 -14

turns on the LED on I/O pin 13 and turns off the LED on pin 14.

Below is the complete script for this functionality. The script plays only a single file, but it is trivially extended to play any number of files —in a loop, at random, or on a user command. Note that the I/O pins must be configured before use, through function configiopin().

main()
    {
    for (new i = 12; i <= 15; i++)
        configiopin i, Output

    play !"From-The-Machine-World.mp3"
    }

@synch(const event[])
    {
    for (new index = 0; /* test is in the middle */ ; index++)
        {
        /* find first '+' or '-' */
        new c
        while ((c = event{index}) != '-' && c != '+' && c != EOS)
            index++
        if (c == EOS)
            break       /* exit the loop on an End-Of-String */

        /* get the value behind the operator ('+' or '-') */
        new pin = strval(event, index + 1)

        /* turn on or off the led (based on the operator) */
        setiopin pin, (c == '+')
        }
    }

Adding synchronized lyrics to the MP3 file

What I have skipped so far is that for the above script to do anything at all, we first need to have an MP3 file that contains an ID3 tag with a SYLT frame. Or rather, we need to add a SYLT frame to the MP3 file.

Many ID3 tag editors are available, both freeware and commercial, but only few support the SYLT frame. For a stand-alone utility that supports SYLT frames, you can use our "SYLT Editor". The screen grab below shows the SYLT Editor on Microsoft Windows; the program is available for Linux as well.

Screen-shot of the SYLT Editor

On Microsoft Windows, an alternative is to use the "SYLT Lyrics Plugin" for WinAmp. See the bottom of this page for links to a description, screen grabs and downloads.

Downloads

To get you started, below are a few files that you can download and runs as is, or modify for your purposes.

The script file from this article that you can put on a CompactFlash card and run (this file was built for model H0420).
A prepared MP3 track with a SYLT frame containing codes for I/O pins. The track was created by Orb Gettarr, the original (without the SYLT frame) is available from the Open Sound resource.

Synchronizing activities with music

Synchronizing activities with music

So what's an ID3 tag?

Reacting on time-stamped events

Adding synchronized lyrics to the MP3 file

Downloads

Further reading & references