Synchronizing activities with music

It is often desired to synchronize actions or "output" to an audio stream; for example, to show individual phrases of the lyrics as subtitles, or to synchronize lighting effects with the progression of the music. The H0420 programmable MP3 player supports this through an extension of the MP3 file format, called the ID3 tag.

So what's an ID3 tag?

The MP3 file format lacks a file header that gives information on the entire file; it only has frame headers for chunks of audio with a duration of approximately 26 milliseconds. Data such as the total duration of the track can only be determined by parsing the file from the beginning to the end, and artist/author or copyright information cannot be stored at all.

An ID3 tag is a general purpose appendage that contains such textual information. Although ID3 tags can be added to almost any multimedia file, it is in practice used for MP3 files almost exclusively. An ID3 tag is inside the same file as the MP3 audio data. Various software packages exist that add or edit ID3 tags for MP3 files.

There are two versions of the ID3 tag in existence today. Version 1 provided only a fixed set of fields (and with a fixed size). In version 2, each ID3 tag holds one or more variable-length chunks of information, called frames. Every frame holds a particular item: the title, the album name, the artist and/or band, the composer, the lyrics, the preferred volume and tone presets, etc. Version 2 of the ID3 tag is still evolving and the earlier revisions (2.0, 2.1 and 2.2) of the tag are now obsolete. The H0420 MP3 player supports versions 2.x of the ID3 tag.

The ID3 tag, as of version 2, supports a frame for synchronized lyrics, originally intended for subtitling songs or Karoke. In the synchronized lyrics frame (abbreviated to SYLT), the text (of what is presumed to be the lyrics) is split into lines and every line gets a precision time-stamp. This allows a player to display the line at precisely the right moment.

The H0420 pases the text of the time-stamped line to an event function in the script, and it provides the script writer all the needed tools to analyze the line. As such, a script writer may come up with a protocol where special text in the lines cause activities such as sending commands over the serial port or toggling I/O lines (the H0420 is equiped with a standard RS-232 port and it has 16 general purpose I-O pins).

Reacting on time-stamped events

When the script for the H0420 programmable MP3 player implements the function @synch(), the MP3 player will invoke this function for every line of text and at the appropriate time. The line of text itself is in the paramter. For purposes of Karaoke or subtitling, all that is needed is to print this text, so the @synch() function could be as simple as the snippet below.

@synch(const event[])
    print event

Instead of printing text, I want the next example to turn on and off four LEDs that I have branched onto the top four I/O pins --the pins numbered 12 to 15. To this end, the script assumes a simple protocol where each line holds a series of codes like "+13" or "-14". The operator ("+" or "-") tells whether to turn an I/O on or off, and the number following the operator gives the I/O pin to act upon.

As an example, the line

+13 -14

turns on the LED on I/O pin 13 and turns off the LED on pin 14.

Below is the complete script for this functionality. The script plays only a single file, but it is trivially extended to play any number of files --in a loop, at random, or on a user command. Note that the I/O pins must be configured before use, through function configiopin().

main()
    {
    for (new i = 12; i <= 15; i++)
        configiopin i, Output

    play !"From-The-Machine-World.mp3"
    }

@synch(const event[])
    {
    for (new index = 0; /* test is in the middle */ ; index++)
        {
        /* find first '+' or '-' */
        new c
        while ((c = event{index}) != '-' && c != '+' && c != EOS)
            index++
        if (c == EOS)
            break       /* exit the loop on an End-Of-String */

        /* get the value behind the operator ('+' or '-') */
        new pin = strval(event, index + 1)

        /* turn on or off the led (based on the operator) */
        setiopin pin, (c == '+')
        }
    }

Adding synchronized lyrics to the MP3 file

What I have skipped so far is that for the above script to do anything at all, we first need to have an MP3 file that contains an ID3 tag with a SYLT frame. Or rather, we need to add a SYLT frame to the MP3 file.

Many ID3 tag editors are available, both freeware and commercial, but only few support the SYLT frame. Microsoft Windows users can use the "SYLT Lyrics Plugin" for WinAmp. The screen grab below shows the SYLT plugin in action.

Screen grab of the SYLT Lyrics Plugin for WinAmp

Downloads

To get you started, below are a few files that you can download and runs as is, or modify for your purposes.

The script file from this article that you can put on a CompactFlash card and run.
A prepared MP3 track with a SYLT frame containing codes for I/O pins. The track was created by Orb Gettarr, the original (without the SYLT frame) is available from the Open Sound resource.

	Introduction
	Application notes
	Microsoft Windows - technical articles
	Embedded systems
	Computer graphics
	Computer animation
	File formats
	Miscellaneous articles
	Published articles

Synchronizing activities with music

So what's an ID3 tag?

Reacting on time-stamped events

Adding synchronized lyrics to the MP3 file

Downloads

Further reading & references