View the prototype here. It should work in any modern web browser that handles html video (even Internet Explorer 9!). Below, I’ll discuss the challenges and solutions we discovered through the process of building a prototype that tells a story as well as it demonstrates the technology.

The goal of the prototype was to give an overall sense of the subject while giving a viewer the opportunity to “drill down” to additional media for more detail. As powerful as HTML video and Popcorn are, it’s easy to overwhelm the viewer with too much information. So we had to choose the content and design the layout and interaction to tell the story while making it beautiful and avoiding distraction.

Content

Yasmine decided to highlight a conflict at the Qasr al-Nil Bridge, near Tahrir Square in Cairo. To best tell this particular story, we chose to use types of additional media, each implemented as Popcorn plugin. Relevant additional photos and video would give a deeper emotional and expository experience. Since most of our audience might not be familiar with the details of the place, we added pop-up facts, which can be expanded by hovering the mouse. Finally, a simple link appears in the lower left corner crediting the original source of the main video clip.

We considered adding a Google Map, since Popcorn has a plugin that will move the map to specified locations at any point in the video. But the Tahrir Square protests all took place around too small a space to make a moving map useful, and the source videos didn’t contain accurate enough location metadata to pinpoint specific enough locations around the square.

For the main video, Yasmine chose three short video clips, around 30 seconds each. All the clips are from a distance and give a visual overview of the event. The detail video and photos are all either closer shots or much longer, appropriate to the higher level of engagement the viewer will demonstrate by clicking to activate them.

Design

We wanted to focus more on the story than the technology and to preserve the artistic aspects of film, so it was important to make the prototype as artful as possible. Since neither of us are graphic artists, we decided to keep it as simple as possible. Yasmine made things easier by choosing beautifully shot video and photos. I designed the page to so that it’s entirely filled by video and minimized any extra visual flashes that might clash without the help of an additional designer. The videos of the event happened to appear a bit muted, which made them look nice when layered. For a more general and unpredictable set of images, I might consider using CSS opacity to mute certain components when others have focus.

Interaction

Finally, getting the interaction right took a bit of experimentation and problem-solving. There was a lot to explain, so we needed to have multiple pop-up tips. The best way we could think to organize them was to place them within the video, and to balance the level of interruption, we made them semi-transparent and showed only the title unless the mouse hovers over them. It would have been nice to have those move around the video frame as their subjects moved, but that’s quite a lot more work than we had time for in an afternoon.

The additional videos presented a bigger challenge, since they are seven or eight minutes long, while each is applied to a 30-second section of the main video. My initial thought was to pause the main video while a detail video was playing, but Yasmine thought it would be more cinematic to keep the main video playing in the background. The first thing we need to do is drop the volume of the main video to 10%, so it’s still in the background but not distracting. But what would happen if you’re playing through a seven-minute detail video and the main video gets to a new section? Your video would disappear. So now, when a detail video is playing, and the main video gets close to a new section, it skips back to the point when you activated the detail video and loops until you’re done. When you close that video, the main video returns to full volume and continues to play. Hopefully, the result is a cinematic experience that lets you explore the deeper aspects of the story while maintaining context.

Nuts and Bolts

There are a few technical tricks to keep in mind when building an interactive film such as this. At the time we built it, Popcorn only supported HTML video, using the <video> tag, so even though the source videos came from YouTube, we had to splice and re-encode the video for all the different browsers (webm for Chrome and Firefox and H264 for IE9 and Safari). We used Miro Converter, which is free and will easily convert into just about any format one needs, though there are other, more complicated free tools out there if you need to fine-tune your encode. And it’s worth noting that most web servers require a tweak to a .htaccess file to make sure mime types are properly configured to serve up the video files.

Now, Popcorn supports YouTube and Vimeo, but the native HTML video approach allows for tighter feedback and control of the timing. Splicing multiple YouTube videos in a browser could be tricky and unreliable.

Out of the box, Popcorn makes it easy to display timed content in different formats, with pretty basic Javascript skills. I was able to use the existing footnote plugin for the source credits, but I had to build custom plugins to get the positioned pop-ups and the detail videos. With Popcorn, the main video drives the timing of all the other events, so one must be careful when having the events change that main video time. Otherwise, you can end up with a loop that will make a mess of the whole thing.

Most of the rest of it can be done with Cascading Style Sheets, which handles positioning and visibility, including fades. Very soon, CSS will handle more advanced animations on all the major browsers.

Going Forward

I’m not sure what Yasmine and Jigar Mehta (her partner on 18 Days in Egypt) plan to do with the project, but we talked a bit about generating these interactive videos dynamically. Fortunately, Popcorn makes it easy to build a scene based on JSON data, which can be generated by a database. Mozilla is working on a tool that should eventually make authoring easier.

But automating the composition process would be a massive challenge, requiring precise and carefully managed metadata, which is as much a social challenge as it is a technical one. I suspect that a wiki-like community approach would work best.

Either way, this kind of interaction around video is going to take a lot of experimentation to make the technology disappear and reach the full potential of more powerful art and information.

Bookmark to:

With just a few hours advance notice, the Popcornjs crew put together a video/data mash-up of President Obama’s State of the Union speech for PBS, in which text analysis is displayed in time with the video. Among such a long video, with so much data, WMM leader Brett Gaylor asked if I could build in a feature making it easy to Twitter a short URL right to a point in the middle of the video. Sure I could. Unfortunately, an external service we were using to access the Twitter API broke down, so we couldn’t get the feature working in time to go live. I’ve worked around it, and here it is now.

Watch the video from the beginning or start 20 minutes in. Try the button just below the video. Read on to learn how it works.

The process is pretty simple. (I decided to use jQuery, since it was already loaded up for Popcorn.js, but it’s not necessary.) The three main steps are:

1. Check the URL for a time code and advance the video to the appropriate starting point.
2. Update the time code on the button.
3. Set up a “click” handler on the button to generate the link and open a Twitter window.

Here is the code to get the video started at the right place:

/* get the <video> element. Don't use jQuery here because it doesn't expose video properties and methods */
var video = document.getElementById('video');

/* advance video and auto-play */
var startVideo = function() {
    if (targetTime < video.duration) {
        video.currentTime = targetTime;
        video.play();
    }
};

/* parse URL hash after # for a number */
var targetTime = window.location.hash.substr(1);
targetTime = parseFloat(targetTime);
if (!isNaN(targetTime) && targetTime > 0) {
    if (video.readyState >= 3) {
        startVideo();
    } else {
        $('#video').bind('loadedmetadata', startVideo);
    }
}

The information we need here is how far into the video we want to stop. To keep it simple, we’re just looking for a floating point number to represent seconds. Since all this processing is happening in the browser and I don’t have access to the server anyway, we put the number in the “hash”, which is the part after the # symbol.

After checking to make sure we have a positive number, we test if the video is loaded, since we don’t always know the order in which our media and scripts will be loaded. If the video is already loaded, start up the video right away. Otherwise, bind our function to the “loadedmetadata” event so the browser will get back to us when it’s ready to start the video. By using an event handler, we don’t have to set a timeout to keep checking every few milliseconds. (Though I’ve been told “loadmetadata” can be slow to fire, so some people use a timeout, even though it’s ugly.) Once the metadata has loaded, we can find out how long the video is to make sure we’re not fast-forwarding past the end, and then it’s simple to set the time and start playing.

Next, update the time code on our button. (It’s not actually a button, just a div, but it works anyway.)

$('#video').bind('timeupdate', function(event) {
    var time = this.currentTime;
    if (time < 0) {
        time = 0;
    }

    var mins = Math.floor(time/60,10);
    var secs = Math.floor(time - mins*60);
    var hours = Math.floor(mins/60,10);
    secs = (secs > 9 ? secs : '0' + secs);

    if (hours > 0) {
        mins = mins % 60;
        mins = (mins > 9 ? mins : '0' + mins);
        $("#tweet-time").html('Tweet to ' + hours + ':' + mins + ':' + secs);
    } else {
        $("#tweet-time").html('Tweet to ' + mins + ':' + secs);
    }
});

This feature is not absolutely necessary to make this work, but it’s helpful to tell the user exactly how far into the video we’re linking and indicate that the button action is time-sensitive. The “timeupdate” event fires a few times a second, whether we’re playing or scrubbing the video, forward or reverse. Javascript doesn’t provide an easy way to make a string showing elapsed time from seconds, so we do it ourselves. jQuery makes it easy to change the text.

One problem with this code is that the button changes width as the time changes. Ideally, you’d probably check the duration of the video once at the beginning to determine whether you’re ever going to need the hours, and pick a fixed-width font.

Finally, set up the “click” event to create the link and pass it to Twitter.

var base = window.location.protocol + '//' + window.location.host + window.location.pathname;
$("#tweet-time").click(function() {
    var time = video.currentTime;
    if (time < 0) {
        time = 0;
    }

    var height = screen.height;
    var width = screen.width;

    window.open('http://twitter.com/share?count=none&counturl=' +
        encodeURIComponent(base) + '&text=&url=' + encodeURIComponent(base + '#' + time),
        'tweet-time-sotu', "left=" + Math.round((width/2)-(550/2)) +
        ",top=" + Math.round((height/2)-(450/2)) +
        ",width=550,height=450,personalbar=0,toolbar=0,scrollbars=1,resizable=1");
});

The most important point here is creating the URL to pass along to Twitter ("base + '#' + time"). One thing we need to be very careful about is that most Twitter clients parse everything after "#" as a hashtag, separate from the URL. The solution is to use a URL shortener, which is handy for hiding the "#" symbol as well as saving us characters out of the allowed 140. Originally, I used the is.gd API for this (we need a different short URL for every timecode someone uses). This worked well enough, but it required an extra call out to the network, which made things crawl on a slow connection.

Fortunately, Twitter provides a nice, clean sharing tool that will create a short URL for us. (This page explains how to use it.) We just have to make sure to wrap our target URL in encodeURIComponent so the "#" gets escaped, otherwise everything after it won't make it to the server. Normally, I hate using pop-up windows, but Twitter was pulling some kind of referrer-checking shenanigans, giving me a "Forbidden" server error when I tried to load this page inside an <iframe> so I didn't have a choice.

That's everything. Hopefully, the next time PBS and Mozilla do another one of these, they'll integrate this feature. Please comment below if you decide to use it or if you want to suggest any improvements.

Bookmark to:

For the past year or so, I’ve been hearing bits of the film industry slowly wake up to the fact that the state of media metadata is a mess. The complete lack of open, common data formats leaves media producers struggling to conform their data to a variety of proprietary formats, only to result in outdated and inaccurate information left under the control of others. Worse, existing processes discriminate against independent producers by limiting what counts as a “real” movie, album or other work.

I have been working on a project called Open Media Object Format, which is a free (both as in beer and as in speech) protocol and file format for publishing rich metadata on a range of media. The protocol creates a distributed network of information on media and its participants, allowing the information to spread while every creator maintains authority over their own work. And the format is designed to store semantic data in more detail than we’ve seen before. The Open Media Object Format is an effort to address the above problems while bringing metadata away from the mere administrative, opening up possibilities for pretty cool stuff we haven’t seen before.

Details for the roundtable and the demo are below.

Open Video Conference Roundtable

Metadata Roundtable II: Best Practices
When: Saturday, June 20, 2009 at 2:35 pm
Where: Vanderbilt Hall, NYU
Conference registration required (it’s pretty cheap)
The participants are:

• Devon Copley (session leader) – Fellow, Berkman Center for Internet and Society at Harvard University
• Brian Chirls – You know it.
• Ben Dean-Kawamura – Developer, Participatory Culture Foundation
• Mario Pena – SafeCreative
• David Rice – AudioVisual Preservation Solutions

Hack Day Demo

On Sunday June 21, 2009, I will talk through the details of Open Media Object Format and demo the first implementation of it, a major WordPress plugin (still in early beta) that any creator can use to bring media in from almost any source (YouTube, Twitter, Flickr, etc.) and republish it as a blog post with OpenMOFo XML metadata output.

It will be a great opportunity to participate in shaping the metadata standard and to get a first look at a new blog publishing platform. I’d appreciate the feedback. Hack Day is at Vanderbilt Hall, NYU, the same location as the conference, and it’s open to the public for free. I’m still figuring out the rest of the details, so check back here or follow me on twitter for the exact time. Contact me if you want to suggest a time or let me know you’re coming.

Bookmark to: