# Transparent videos

Starting with version 1.0 beta 17, PinballY is capable of showing videos with "alpha" transparency information embedded in the video, as well as animated GIF files with transparency.

Alpha transparency is probably a familiar concept if you've done any work with image editors. Image formats like PNG can represent partially transparent pixels, so that one image can be superimposed on top of another one, with areas of the background image showing through areas of the top image.

There's a separate kind of transparency used in GIF images, where a pixel color in the image is designated as the "invisible ink" color. All pixels in the image that match the designated invisible ink color are drawn as though there were completely transparent, and all other pixels are opaque. This is less sophisticated than alpha transparency, since it only allows an all-or-nothing kind of transparency, whereas alpha allows for continuous gradations of partial transparency.

## Where can transparent videos be used in PinballY?

Quick answer: transparent videos can be used anywhere videos can be used.

PinballY uses transparent rendering for all videos that contain alpha channel information, regardless of context. Alpha transparency is a property of the video itself, so PinballY simply uses alpha for a video if the video has alpha information in the first place.

This applies to all videos, including the background videos in the playfield window and other windows. However, it's not particularly useful for background videos, because there's nothing "behind" them to show through the transparent areas in the video - just a black background. Any completely transparent areas in a background video will thus be filled with black, and partially transparent areas will effectively look like they're shown at reduced brightness, as they'll be mixed proportionally with the black background.

Transparency is therefore really only useful for videos that are layered on top of other display areas. Currently, PinballY uses layered videos in the following contexts:

- Custom drawing layers. You can play a video in any custom drawing layer, and since these can be layered on top of other graphics, they can benefit from transparency in the video source material. See Drawing Layers.
- Launch overlays. A launch overlay is layered on top of the main playfield window, so transparent areas in a launch overlay video will show the playfield video and other elements of the normal wheel UI in the main window. See Launch overlay events and mainWindow.launchOverlay.

## Animated GIFs

The most popular current "video" format with transparency support isn't really a video format at all: it's the GIF image file format. GIF files support a simple sort of animation where a single GIF file can store a series of image frames that are played back sequentially on a timer. It's not exactly a "video" format in the usual sense, and it's not useful for anything lengthy because it doesn't compress well, but it's widely used on the Web for creating animated icons and short video loops. GIF also has a simple form of transparency, so it can be used for animated overlays - just the sort of thing you'd use alpha-channel video for.

Starting in 1.0 beta 17, PinballY supports animated GIF files, including transparent animated GIFs. You can use an animated GIF anywhere a regular still image can be used.

It might seem a bit strange, but note that all GIF files are considered image files in PinballY, never video files - even when animated. That's sometimes important when you're deciding where to store a particular media file. The rule is really simple: if it's a GIF file, treat it like an image file. Always put GIF files in the PinballY media folders designated for image files; don't put GIF files in video folders.

## Supported video formats with alpha channels

As of this writing, alpha channel support in the standard video formats is quite limited. It's not supported at all in the most common formats and codecs, including MP4 and H.264. (That's not because of PinballY limitations or libvlc limitations - it's inherent in the formats.)

The only video file formats that currently support alpha appear to be MOV (Apple's QuickTime video format), FLV (Adobe Flash Video), and AVI (Windows). However, that's hardly the full story, because you also have to use a codec that supports alpha. Here's a list of what works now:

- AVI files: PNG codec
- FLV files: PNG codec
- MOV files: PNG, QTRLE, or Apple's ProRes_KS with YUVA 4:4:4:4 at 10 bits per pixel (yuva444p10le)

Note that "PNG" in this context is a video codec, not a file format. (It does happen to use the PNG image format to encode each frame within the video, which is why the name is reused in this somewhat confusing way.) A PNG-encoded video stream can be stored in an AVI or FLV file. You can encode videos this way with ffmpeg, for example.

PNG and QTRLE are usable only for very short clips, because they use lossless encodings that result in extremely poor compression ratios compared to what we're accustomed to with modern video codecs like MP4 or H.264. So at the moment, the ProRes YUVA 4:4:4:4 format is really the only good option for full video. Fortunately, ProRes seems to be a high-quality format with great alpha support, so there's at least one good option.

This list might eventually grow, and it sure seems primed for growth given how sparse it is, but I actually don't see it growing quickly or soon. Alpha channel support in video seems to be a difficult technical problem that's of little interest to the video engineering community. It doesn't seem that anyone except the Apple ProRes group has given it much thought. (That makes ProRes a curious anomaly: computer technologies usually develop incrementally, with a series of false starts and half measures before a good solution appears. But in this case, it seems like no one else was even trying, and along comes Apple to just solve it for real.)

## Creating alpha channel video from chroma-keyed ("green screen") material

Even though alpha channel support in video formats seems to be a work in progress that's barely begun, the concept of motion overlays is hardly new. It goes back to film-based special effects. Traditionally, layered film and video composition was handled with an entirely different process, called chroma keying. This is also known as the "green screen" or "blue screen" technique, because it's widely used in film and video productions to shoot a live subject against a green or blue backdrop that's replaced in the final editing process with a special effects background.

In chroma keying, there isn't a separate "alpha" channel that records which pixels are opaque and which are transparent. Instead, you simply treat all pixels that match the key color as transparent. Every pixel that matches the key color is transparent, and the rest are opaque.

The reason I mention chroma keying is that it's been used for a long time in film and video production, so there's a lot of stock footage on the Web that uses this technique. If you're looking for material to use for overlay videos, you'll probably find lots of chroma-keyed videos and few videos using an alpha-channel format. But PinballY can only use alpha-channel videos, so chroma-keyed videos will have to be converted. That's something that FFMPEG (the popular open-source video converter program) can accomplish. Here's a fairly simple procedure you can use:

- First, identify the key color. An easy way to do this is to play the video in any video player, and take a screen shot of any frame showing the background color. Open up the screen shot in a photo editor and use the "eyedropper" tool to pick the pixel color. Note its hex RGB value. Most photo editors these days will give you an HTML "#" code for each color, like "#00FF00" for bright green. That's the hex RGB value.
- Now fire up a CMD prompt, and enter the following ffmpeg command. (I know it's awfully long and ugly, but you can mostly just copy and paste it.) You can use the copy of ffmpeg that's in your PinballY install folder. Replace the 0x00ff00 part in the chromakey section with the hex RGB color value you noted above. (Note that this all goes on one command line in the CMD window - it's very long so it'll probably wrap as displayed here.)
  ffmpeg -i source_video_file -vf "chromakey=0x00ff00:0.01:0.01" -codec prores_ks -pix_fmt yuva444p10le -f mov output_file_name.mov

  Note that there are two extra parameters in the chromakey section, following the hex RGB color value, which you might also want to play with to optimize the conversion. The first 0.01 value is the "tolerance" for matching the key color. 0.01 is 1% tolerance, meaning that the color has to match almost exactly. This works well for computer-generated videos with perfectly uniform backgrounds, but material with a live subject against a green screen might have a lot more variation in the background. Adjust this upwards if areas of the background remain opaque in the converted video. The second 0.01 value sets how much of a transparency ramp is used to adjust pixels that are close to the background color but not exact. Again, a small value like .01 is good for computer-generated material; this will result in sharp edges around the subject. A higher value will make the edges appear softer by making them partially transparent, which might look better with live subjects. But too high a value will make the subject look too transparent, so you might have to experiment to find the right value for a given video.