Silverlight and Pluggable Codecs
Flash has been doing video on the web for years now, and doing an awesome job at it. The Flash Player has a number of built-in codecs, and all the Flash video on the web plays in the latest Flash player, no hassles. And HD videos look great.
Microsoft, on the other hand, has their various video container formats (ASF, WMV, AVI) where the codec that you need to play the content is dependent on the streams in the containers. In other words, given a *.avi file, you have no idea what you need to have installed to play it.
This leads to the various codec packs you can download, which almost invariably are a vehicle for malware. It’s a terrible situation, and I think one of the reasons Flash video has done so well.
Microsoft apparently doesn’t get this, because the latest version of Silverlight has “Extensible Media Format Support”, which means developers can plug in arbitrary video or audio codecs to support other video formats.
Argh.
So one site will be posting Ogg-contained videos and another DivX videos, and I’m going to need to download a bunch of codecs to view videos with Silverlight? No thanks.
April 8th, 2009 at 4:04 am
I think that the majority of sites will be using the new H.264 support in Silverlight 3 rather than relying on separate codecs, which may limit their reach. Certainly, the extensibility may be advantageous in certain circumstances (e.g. if a site specialised in MIDI files or 8-bit sound formats). The other advantage is that new formats can be added without having to refresh the entire runtime, which has had to happen with Flash in the past. As for the malware reference, well that’s just FUD – the codecs are authored using a .NET language and are appropriately sandboxed; there’s no need to run an exe to download a codec pack (see Miguel de Icaza’s blog entry here: http://tirania.org/blog/archive/2009/Mar-23-1.html). Maybe someone will write an FLV viewer ;)
May 2nd, 2009 at 6:54 pm
Wow, the things we find when trying to search for something else :).
I’m the video strategist on the Silverlight platform, so this is something I’ve been working on for a while.
The key thing about the Silverlight Raw AV pipeline is that the “managed decoders” are packaged into the Silverlight application itself, just like the player graphics, managed code for the player itself, etcetera. So there’s not anything for the user to download, no authentication or anything. This is possible because the decoders must be pure managed code that runs entirely in the Silverlight sandbox, like all other code running inside of Silverlight. So, the user experience will be the exact same between using a native and a managed codec.
For security and maintainence reasons, there’s no way to override the .dll used by a Silverlight app unless the Silverlight app is designed with that kind of flexibiility, so the Codec Pack nightmare won’t happen here. FWIW, Windows 7′s built-in media apps are locked to their own H.264 and other decoders, so that codec packs can’t mess with those either anymore.
Of course, managed code won’t have the full performance of a highly-tuned SSE2 decoder; that’s why we’re also adding native H.264 and AAC-LC to add to the existing VC-1 and WMA support. But for legacy content that’s lightweight to decode, or stuff that’s not HD, it’ll just be fine.
This guy had the crazy idea of sticking a full Commadore 64 emulator inside Silverlight using the API:
http://community.irritatedvowel.com/blogs/pete_browns_blog/archive/2009/03/18/Silverlight-3-_1320_-Creating-Video-from-Raw-Bits-using-a-MediaStreamSource.aspx
Not anything we imagined when we came up with the feature, but a fine demonstration of its flexibility.
May 4th, 2009 at 7:14 am
Thanks for the update Ben – so the H.264 decoder won’t be a plugin, it’ll be a new version of Silverlight?
(That’s how it should be IMHO).