WPF Audio Visualizer

WPF C# Audio Visualizer from Anton Winter on Vimeo.

Here is a quick 30 second clip of my WPFaudio visualizer.


WPF does a lot of things well, but the things it doesnt do require us developers to delve deep into the internals of Windows.

Its built with WPF including the new shader effects and Directshow with the sample grabber. The WPF communicates via interop to a small C++ component that then builds a filter graph for the MP3 and connects the sample grabber.

The Shader effects come from the WPF toolkit. In particular, the effects i am using are Pinch, Swirl and twist.


On a political note, i picked NIN as the song because of their strong stance on Creative Commons Releases. The last thing i want is a music company lawyer giving me a takedown notice for having this demo up on my blog. So, thanks Nine Inch Nails for the music.

wpf solution

In my previous post i talked about running into a WPF performance problem. I had several thousand controls that i added to a stackpanel. the choke point was the stackpanel.children.add function.

Adding 10,000 items was taking around about 10 seconds to run. 8.5 seconds was spent on the stackpanel.children.add. around 1.5 seconds was spent on creating the controls.

To fix the problem I have used a variation on the WPF virtualizingstackpanel. The theory behind virtualizing stackpanels is that the stackpanel only initialises and displays the items that are actually on screen. When the stackpanel has to show new gui elements, it creates the new elements and distroys any that are no longer on screen.

The 2 engineering problems this brings up are 1- how to work out what is on screen and 2 how to store all the objects that may be potentially turned into gui elements.

For my project, i have a tv guide that has 100's of tv stations and each one needs 1 weeks worth of guide data.

To store the data i have essentially made a big 2d array.

When the user scrolls the guide, new gui elements are created on the fly and added to the stackpanel.

The resulting code now takes less than 100 milliseconds to load up rather than 10 seconds.

So that gives a speed up of around 100 times.

WPF gotcha stackpanel.children.add

I havent blogged for a while because i've been working on integrated live tv tuning and WPF into my media player. This has not been a trivial task. Getting access to the directshow filter graph to manually build the graph is not something that WPF was designed for. More on this on a later blog.

Todays note is a problem that i have run into in dynamically constructing a large GUI in WPF. i have around about 10,000 custom controls that are added to a stackpanel. ( the custom controls each represent a tv show in a TV guide ).

Constructing and populating the custom controls is pretty quick. Adding them to the stackpanel is a problem. 10,000 controls take about 10 seconds to add. This is way to slow for my liking. My first thought was to move it into a worker thread, however because WPF is STA and adding anything to the GUI has to be done on the main thread, the way to do this isn't obvious. Once I have found a solution, I'll post it up here.

C# WPF SoundPlayer

[link to code file]

When you want a lite weight sound object in WPF, the SoundPlayer object is the thing to use.

Its got a very small API.

For most applications, all you need to do is create an instance with a string to the wav file that you want to load and then call the Play() function

example

SoundPlayer sp = new SoundPlayer(@"c:\temp\mysound.wav");
sp.Play();

SoundPlayer goes off and loads the wav file and the plays the wav file in a background thread, like it should.

Thats great unless I want it associated with a remote control button press

My first run at SoundPlayer left me with a less than optimum user experience. I wanted to play a sound when I handled a button press on the Microsoft Media Center remote control.

If the user holds down a remote control button, then under Windows XP, it makes the sound, then intermittently makes the sound from then on. Not too bad a result.

Under Vista, its a different story. Vista starts to play the sound, then blocks it and tries to play the sound again. The result is a "stutter" from the wav file being played over and over again. Even worse, it queues up so that the stutter continues long after the remote button is released.

Welcome to the extended SoundPlayer

Extended SoundPlayer. Is a simple derived class based on SoundPlayer. The big change is that it doesn't try to play another sound if it is already playing a sound. The other functionality it gives is that you can check if the sound is playing from your own code.

Feel free to use Extended Sound Player for your own projects. It is provided without any support or warranty. The blogging editor removes the Tabs from the code, so you'll have to reformat the code.

using System;
using System.Collections.Generic;
using System.Text;
using System.Media;
using System.Threading;
using System.Windows.Threading;
namespace ExtendedSoundPlayer
{
class CExtendedSoundPlayer : SoundPlayer
{
private bool myBoolIsplaying = false;

public bool bIsPlaying
{
get { return myBoolIsplaying; }
}

public CExtendedSoundPlayer(String strFilename) : base(strFilename) { }

public void PlaySound()
{
if (!bIsPlaying)
{
Thread threadSound = new Thread(new ThreadStart(PlaySoundThread));
threadSound.Start();

}

}

protected virtual void PlaySoundThread()
{
myBoolIsplaying = true;

//PlaySync plays the sound in the same thread and doesn't return till it is finished.
PlaySync();

myBoolIsplaying = false;
}
}
}

More ways to skins a cat

C# also includes the SoundPlayerAction class which allows you to connect up the sound player to specific events in WPF.

XAML, C# & WPF - How to structure your project

For years I have been a big Flash fan and developer. When I first opened up XAML and Expression Blend, I had expectations that I would be developing GUI's and animations the same way I have been in Flash. I found out that although there are similarities, it is not possible to structure your project in the same way.

The best way in Flash

In Flash, the best way to structure your project is to create a lot of separate animations that are self contained add them to your library and then drop them either directly into main time line or into an encapsulating animation which subsequently gets dropped into the main time line.

It is then easy to let your animations jump from one animation to the next. In Flash the animations and the GUI can be highly organized so that as your project gets big, it is still manageable.

Expression doesn't work the same way, I wish it did - how to fix it

Expressions main navigation mechanism is the page. The page gives you some free functionality. Users are able to navigate forward and back exactly the same way they can in a web browser.

So with page navigation, as developers we create separate pages that encapsulate their own GUI, animation and WPF takes care of the rest of the navigation.

There are 2 really big downfalls with pages.

Pages don't allow you to have animation transitions from one page to the next.

And

The GUI makes a "clicking sound" when you press the forward and back button as well as any link you have from one page to the next.

What the hell were they thinking with that click. I have read some articles on it and they say its because it is built into the browser. As of .NET 3.0, there is no way for us to turn off the clicking from our application. The suggested way is to have the user turn it off from their system sound settings. This completely blows.

The second navigation choice - User controls

With Blend and WPF you can create User controls which are independent chunks of GUI, animation and code.

User controls allow us to create an organized and structured project that can scale in size and complexity without getting unwieldy.

So now we can get back to organizing our GUI and animations back in a format that makes sense and is similar to the way Flash does it.

Just one more hurdle to jump

If like me you have used Flash, then you would expect that there would be an easy way to drop user controls into the timeline and they can either play one after another or you can jump to another user control when user clicks on a button. Blend doesn't give you this functionality.

Attempting to do it purely in Blend

As far as I can make out the only way to have animations play one after another or have the animations change to the next user control in Blend is to have all the user controls sitting in the timeline right from the start and have their visibility turned to Hidden or Collapsed.

This is a truly messy solution. It works for trivial projects, but not for full blown applications.

Finally a way to do it

Using pages and using user controls in Blend really are dead ends. I spent a number of days banging my head and searching high and low for the best practice on how to structure WPF.

The solution is to stick with the user controls, have your timeline empty or just have an initial user control in there then drop back into C# to control the adding and removing of user controls to the GUI.

In C# we can hook into the user controls animation events and user interaction events.

Code to animate/fade in a user control


public void FadeIn()
{
myLayoutGrid.Children.Add(this);

Storyboard sbMenuFadeIn = (Storyboard)FindResource("MenuFadeIn");
sbMenuFadeIn.FillBehavior = FillBehavior.HoldEnd;
this.BeginStoryboard(sbMenuFadeIn);
}


This code is inside my user control C# class. Stepping through the code line by line

myLayoutGrid is a property of the class that is points to the main Grid used in my XAML. You can think of the Grid as an empty screen that is ready to put user controls onto.

I have my main window pass the user control a ref to the grid when it initialises the user control.
Using the Children.Add(this) places the user control onto the grid.

The rest of the code finds the storyboard called MenuFadeIn, sets it to stop animating at the end of the storyboard and then actually starts the animation.

MenuFadeIn must be a valid storyboard animation that belongs to the user control. I built the animation in Blend to make it easier.

Code to fade out a user control

fading out the control takes a little bit more work, but not much more.

The steps are

• Find the resource to the animation
• Add an event handler to call when the animation ends
• Play the animation
• when the animation end event fires, remove the user control from the grid.

And here is the code

public void FadeOut()
{
Storyboard sbMenuFadeOut = (Storyboard)FindResource("MenuFadeOut");

sbMenuFadeOut.Completed += new EventHandler(FadeOutFinished);
this.BeginStoryboard(sbMenuFadeOut);
}

// event that gets fired when the animation completes
void FadeOutFinished(object sender, EventArgs e)
{
myLayoutGrid.Children.Remove(this);
}

So there you have it, now you can happily create some stunning interfaces and be able to structure your code well.

Is WPF ready for primetime?

Like everything Microsoft, I'm skeptical that wpf will work before version 7. So is WPF ready with .NET 3.0?

I dive into the samples, look at video tutorials on it and whip up some XAML with C# backends to see how it all hangs together.

I am in hog heaven, little did I know that the painful side of Microsoft's API was yet to show itself.

On the surface it looks good, really good. In fact I start hearing myself saying "this is going to change windows coding forever" to my geekier friends.

I get a super slick Apple style Flow menu going with some movie posters from www.impawards.com and code them against a couple digital movies I have to see how it all goes.

Damn, its fast to put it together. its slick and couldn't be easier.


Here's a screen shot. Its hard to convey how good it looks in a still image. I set up a moving "lightshow" background that animates while the menu is running. The background movie is set for 720p so there is no graininess at all.




I start with XAML and use Blend to get the animations going. MediaElement is the main workhorse for displaying the movies.

So I've got one damn fine prototype going and I start thinking to myself, hey this thing is doable, in fact it might be trivial to do. Boy was I wrong. more info next blog

Back into coding

I've been getting back into code recently. It's been a while, but its all coming back to me.

Microsofts WPF looks pretty good. I've decided to use it as the foundation for my media center killer app.

I couldn't find a WPF logo, so the silverlight one will have to do