Subservient Cat is a sort of “virtual pet” app. Unlike most cats, the subservient cat can actually obey commands! However, users have to figure out what commands the cat will respond to. It’s a bit of a game, because users must keep guessing to see how many commands they can discover.
This app uses video footage of a black cat (whose real name is Boo) as its primary user interface. Therefore, this is the perfect app for learning about MediaElement, the element for playing video inside apps.
Playing Video with MediaElement
If you want to enable users to play, pause, and otherwise control videos, your best option is to use the Media Player launcher. However, if you want to play video inline on your own page, MediaElement enables you to do so.
MediaElement is a UI element that renders the video specified via its Source property, for example:
[code]
<MediaElement Source=”cat.wmv”/>
[/code]
The source URI can point to a file included in your project or to an online video. By default, MediaElement automatically starts playing the video as soon as the element is rendered (or as soon as enough of the video has been downloaded, in the online case), but you can change this by setting its AutoPlay property to false. MediaElement has Play, Pause, and Stop methods that can be used from code-behind. It also has a Position property that reveals the current playback position (as a time span). If the video supports seeking, you can also set Position to move playback to that point in time. MediaElement can be transformed and clipped just like other Silverlight elements, and it can blended amongst other elements.
On the surface, it appears that MediaElement is straightforward to use. It has tons of caveats, however. Several warnings follow that describe some of the biggest caveats.
An app’s frame can only contain one MediaElement!
Attempting to use more than one MediaElement is not supported and fails in various ways.Note that the limitation is more strict than one per page; only one can be attached to the frame at any time. (It doesn’t matter if they’re stopped, paused, or playing.) Therefore, multiple pages can each have a MediaElement only if they never reside on the navigation stack at the same time. Otherwise, if you need to play multiple videos, you’ll either need to reuse the same MediaElement or remove the unused MediaElement from the element tree.
When including a video file in your app, make sure its Build Action is set to Content, not Resource!
This improves the performance of starting the video. When the video is embedded as a resource, it is first extracted and temporarily saved to isolated storage before it is played! (The same warning applies to audio files played by MediaElement, although this is not normally done.)
When MediaElement starts playing, any background audio (such as music playing from Zune) is paused!
This is the main reason that MediaElement should never be used for playing sound effects. Note that this is true even if your video contains no audio.
MediaElement doesn’t work in the emulator under the light theme!
It’s strange, but true.To test your use of a MediaElement on the emulator, you must ensure that it is running under the dark theme.Don’t worry; this bug doesn’t affect real phones.
MediaElement doesn’t render fully opaque!
If any elements exist underneath a MediaElement, you can see them clearly through the video, even when MediaElement’s Opacity property is set to 1 (as it is by default)! This is an anomaly with the composition between the phone’s media player (which internally renders MediaElement’s video) and the rest of Silverlight.
See “Supported Media Codecs forWindows Phone” (http://goo.gl/6NhuD) for details about the video formats supported by MediaElement and the “Recommended Video Encoding Settings” section of http://goo.gl/ttPkO for more details about what encodings work best. If you use Expression Encoder, you can encode videos with a preconfigured profile optimized for Windows Phone (and Zune HD).
The User Interface
The Subservient Cat app uses a single page—MainPage—in addition to its instructions page (not shown in this chapter). The XAML for MainPage is shown in Listing 33.1. The root grid contains three distinct pieces of user interface, all shown in Figure 33.1:
- The MediaElement that contains the video
- A simple “intro screen” that introduces each command done by the cat before the corresponding video clip plays
- A panel with a text box for guessing new commands
LISTING 33.1 MainPage.xaml—The User Interface for Subservient Cat’s Main Page
[code]
<phone:PhoneApplicationPage
x:Class=”WindowsPhoneApp.MainPage”
xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
xmlns:shell=”clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone”
FontFamily=”{StaticResource PhoneFontFamilyNormal}”
FontSize=”{StaticResource PhoneFontSizeNormal}”
Foreground=”{StaticResource PhoneForegroundBrush}”
SupportedOrientations=”Landscape” Orientation=”Landscape”>
<!– The application bar–>
<phone:PhoneApplicationPage.ApplicationBar>
<shell:ApplicationBar Opacity=”.8”>
<shell:ApplicationBarIconButton Text=”command”
IconUri=”/Shared/Images/appbar.command.png”
Click=”CommandButton_Click”/>
<shell:ApplicationBarIconButton Text=”instructions”
IconUri=”/Shared/Images/appbar.instructions.png”
Click=”InstructionsButton_Click”/>
<shell:ApplicationBarIconButton Text=”discovered”
IconUri=”/Shared/Images/appbar.1.png”
Click=”DiscoveredButton_Click”/>
</shell:ApplicationBar>
</phone:PhoneApplicationPage.ApplicationBar>
<Grid>
<!– The video, scaled to fill the page without visible letterboxing –>
<MediaElement x:Name=”MediaElement” Source=”cat.wmv” Stretch=”None”
Volume=”1” MediaOpened=”MediaElement_MediaOpened”
MediaFailed=”MediaElement_MediaFailed”>
<MediaElement.RenderTransform>
<CompositeTransform TranslateY=”-115” TranslateX=”-150”
ScaleX=”1.68” ScaleY=”1.68”/>
</MediaElement.RenderTransform>
</MediaElement>
<!– Something to show on top of the video during seeking –>
<Border x:Name=”IntroScreen” Background=”#6E5962” Visibility=”Collapsed”>
<Grid>
<Image Source=”Images/paw.png” Stretch=”None” HorizontalAlignment=”Left”
VerticalAlignment=”Top” Margin=”100,100,0,0”/>
<Image Source=”Images/paw.png” Stretch=”None” HorizontalAlignment=”Left”
VerticalAlignment=”Top” Margin=”230,50,0,0”/>
<Image Source=”Images/paw.png” Stretch=”None” HorizontalAlignment=”Left”
VerticalAlignment=”Top” Margin=”628,300,0,0”/>
<Image Source=”Images/paw.png” Stretch=”None” HorizontalAlignment=”Left”
VerticalAlignment=”Top” Margin=”528,350,0,0”/>
<TextBlock x:Name=”NextCommandTextBlock” Foreground=”Black” FontSize=”40”
VerticalAlignment=”Center” HorizontalAlignment=”Center”/>
</Grid>
</Border>
<!– The user interface for typing new commands –>
<Grid x:Name=”CommandPanel”
Background=”#A000”
Visibility=”Collapsed” VerticalAlignment=”Bottom”>
<Grid.RowDefinitions>
<RowDefinition/>
<!– A bottom margin, so the auto-scroll while the textbox has focus
doesn’t show extra space below the video –>
<RowDefinition Height=”84”/>
</Grid.RowDefinitions>
<Grid.ColumnDefinitions>
<ColumnDefinition Width=”Auto”/>
<ColumnDefinition/>
</Grid.ColumnDefinitions>
<Image Source=”/Shared/Images/appbar.command.png” Stretch=”None”
Margin=”70,0,0,0”/>
<Rectangle Grid.Column=”1” Fill=”White” Margin=”12,12,82,12”/>
<TextBox x:Name=”CommandTextBox” Grid.Column=”1” InputScope=”Text”
Margin=”0,0,70,0” TextChanged=”CommandTextBox_TextChanged”
LostFocus=”CommandTextBox_LostFocus”/>
</Grid>
</Grid>
</phone:PhoneApplicationPage>
[/code]
- The video is landscape-oriented, so this is a landscape-only page. However, because a text box is used for guessing new commands, the code-behind temporarily changes SupportedOrientations to allow any orientation while the text box has focus. This way, phones with portrait hardware keyboards still get a good experience.
- The application bar has three buttons: one for revealing the command input panel, one for navigating to the instructions page, and one whose icon reveals how many commands have been discovered (updated in code-behind). Tapping this last button also tells you whether there are more commands to be discovered, as the total number is a secret. The application bar menu, dynamically filled from code-behind, contains the list of discovered commands and makes the cat perform each one when tapped. This is shown in Figure 33.2.
- Although the app appears to play several different short videos, it actually uses a single longer video (cat.wmv) for performance reasons. The code-behind is responsible for playing the appropriate segments of the video at the appropriate times.
- The MediaElement is moved and enlarged with a CompositeTransform because the source cat.wmv file has black bars along the top and bottom that we don’t want to see on the screen.
- MediaElement’s Volume property (a value from 0 to 1) is set to 1 (the loudest possible volume) because the default value is .85. Although the sound can only be as loud as the user’s volume level, this helps ensure that the tiny bit of audio in this app’s video (a short meow) can be heard.
Be sure to give your MediaElement a name!
If you don’t, it’s possible that the marketplace publishing process won’t detect your use of MediaElement, and therefore will not grant your app the “media library” capability that is necessary for your app to work.
The Code-Behind
Listing 33.2 contains the code-behind for the main page.
LISTING 33.2 MainPage.xaml.cs—The Code-Behind for Subservient Cat’s Main Page
[code]
using System;
using System.Collections.Generic;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Media;
using System.Windows.Navigation;
using System.Windows.Threading;
using Microsoft.Phone.Controls;
using Microsoft.Phone.Shell;
namespace WindowsPhoneApp
{
public partial class MainPage : PhoneApplicationPage
{
// Captures the slice in the big video where each command resides
class VideoClip
{
public TimeSpan Start;
public TimeSpan End;
}
DispatcherTimer videoTimer = new DispatcherTimer();
DispatcherTimer delayTimer = new DispatcherTimer();
VideoClip pendingNewClip;
Dictionary<string, VideoClip> possibleCommands =
new Dictionary<string, VideoClip>();
Dictionary<string, string> aliases = new Dictionary<string, string>();
// Start users off with one known command: yawn
Setting<List<string>> discoveredCommands =
new Setting<List<string>>(“DiscoveredCommands”,
new List<string>(new string[] { “yawn” }));
IApplicationBarIconButton discoveredButton;
public MainPage()
{
InitializeComponent();
this.discoveredButton = this.ApplicationBar.Buttons[2]
as IApplicationBarIconButton;
this.videoTimer.Tick += VideoTimer_Tick;
this.delayTimer.Tick += DelayTimer_Tick;
// All the commands and their positions in the video
this.possibleCommands.Add(“yawn”, new VideoClip {
Start = TimeSpan.FromSeconds(98.36),
End = TimeSpan.FromSeconds(101.28) });
this.possibleCommands.Add(“meow”, new VideoClip {
Start = TimeSpan.FromSeconds(79.4),
End = TimeSpan.FromSeconds(81.863) });
…
// Permitted variations for each command
this.aliases.Add(“yawn”, “yawn”);
this.aliases.Add(“meow”, “meow”);
this.aliases.Add(“speak”, “meow”);
this.aliases.Add(“talk”, “meow”);
this.aliases.Add(“say your name”, “meow”);
…
}
protected override void OnNavigatedTo(NavigationEventArgs e)
{
base.OnNavigatedTo(e);
// Fill the application bar menu with all previously-discovered commands
this.ApplicationBar.MenuItems.Clear();
foreach (string command in this.discoveredCommands.Value)
{
ApplicationBarMenuItem item = new ApplicationBarMenuItem(command);
item.Click += ApplicationBarMenuItem_Click;
this.ApplicationBar.MenuItems.Add(item);
}
// Show how many commands have been discovered on the button imag
this.discoveredButton.IconUri = new Uri(“/Shared/Images/appbar.”
+ this.discoveredCommands.Value.Count + “.png”, UriKind.Relative);
}
void MediaElement_MediaOpened(object sender, RoutedEventArgs e)
{
// Play a short intro clip
this.videoTimer.Interval = TimeSpan.FromSeconds(1.48);
this.videoTimer.Start();
}
void MediaElement_MediaFailed(object sender, ExceptionRoutedEventArgs e)
{
MessageBox.Show(“To see the subservient cat, please disconnect your “ +
“phone from Zune.”, “Please Disconnect”, MessageBoxButton.OK);
}
void PlayClip(string title, TimeSpan beginTime, TimeSpan endTime)
{
// Set up the timer to stop playback approximately when endTime is reached
this.videoTimer.Stop();
this.videoTimer.Interval = endTime – beginTime;
// Hide the video and show the intro screen
this.MediaElement.Pause();
this.NextCommandTextBlock.Text = title;
this.IntroScreen.Visibility = Visibility.Visible;
// Give the intro screen a chance to show before doing the following work
this.Dispatcher.BeginInvoke(delegate()
{
// Delay the reappearance of the video until after the seek completes
this.delayTimer.Interval = TimeSpan.FromSeconds(2);
this.delayTimer.Start();
// Seek to the correct spot in the video
this.MediaElement.Position = beginTime;
});
}
void VideoTimer_Tick(object sender, EventArgs e)
{
// We’ve reached the end of the current clip, so pause the video
this.MediaElement.Pause();
// Prevent the timer from continuing to tick
this.videoTimer.Stop();
}
void DelayTimer_Tick(object sender, EventArgs e)
{
// This timer is used for two reasons, either to delay the execution of a
// new command after typing it in, or to delay the beginning of playing a
// video clip after the intro screen is shown.
if (this.IntroScreen.Visibility == Visibility.Collapsed)
{
// This is the execution of a new command
string text = this.CommandTextBox.Text;
this.CommandTextBox.Foreground = new SolidColorBrush(Colors.Black);
this.CommandTextBox.Text = “”;
this.Focus(); // Closes the command input panel
PlayClip(text.ToLowerInvariant(), this.pendingNewClip.Start,
this.pendingNewClip.End);
}
else
{
// We’re ready to actually play the video clip
this.videoTimer.Start();
this.MediaElement.Play();
this.IntroScreen.Visibility = Visibility.Collapsed;
}
// Prevent the timer from continuing to tick
this.delayTimer.Stop();
}
void CommandTextBox_LostFocus(object sender, RoutedEventArgs e)
{
// Restore the page to landscape-only and hide the command input panel
this.SupportedOrientations = SupportedPageOrientation.Landscape;
this.CommandPanel.Visibility = Visibility.Collapsed;
}
void CommandTextBox_TextChanged(object sender, TextChangedEventArgs e)
{
string text = this.CommandTextBox.Text.Trim().ToLowerInvariant();
// Don’t bother checking when the text is shorter than any valid command
if (text.Length < 3)
return;
if (this.aliases.ContainsKey(text))
{
string commandName = this.aliases[text];
// Only acknowledge the command if it’s not already in the list
if (!this.discoveredCommands.Value.Contains(text))
{
// Signal a successful guess
this.CommandTextBox.Foreground =
Application.Current.Resources[“PhoneAccentBrush”] as Brush;
this.pendingNewClip = this.possibleCommands[commandName];
// Append the new command to the application bar menu
ApplicationBarMenuItem item = new ApplicationBarMenuItem(text);
item.Click += ApplicationBarMenuItem_Click;
this.ApplicationBar.MenuItems.Add(item);
// Record the discovered command
this.discoveredCommands.Value.Add(text);
this.discoveredButton.IconUri = new Uri(“/Shared/Images/appbar.” +
this.discoveredCommands.Value.Count + “.png”, UriKind.Relative);
// Wait a second before hiding the text box and showing the intro screen
this.delayTimer.Interval = TimeSpan.FromSeconds(1);
this.delayTimer.Start();
}
}
}
// Application bar handlers
void ApplicationBarMenuItem_Click(object sender, EventArgs e)
{
IApplicationBarMenuItem item = sender as IApplicationBarMenuItem;
// Grab the right clip based on the menu item’s text
VideoClip command = this.possibleCommands[this.aliases[item.Text]];
this.Focus(); // In case the command input panel is currently showing
PlayClip(item.Text, command.Start, command.End);
}
void CommandButton_Click(object sender, EventArgs e)
{
// Temporarily allow a portrait orientation while the text box is in use
this.SupportedOrientations = SupportedPageOrientation.PortraitOrLandscape;
this.CommandPanel.Visibility = Visibility.Visible;
// Enable automatic deployment of the software keyboard
this.CommandTextBox.Focus();
this.CommandTextBox.SelectAll();
}
void InstructionsButton_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/InstructionsPage.xaml”,
UriKind.Relative));
}
void DiscoveredButton_Click(object sender, EventArgs e)
{
// Pause, otherwise the video will keep playing while the message box is
// shown and the timer Tick handler won’t be raised to stop it!
this.MediaElement.Pause();
if (this.discoveredCommands.Value.Count == this.possibleCommands.Count)
{
MessageBox.Show(“Congratulations! You have discovered all the commands!”,
“Discovered Commands”, MessageBoxButton.OK);
}
else if (this.discoveredCommands.Value.Count == 1)
{
MessageBox.Show(“You have discovered only one command, and it was given” +
“ to you! Keep trying! (You can see and use your command by tapping” +
“ ”…”)”, “Discovered Commands”, MessageBoxButton.OK);
}
else
{
MessageBox.Show(“You have discovered “ +
this.discoveredCommands.Value.Count + “ commands. (You can see them” +
“ and use them by tapping ”…”) There are more to discover!”,
“Discovered Commands”, MessageBoxButton.OK);
}
}
}
}
[/code]
- The constructor populates possibleCommands with the list of commands along with their starting and ending times in the cat.wmv video. Because guessing each command by its exact name can be incredibly hard, an aliases dictionary is used that enables alternate forms of some commands, such as “speak” instead of “meow.”
- In OnNavigatedTo, the application bar menu is filled with all previously discovered commands. These are stored in discoveredCommands, which is a Setting so it always gets persisted.
- In order to show the number of discovered commands in the application bar button’s icon, several distinct images are included in this project: appbar.1.png, appbar.2.png, appbar.3.png, and so on. The right one is selected based on the Count of the discoveredCommands collection.
- The video starts automatically playing when the page is loaded (because AutoPlay was not set to false in Listing 33.1), but we do not want the entire video to play and reveal all the cat’s actions. Instead, only the first second and a half should play. Therefore, in MediaElement’s MediaOpened event handler (raised when the media has loaded and is ready to start playing), videoTimer is used to pause the video after 1.48 seconds have elapsed. The pausing is done in videoTimer’s Tick event handler, VideoTimer_Tick.
You cannot call Play on a MediaElement until its MediaOpened event is raised!
When a MediaElement’s Source is set (either in XAML or code-behind), you cannot instantly begin interacting with the media. Instead, you must wait for MediaOpened to be raised. If the media cannot be loaded for some reason, a MediaFailed event is raised instead. Subservient Cat avoids the need to manually call Play because it uses the auto-play feature, but if it didn’t use auto-play, then it should call Play inside MediaElement_MediaOpened.
Why doesn’t my video play on a physical phone while it is connected to a computer running Zune?
This is due to the same problem discussed in the preceding chapter.The Zune desktop program locks the media library, which prevents MediaElement from loading its media. Remember, if you need to debug a part of your app that depends on video actually playing, you can use the Windows Phone Connect Tool that ships with the Windows Phone Developer Tools to connect to your phone without Zune running. Subservient Cat detects this situation by handling the MediaFailed event. It assumes the failure is due to the Zune connection, which is a pretty safe assumption because the source video is local to the app.
- The PlayClip method ensures that the video is paused, seeks to the specified beginTime, and reinitializes videoTimer so it pauses the video at endTime. However, because setting MediaElement’s Position has an annoying effect of seeing the video quickly fastforward or rewind to the desired spot (rather than an instant jump), the intro screen is shown to hide the video during the transition. (We don’t want to reveal yet-to-be-discovered parts of the video!) The setting of Position is done inside a BeginInvoke callback so the showing of the intro screen has a chance to take effect. Without this, you still see the unwanted behavior. Two seconds is chosen as the length of the delay, during which the user can read the command text on the intro screen. We don’t have a way to know when seek has actually finished, but two seconds is long enough.
When you set MediaElement’s Position, the change is not instantaneous!
Instead, you see a little bit of the video before or after the new position, as if you’re watching the video in fast-forward or rewind mode.The workaround used by Subservient Cat is to temporarily obscure the video with other elements.
In the current version of Windows Phone, MediaElement does not support markers.The use of markers to designate when individual clips inside cat.wmv begin and end would have been ideal, and would have greatly reduced the amount of code needed in Listing 33.2. However, using a DispatcherTimer to be notified when the relevant clip has ended is a reasonable workaround.There are two things to be aware of, however:
- The timer doesn’t give you frame-by-frame accuracy.The video used by this app has a little bit of a buffer at the end of each clip, in case videoTimer’s Tick event gets raised slightly late.
- If you show a message box, the video will keep playing in the background but the timer’s Tick event handler cannot be called until the message box is dismissed, no matter how much time has elapsed. (MessageBox.Show is a blocking operation.) This is why DiscoveredButton_Click in Listing 33.2 pauses the video first.
When I first wrote Subservient Cat, I called MediaElement’s Stop method inside OnNavigatedFrom because I was worried about the performance impact of unnecessarily playing video while the instructions page is shown and the main page is on the back stack. However, this is unnecessary because MediaElement is automatically paused when a page navigates away. If you don’t want this behavior (perhaps because you want to hear the audio from the video continue while other pages are shown), the MediaElement must be attached to the frame rather than to a specific page.
MediaElement and Files in Isolated Storage
If you want to play a video from isolated storage (presumably because your app downloaded it and then saved it there), you can call MediaElement’s SetSource method that expects a stream rather than a URI. With this, you can pass an appropriate IsolatedStorageFileStream.
The Finished Product
