Musical Robot is a quirky musical instrument app that can play two-octaves-worth of robotic sounds based on where you place your fingers. Touching toward the left produces lower notes, and touching toward the right produces higher notes. You can slide your fingers around to produce interesting effects. You can use multiple fingers— as many as your phone supports simultaneously—to play chords (multiple notes at once). You’re more likely to use this app to annoy your friends rather than play actual compositions, but it’s fun nevertheless!
The User Interface
Musical Robot’s main page, pictured in Figure 38.1 in its initial state, contains a few visual elements that have nothing to do with the core functionality of this app, but provide some visual flair and simple instructions. Listing 38.1 contains the XAML.
LISTING 38.1 MainPage.xaml—The User Interface for Musical Robot’s Main Page
[code]
<phone:PhoneApplicationPage x:Class=”WindowsPhoneApp.MainPage”
xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
SupportedOrientations=”Landscape” Orientation=”Landscape”>
<Canvas>
<!– The dynamic mouth that is visible through the image’s mouth hole –>
<Rectangle Canvas.Left=”168” Canvas.Top=”127” Width=”114” Height=”23”
RadiusX=”10” RadiusY=”10” RenderTransformOrigin=”.5,.5”
Fill=”{StaticResource PhoneForegroundBrush}”>
<Rectangle.RenderTransform>
<!– The scale is continually changed from code-behind –>
<ScaleTransform x:Name=”MouthScale” ScaleX=”0”/>
</Rectangle.RenderTransform>
</Rectangle>
<!– 5 lights representing up to 5 simultaneous fingers –>
<Ellipse x:Name=”Light1” Visibility=”Collapsed” Canvas.Left=”137”
Canvas.Top=”284” Width=”23” Height=”23” Fill=”Red”/>
<Ellipse x:Name=”Light2” Visibility=”Collapsed” Canvas.Left=”174”
Canvas.Top=”294” Width=”23” Height=”23” Fill=”Red”/>
<Ellipse x:Name=”Light3” Visibility=”Collapsed” Canvas.Left=”213”
Canvas.Top=”298” Width=”23” Height=”23” Fill=”Red”/>
<Ellipse x:Name=”Light4” Visibility=”Collapsed” Canvas.Left=”252”
Canvas.Top=”294” Width=”23” Height=”23” Fill=”Red”/>
<Ellipse x:Name=”Light5” Visibility=”Collapsed” Canvas.Left=”290”
Canvas.Top=”284” Width=”23” Height=”23” Fill=”Red”/>
<!– The accent-colored robot –>
<Rectangle Width=”453” Height=”480” Fill=”{StaticResource PhoneAccentBrush}”>
<Rectangle.OpacityMask>
<ImageBrush ImageSource=”Images/robot.png”/>
</Rectangle.OpacityMask>
</Rectangle>
<!– Instructions –>
<TextBlock Canvas.Left=”350” Canvas.Top=”40” FontFamily=”Segoe WP Black”
FontSize=”40” Foreground=”{StaticResource PhoneAccentBrush}”>
<TextBlock.RenderTransform>
<RotateTransform Angle=”-10”/>
</TextBlock.RenderTransform>
TAP & DRAG.
<LineBreak/>
USE MANY FINGERS!
</TextBlock>
</Canvas>
</phone:PhoneApplicationPage>
[/code]
- MouthScale’s ScaleX value is randomly set anywhere from 0 to 1 whenever a finger makes contact with or moves across the screen. This provides the illusion that the robot is singing as it makes its noises.
- The circles are filled with red to indicate how many fingers are simultaneously in contact with the screen (up to 5). The limit of 5 is simply due to space constraints in the artwork. It does not reflect on multi-touch limitations of the operating system or any particular device.
How many simultaneous touch points does Windows Phone support?
All Windows phones are guaranteed to support at least four simultaneous touch points. (Current models support exactly four.) The operating system can support up to 10, in case an ambitious device wants to support it.
The Code-Behind
Listing 38.2 contains the code-behind for the main page.
LISTING 38.2 MainPage.xaml.cs—The Code-Behind for Musical Robot’s Main Page
[code]
using System;
using System.Collections.Generic;
using System.Linq;
using System.Windows;
using System.Windows.Input;
using System.Windows.Navigation;
using Microsoft.Phone.Controls;
using Microsoft.Xna.Framework.Audio;
namespace WindowsPhoneApp
{
public partial class MainPage : PhoneApplicationPage
{
// Store a separate sound effect instance for each unique finger
Dictionary<int, SoundEffectInstance> fingerSounds =
new Dictionary<int, SoundEffectInstance>();
// For the random mouth movement
Random random = new Random();
public MainPage()
{
InitializeComponent();
SoundEffects.Initialize();
}
protected override void OnNavigatedTo(NavigationEventArgs e)
{
base.OnNavigatedTo(e);
// Subscribe to the touch/multi-touch event.
// This is application-wide, so only do this when on this page.
Touch.FrameReported += Touch_FrameReported;
}
protected override void OnNavigatedFrom(NavigationEventArgs e)
{
base.OnNavigatedFrom(e);
// Unsubscribe from this application-wide event
Touch.FrameReported -= Touch_FrameReported;
}
void Touch_FrameReported(object sender, TouchFrameEventArgs e)
{
// Get all touch points
TouchPointCollection points = e.GetTouchPoints(this);
// Filter out the “up” touch points because those fingers are
// no longer in contact with the screen
int numPoints =
(from p in points where p.Action != TouchAction.Up select p).Count();
// Update up to 5 robot lights to indicate how many fingers are in contact
this.Light1.Visibility =
(numPoints >= 1 ? Visibility.Visible : Visibility.Collapsed);
this.Light2.Visibility =
(numPoints >= 2 ? Visibility.Visible : Visibility.Collapsed);
this.Light3.Visibility =
(numPoints >= 3 ? Visibility.Visible : Visibility.Collapsed);
this.Light4.Visibility =
(numPoints >= 4 ? Visibility.Visible : Visibility.Collapsed);
this.Light5.Visibility =
(numPoints >= 5 ? Visibility.Visible : Visibility.Collapsed);
// If any fingers are in contact, stretch the inner mouth anywhere from
// 0 to 100%
if (numPoints == 0)
this.MouthScale.ScaleX = 0;
else
this.MouthScale.ScaleX = this.random.NextDouble(); // Returns a # from 0-1
// Process each touch point individually
foreach (TouchPoint point in points)
{
// The “touch device” is each finger, and it has a unique ID
int fingerId = point.TouchDevice.Id;
if (point.Action == TouchAction.Up)
{
// Stop the sound corresponding to this just-lifted finger
if (this.fingerSounds.ContainsKey(fingerId))
this.fingerSounds[fingerId].Stop();
// Remove the sound from the dictionary
this.fingerSounds.Remove(fingerId);
}
else
{
// Turn the horizontal position into a pitch from -1 to 1.
// -1 represents 1 octave lower, 1 represents 1 octave higher.
float pitch = (float)(2 * point.Position.X / this.ActualWidth) – 1;
if (!this.fingerSounds.ContainsKey(fingerId))
{
// We haven’t yet created the sound effect for this finger, so do it
this.fingerSounds.Add(fingerId, SoundEffects.Sound.CreateInstance());
this.fingerSounds[fingerId].IsLooped = true;
}
// Start playing the looped sound at the correct pitch
this.fingerSounds[fingerId].Pitch = pitch;
this.fingerSounds[fingerId].Play();
}
}
// Work around the fact that we sometimes don’t get Up actions reported
CheckForStuckSounds(points);
}
void CheckForStuckSounds(TouchPointCollection points)
{
List<int> soundsToRemove = new List<int>();
// Inspect each active sound
foreach (var sound in this.fingerSounds)
{
bool found = false;
// See if this sound corresponds to an active finger
foreach (TouchPoint point in points)
{
if (point.TouchDevice.Id == sound.Key)
{
found = true;
break;
}
}
// It doesn’t, so stop the sound and mark it for removal
if (!found)
{
sound.Value.Stop();
soundsToRemove.Add(sound.Key);
}
}
// Remove each orphaned sound
foreach (int id in soundsToRemove)
this.fingerSounds.Remove(id);
}
}
}
[/code]
- Because this app contains only a single page, unsubscribing from the FrameReported event in OnNavigatedFrom is not necessary, but this is done to keep good hygiene in case another page is ever added (or this code is used in a different app).
- Inside the FrameReported event handler (Touch_FrameReported), GetTouchPoints is called to get the entire collection of touch points. On most devices, this collection will always contain 1–4 items. It never is zero-length or null.
- Because each touch point can be in the returned collection for one of three reasons (making initial contact with the screen, moving on the screen, or being released from the screen), simply counting its items does not tell us how many fingers are currently in contact with the screen. To do this, we must filter out any touch points whose Action property is set to Up. This could be done by manually enumerating the points collection, but this code instead opts for a LINQ query to set the value of numPoints.
- The code responsible for starting and stopping each sound makes use of a property on TouchPoint not mentioned in the preceding chapter: TouchDevice. This oddsounding property represents the user’s finger responsible for the touch point. Each finger is assigned an integer ID, exposed as a property on TouchDevice, which can be used to track each finger individually. This would otherwise be impossible when multiple fingers are triggering events simultaneously. The ID assigned to any finger is guaranteed to remain unique during the lifetime of a down/move/up action cycle.
- The fingerSounds dictionary leverages the unique finger IDs for its keys. This listing starts playing a looped sound as each new finger makes contact with the screen, it adjusts the pitch of the sound as the finger is moved, and it stops the sound as the finger is released.
- The SoundEffects class used by this listing is just like the same-named class used by many of this book’s apps, but customized to expose a single sound through its Sound property. The included sound is so short that it is barely audible when played by itself, but the IsLooped property on SoundEffectInstance is leveraged to produce a smooth and very audible sound. The pitch of each sound is varied based on the horizontal position of each touch point.
Occasionally, a finger-up action might not be reported!
Due to a bug in Silverlight (or perhaps in some touch drivers), a finger that has reported touching down and moving around might never report that it has been lifted up. Instead, the corresponding touch point simply vanishes from the collection returned by GetTouchPoints. In Musical Robot, this would manifest as sounds that never stop playing.
To prevent such “stuck” sounds, the CheckForStuckSounds method in Listing 38.2 looks at every active sound and attempts to find a current touch point that corresponds to it. If a sound is not associated with an active touch point, it is stopped and removed from the dictionary, just like what happens when a finger properly reports being lifted up.
Can a finger ID continue to identity a specific finger even if it temporarily leaves the screen?
No, the phone cannot continue to track a specific finger once it has broken contact with the screen.The unique ID, assigned during each new initial contact, is only valid until the FrameReported event reports an Up action for that touch point.
The Finished Product
