The Talking Parrot app provides more entertainment with the microphone. After greeting you with a friendly “hello,” the parrot listens to what you say and then repeats it in its own voice, with a whistle or squawk thrown in. This app must not only turn the buffer collected from the microphone into a playable XNA sound effect, but it also must determine when is a good time to listen to the user and when is a good time to play the captured audio.
The Main User Interface
Listing 35.1 contains the XAML for the main page. It consists of three parts: a bunch of animations that are triggered by code-behind, an application bar, and several images (plus one ellipse) placed in specific spots on a canvas. The individual images that form the parrot are shown in Figure 35.1.
LISTING 35.1 MainPage.xaml—The User Interface for Talking Parrot’s Main Page
[code]
<phone:PhoneApplicationPage x:Class=”WindowsPhoneApp.MainPage”
xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
xmlns:shell=”clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone”
SupportedOrientations=”Portrait”>
<!– Animations as named resources –>
<phone:PhoneApplicationPage.Resources>
<Storyboard x:Name=”BlinkStoryboard” Duration=”0:0:4” RepeatBehavior=”Forever”
Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.TranslateY)”>
<DoubleAnimation Storyboard.TargetName=”TopEyelidImage” To=”0”
Duration=”0:0:.1” AutoReverse=”True”/>
<DoubleAnimation Storyboard.TargetName=”BottomEyelidImage” To=”77”
Duration=”0:0:.1” AutoReverse=”True”/>
</Storyboard>
<Storyboard x:Name=”HeadUpStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.TranslateY)”>
<DoubleAnimation Storyboard.TargetName=”HeadCanvas” To=”-7”
Duration=”0:0:1”>
<DoubleAnimation.EasingFunction>
<QuinticEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
</Storyboard>
<Storyboard x:Name=”HeadDownStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.TranslateY)”>
<DoubleAnimation Storyboard.TargetName=”HeadCanvas” To=”0” Duration=”0:0:1”>
<DoubleAnimation.EasingFunction>
<QuadraticEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
</Storyboard>
<Storyboard x:Name=”WingFlutterStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.Rotation)”>
<DoubleAnimation Storyboard.TargetName=”WingImage” To=”35” Duration=”0:0:1”
AutoReverse=”True”>
<DoubleAnimation.EasingFunction>
<BounceEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
</Storyboard>
<Storyboard x:Name=”StompStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.TranslateY)”>
<DoubleAnimation Storyboard.TargetName=”LeftFootImage” To=”-30”
Duration=”0:0:.2” AutoReverse=”True”>
<DoubleAnimation.EasingFunction>
<QuinticEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
<DoubleAnimation Storyboard.TargetName=”RightFootImage” To=”-10”
BeginTime=”0:0:.1” Duration=”0:0:.2” AutoReverse=”True”>
<DoubleAnimation.EasingFunction>
<QuinticEase/>
</DoubleAnimation.EasingFunction>
</DoubleAnimation>
</Storyboard>
<Storyboard x:Name=”SpeakStoryboard” Storyboard.TargetProperty=
“(UIElement.RenderTransform).(CompositeTransform.Rotation)”>
<DoubleAnimationUsingKeyFrames x:Name=”TopBeakAnimation”
Storyboard.TargetName=”TopBeakImage” />
<DoubleAnimationUsingKeyFrames x:Name=”BottomBeakAnimation”
Storyboard.TargetName=”BottomBeakImage” />
</Storyboard>
</phone:PhoneApplicationPage.Resources>
<!– The application bar –>
<phone:PhoneApplicationPage.ApplicationBar>
<shell:ApplicationBar Opacity=”.5”>
<shell:ApplicationBarIconButton Text=”instructions”
IconUri=”/Shared/Images/appbar.instructions.png”
Click=”InstructionsButton_Click”/>
<shell:ApplicationBar.MenuItems>
<shell:ApplicationBarMenuItem Text=”settings”
Click=”SettingsMenuItem_Click”/>
<shell:ApplicationBarMenuItem Text=”about”
Click=”AboutMenuItem_Click”/>
</shell:ApplicationBar.MenuItems>
</shell:ApplicationBar>
</phone:PhoneApplicationPage.ApplicationBar>
<!– Many images placed in a canvas –>
<Canvas>
<Image Source=”Images/background.png” Stretch=”None”/>
<Image x:Name=”RightFootImage” Source=”Images/rightFoot.png”
Canvas.Left=”159” Canvas.Top=”537”>
<Image.RenderTransform>
<CompositeTransform TranslateY=”0”/>
</Image.RenderTransform>
</Image>
<Image x:Name=”LeftFootImage” Source=”Images/leftFoot.png”
Canvas.Left=”109” Canvas.Top=”532”>
<Image.RenderTransform>
<CompositeTransform TranslateY=”0”/>
</Image.RenderTransform>
</Image>
<Image Source=”Images/body.png” Canvas.Left=”-164” Canvas.Top=”346”/>
<Canvas x:Name=”HeadCanvas”>
<Canvas.RenderTransform>
<CompositeTransform TranslateY=”0”/>
</Canvas.RenderTransform>
<Image Source=”Images/head.png” Canvas.Left=”155” Canvas.Top=”203”/>
<Image x:Name=”BottomBeakImage” Source=”Images/bottomBeak.png”
Canvas.Left=”282” Canvas.Top=”294” RenderTransformOrigin=”.5,0”>
<Image.RenderTransform>
<CompositeTransform Rotation=”0” CenterX=”15” CenterY=”15”/>
</Image.RenderTransform>
</Image>
<Image x:Name=”TopBeakImage” Source=”Images/topBeak.png”
Canvas.Left=”279” Canvas.Top=”252”>
<Image.RenderTransform>
<CompositeTransform Rotation=”0” CenterX=”38” CenterY=”38”/>
</Image.RenderTransform>
</Image>
<Image Source=”Images/eyeball.png” Canvas.Left=”198” Canvas.Top=”248”/>
<Ellipse x:Name=”Pupil” Fill=”Black” Width=”18” Height=”18”
Canvas.Left=”240” Canvas.Top=”275”/>
<Image x:Name=”AngryEyelidImage” Source=”Images/eyelid.png”
Visibility=”Collapsed” Canvas.Left=”196” Canvas.Top=”246”>
<Image.RenderTransform>
<CompositeTransform Rotation=”15” CenterX=”76” CenterY=”39”/>
</Image.RenderTransform>
</Image>
<Image x:Name=”TopEyelidImage” Source=”Images/eyelid.png”
Canvas.Left=”196” Canvas.Top=”246”>
<Image.RenderTransform>
<CompositeTransform TranslateY=”-37” Rotation=”0”
CenterX=”76” CenterY=”39”/>
</Image.RenderTransform>
</Image>
<Image x:Name=”BottomEyelidImage” Source=”Images/eyelid.png”
Canvas.Left=”196” Canvas.Top=”246”>
<Image.RenderTransform>
<CompositeTransform ScaleY=”-1” TranslateY=”112”/>
</Image.RenderTransform>
</Image>
</Canvas>
<Image x:Name=”WingImage” Source=”Images/wing.png”
Canvas.Left=”68” Canvas.Top=”414”>
<Image.RenderTransform>
<CompositeTransform Rotation=”0” CenterX=”190” CenterY=”26”/>
</Image.RenderTransform>
</Image>
</Canvas>
</phone:PhoneApplicationPage>
[/code]
- The page is portrait-only due to the exact layout required by the background and pieces of the parrot.
- The animations are given names rather than dictionary keys, so they can be easily referenced from code-behind.
- The animations lower, raise, and rotate various pieces of the parrot when triggered by code-behind. Every animatable piece uses a composite transform for consistency and for ease in animating multiple aspects (such as rotation and translation).
- SpeakStoryboard is special. Although it has two keyframe animations to animate the top and bottom beak, they start out empty. These are continually updated from code-behind based on the audio to be spoken, so the parrot appears to mouth the actual words and sounds it speaks.
- Any of the parrot pieces (as well as the background) could have been created as vector shapes instead of images. Instead, the black pupil is the only vector shape used in the parrot (an ellipse). This gives us the flexibility to grow/shrink the pupil without pixelation and the flexibility to change its color, although this app doesn’t take advantage of these capabilities.
The Main Code-Behind
The code-behind for the main page is shown in Listing 35.2.
LISTING 35.2 MainPage.xaml.cs—The Code-Behind for Talking Parrot’s Main Page
[code]
using System;
using System.IO;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Animation;
using Microsoft.Phone.Controls;
using Microsoft.Xna.Framework.Audio;
namespace WindowsPhoneApp
{
public partial class MainPage : PhoneApplicationPage
{
byte[] buffer;
// Used for capturing audio from the microphone
MemoryStream recordedStream;
long playbackStartPosition = -1;
int consecutiveSilentSamples;
DateTime? speakingDoneTime;
// Used for playing the three included sounds: hello, whistle, and squawk
bool playingIncludedSound;
Random random = new Random();
public MainPage()
{
InitializeComponent();
SoundEffects.Initialize();
CompositionTarget.Rendering += CompositionTarget_Rendering;
// Start blinking, which runs the whole time
this.BlinkStoryboard.Begin();
// Prevent the off-screen tail from being seen when
// animating to the instructions or about pages
this.Clip = new RectangleGeometry {
Rect = new Rect(0, 0, Constants.SCREEN_WIDTH, Constants.SCREEN_HEIGHT) };
// Configure the microphone with the smallest supported BufferDuration (.1)
Microphone.Default.BufferDuration = TimeSpan.FromSeconds(.1);
Microphone.Default.BufferReady += Microphone_BufferReady;
// Initialize the buffer for holding microphone data
int size = Microphone.Default.GetSampleSizeInBytes(
Microphone.Default.BufferDuration);
this.buffer = new byte[size];
// Initialize the stream used to record microphone data
this.recordedStream = new MemoryStream();
// Speak a “hello” greeting
PrepareStoryboardForIncludedSound(SoundEffects.HelloBuffer);
Speak(SoundEffects.Hello, 0);
}
protected override void OnMouseLeftButtonDown(MouseButtonEventArgs e)
{
base.OnMouseLeftButtonDown(e);
// Get mad!
this.AngryEyelidImage.Visibility = Visibility.Visible;
}
protected override void OnMouseLeftButtonUp(MouseButtonEventArgs e)
{
base.OnMouseLeftButtonUp(e);
// Become happy again
this.AngryEyelidImage.Visibility = Visibility.Collapsed;
}
void CompositionTarget_Rendering(object sender, EventArgs e)
{
// Required for XNA Microphone API to work
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
// Check if currently-playing audio has finished
if (this.speakingDoneTime != null && DateTime.Now > this.speakingDoneTime)
{
if (!this.playingIncludedSound)
{
// Don’t restart the microphone yet! We just played audio from the
// microphone, so add either a whistle or squawk to make it sound more
// like a parrot. This will reset speakingDoneTime.
int choice = random.Next(2); // A random number: either 0 or 1
byte[] buffer = (choice == 0 ? SoundEffects.WhistleBuffer :
SoundEffects.SquawkBuffer);
SoundEffect effect = (choice == 0 ? SoundEffects.Whistle :
SoundEffects.Squawk);
PrepareStoryboardForIncludedSound(buffer);
Speak(effect, 0); // Play at the normal speed (0)
}
else
{
// Now it’s time to restart the microphone
Microphone.Default.Start();
// Reset state
this.speakingDoneTime = null;
this.playingIncludedSound = false;
// Smoothly return the head to its resting position
this.HeadDownStoryboard.Begin();
}
}
}
void Speak(SoundEffect effect, float speed)
{
// Stop listening, because it is time to talk
Microphone.Default.Stop();
// Determine when the audio will be done playing and it’s time to either
// add a squawk/whistle or restart the microphone.
// The length is halved for microphone-recorded sounds to avoid extra lag
// time seen in practice.
this.speakingDoneTime = DateTime.Now +
TimeSpan.FromTicks((long)(effect.Duration.Ticks *
(speed == 0 ? 1 : speed / 2)));
// Stop any in-progress storyboards
this.SpeakStoryboard.Stop();
this.WingFlutterStoryboard.Stop();
this.HeadUpStoryboard.Stop();
this.StompStoryboard.Stop();
// Start the storyboards
this.SpeakStoryboard.Begin();
this.WingFlutterStoryboard.Begin();
this.HeadUpStoryboard.Begin();
this.StompStoryboard.Begin();
// Play the audio at full volume with the passed-in speed (pitch)
effect.Play(1, speed, 0);
}
// Changes the contents of TopBeakAnimation and BottomBeakAnimation
// to match the audio in the buffer, so the beak appears to speak the sounds
void PrepareStoryboardForIncludedSound(byte[] buffer)
{
ResetSpeakStoryboard();
// Loop through the buffer in 100-millisecond chunks
for (int i = 0; i < buffer.Length;
i += Constants.INCLUDED_SOUND_BYTES_PER_100_MILLISECONDS)
{
// Cast from short to int to prevent -32768 from overflowing Math.Abs
int currentVolume = Math.Abs((int)BitConverter.ToInt16(buffer, i));
// Add a keyframe to the top & bottom beak animations based on the
// current audio level. ANIMATION_ADJUSTMENT is a fudge factor that
// slightly speeds-up the animation to account for lag.
KeyTime keyTime = TimeSpan.FromSeconds(Math.Max(0,
(double)i / Constants.INCLUDED_SOUND_BYTES_PER_SECOND
– Constants.ANIMATION_ADJUSTMENT));
AddSpeakKeyFrame(currentVolume, keyTime);
}
// Add the final keyframe 100 ms later that smoothly closes the beak
KeyTime finalKeyTime = TimeSpan.FromSeconds(Math.Max(0, (double)
(buffer.Length + Constants.INCLUDED_SOUND_BYTES_PER_100_MILLISECONDS) /
Constants.INCLUDED_SOUND_BYTES_PER_SECOND));
AddFinalSpeakKeyFrame(finalKeyTime);
// The preceding work was computationally expensive, so it’s time for
// another update before attempting to play the sound
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
this.playingIncludedSound = true;
}
// Stop the storyboard and empty the keyframes in its two animations
void ResetSpeakStoryboard()
{
SpeakStoryboard.Stop();
TopBeakAnimation.KeyFrames.Clear();
BottomBeakAnimation.KeyFrames.Clear();
}
// Position the top and bottom beak based on the current volume.
// A louder volume results in a wider opening.
void AddSpeakKeyFrame(int currentVolume, KeyTime keyTime)
{
// The top beak rotation should always be an angle between 0 and -50
TopBeakAnimation.KeyFrames.Add(new DiscreteDoubleKeyFrame {
KeyTime = keyTime, Value = Math.Max(-50, currentVolume / -15) });
// The bottom beak rotation should always be an angle between 0 and 30
BottomBeakAnimation.KeyFrames.Add(new DiscreteDoubleKeyFrame {
KeyTime = keyTime, Value = Math.Min(30, currentVolume / 15) });
}
// Close the beak
void AddFinalSpeakKeyFrame(KeyTime keyTime)
{
// Use keyframes that do a smooth quintic ease from the previous values
TopBeakAnimation.KeyFrames.Add(new EasingDoubleKeyFrame {
EasingFunction = new QuinticEase(), KeyTime = keyTime, Value = 0 });
BottomBeakAnimation.KeyFrames.Add(new EasingDoubleKeyFrame {
EasingFunction = new QuinticEase(), KeyTime = keyTime, Value = 0 });
}
void Microphone_BufferReady(object sender, EventArgs e)
{
int size = Microphone.Default.GetData(this.buffer);
if (size == 0)
return;
// Unconditionally record the audio data by writing it to the stream
this.recordedStream.Write(this.buffer, 0, size);
int currentVolume = SoundEffects.GetAverageVolume(this.buffer, size);
if (currentVolume > Settings.VolumeThreshold.Value)
{
// The current volume is loud enough to be considered talking
this.consecutiveSilentSamples = 0;
if (this.playbackStartPosition == -1)
{
// Start a new phrase.
// Back up half a second if we’ve got the data, for a smoother result.
this.playbackStartPosition = Math.Max(0, this.recordedStream.Position
– Constants.MICROPHONE_BYTES_PER_100_MILLISECONDS * 5);
ResetSpeakStoryboard();
}
// Add a keyframe to the beak animations based on the current volume
// ANIMATION_ADJUSTMENT is a fudge factor that slightly speeds-up the
// animation to account for lag.
KeyTime keyTime = TimeSpan.FromSeconds(Math.Max(0,
Constants.SOUND_SPEED_FACTOR * (this.recordedStream.Position
– Constants.MICROPHONE_BYTES_PER_100_MILLISECONDS
– this.playbackStartPosition) / Constants.MICROPHONE_BYTES_PER_SECOND
– Constants.ANIMATION_ADJUSTMENT));
AddSpeakKeyFrame(currentVolume, keyTime);
}
else
{
// The current volume is NOT loud enough to be considered talking
this.consecutiveSilentSamples++; // 10 times == 1 second
// Check for the end of a spoken phrase. This happens when we’ve got a
// nonnegative playback start position followed by a second (10 samples)
// of silence.
if (this.playbackStartPosition != -1 &&
this.consecutiveSilentSamples == 10)
{
this.consecutiveSilentSamples = 0;
// Add the final keyframe that smoothly closes the beak
KeyTime keyTime = TimeSpan.FromSeconds(Math.Max(0,
Constants.SOUND_SPEED_FACTOR * (this.recordedStream.Position
– Constants.MICROPHONE_BYTES_PER_100_MILLISECONDS
– this.playbackStartPosition) / Constants.MICROPHONE_BYTES_PER_SECOND
– Constants.ANIMATION_ADJUSTMENT));
AddFinalSpeakKeyFrame(keyTime);
// Copy the appropriate slice of audio from the recorded stream into
// a buffer
byte[] buffer = new byte[this.recordedStream.Position –
this.playbackStartPosition];
this.recordedStream.Seek(this.playbackStartPosition, SeekOrigin.Begin);
this.recordedStream.Read(buffer, 0, buffer.Length);
// Amplify the recorded audio, as it tends to be softer than desired
if (Settings.VolumeMultiplier.Value > 1)
SoundEffects.AmplifyAudio(buffer, Settings.VolumeMultiplier.Value);
// Reset variables
this.playbackStartPosition = -1;
this.recordedStream.Position = 0;
// Create a new sound effect from the buffer and speak it
SoundEffect effect = new SoundEffect(buffer,
Microphone.Default.SampleRate, AudioChannels.Mono);
Speak(effect, Constants.SOUND_SPEED_FACTOR);
}
}
}
// Application bar handlers
void InstructionsButton_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/InstructionsPage.xaml”,
UriKind.Relative));
}
void SettingsMenuItem_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(new Uri(“/SettingsPage.xaml”,
UriKind.Relative));
}
void AboutMenuItem_Click(object sender, EventArgs e)
{
this.NavigationService.Navigate(
new Uri(“/Shared/About/AboutPage.xaml?appName=Talking Parrot”,
UriKind.Relative));
}
}
}
[/code]
- The constructor is nearly identical to the preceding chapter’s constructor, although it doesn’t start the microphone right away because the parrot speaks a greeting instead. We never use the microphone while the parrot is speaking because it might hear itself talk and begin an infinite pattern of repeating itself! Other than speaking the greeting, the other additional tasks are initializing the SoundEffects class, shown in Listing 35.3, initializing a stream for recording microphone audio, and starting the blinking animation that continuously moves the parrot’s eyelids for the duration of the app. The microphone is once again given the shortest possible buffer duration, so the app can remain as responsive as possible.
- The handlers for screen taps (OnMouseLeftButtonDown and OnMouseLeftButtonUp) toggle the visibility of an “angry eyelid” to give the parrot an annoyed appearance when touched. This could be changed to handle taps specifically on the parrot’s body, but handling taps anywhere on the screen is simpler and likely good enough.
- CompositionTarget_Rendering, besides calling Update, is responsible for taking action each time the parrot is done speaking. It determines when the parrot is done by comparing the current time to speakingDoneTime, a field set later in the code. If the parrot has just finished speaking audio recorded from the microphone, it makes the parrot speak either a whistle or a squawk (randomly chosen), which resets speakingDoneTime to the later time when the sound effect will finish. If the parrot has just finished speaking one of the included sounds (hello, whistle, or squawk), it restarts the microphone, so it can listen for something new to repeat. As with the preceding chapter, all this code continues to run when navigating forward to a different page. However, this is quite handy for this app because it enables you to aurally test the two settings in real-time as you adjust the sliders on the settings page.
- The Speak method does the important work of stopping the microphone, setting speakingDoneTime, playing the passed-in sound effect at the passed-in speed, and animating the parrot to mouth the words, flutter its wing, raise its head, and stomp its feet. The magic of SpeakStoryboard mouthing the words in the current audio is enabled by manually updating its animation based on the raw audio data that will be played. This is done inside PrepareStoryboardForIncludedSound for the three included sounds and inside Microphone_BufferReady for the audio recorded from the microphone.
- PrepareStoryboardForIncludedSound grabs a sample from the passed-in buffer at every 100 milliseconds (mimicking the behavior of the microphone event) and adds a keyframe to the top and bottom beak animations based on the volume of each sample. The louder the sound, the wider the beak needs to be open. The included sounds have a different bitrate than audio captured from the microphone, so the mapping of time to bytes in the buffer is handled by constants specific to these sounds. A final keyframe is added at the end to handle smoothly closing the beak. Because the three included sounds never change, the animations for each one could have been precalculated (or cached after the first time). However, this dynamic approach is done for simplicity and consistency with the code for sounds recorded from the microphone. It also means you can swap in different sound files, and things will work as expected.
- AddSpeakKeyFrame adds each keyframe as a discrete keyframe (meaning no interpolation between each frame). This is reasonable considering the speed at which the frames advance. It manipulates the value of currentVolume to give it an appropriate range for the angle of rotation for each piece of the beak.
- AddFinalSpeakKeyFrame gives the final keyframe a quintic (power of five) ease from the preceding value to zero, so the beak snaps shut smoothly.
- Microphone_BufferReady uses the same approach as the preceding chapter to determine the volume of the last .1 seconds of audio. If the audio is loud enough and we haven’t started tracking the audio as a phrase to play back, we start paying attention to the audio by marking the starting position in the recorded stream and adding a keyframe to the beak animations (as done previously with the included sounds). We continue to listen to the audio (and add keyframes to the animations) until there has been a full second of silence, which equates to ten consecutive samples where the average volume was below the threshold.
- Because human speech gradually ramps up to a volume above the threshold, simply starting playback at the point where the volume is loud enough would cut off the beginning of whatever was spoken and sound strange. Therefore, the starting position in the recorded stream is backed up half a second from the current point when set inside Microphone_BufferReady. This is why the microphone audio is always appended to the stream, regardless of volume.
- When the end of relevant audio (a second of silence) has been detected inside Microphone_BufferReady, it copies all the data placed into the recorded stream from the chosen starting position onward into a new byte array. Although previous apps have obtained a sound effect from the static SoundEffect.FromStream method, this code—after potentially amplifying the audio—calls a constructor that enables passing in the raw audio data as a byte array. It then plays the dynamic sound effect (along with the appropriate animations) by calling Speak. It chooses a playback speed (pitch) 80% higher than normal (specified by the SOUND_SPEED_FACTOR constant) so it sounds more like a parrot speaking than the original person whose voice was recorded.
- The default volume threshold used by Talking Parrot is lower than the one used by Bubble Blower. Here are the two settings used by this app, with their default values:[code]
public static class Settings
{
public static readonly Setting<int> VolumeThreshold =
new Setting<int>(“VolumeThreshold”, 500);
public static readonly Setting<int> VolumeMultiplier =
new Setting<int>(“VolumeMultiplier”, 4);
}
[/code]
Talking Parrot’s SoundEffects class is similar to the same-named class in previous chapters, but it also exposes the AmplifyAudio and GetAverageVolume methods used by Listing 35.2. Listing 35.3 contains the implementation.
LISTING 35.3 SoundEffects.cs—Exposes the Built-In Sound Effects and Audio Utility Methods
[code]
using System;
using System.IO;
using System.Windows.Resources;
using Microsoft.Xna.Framework.Audio;
namespace WindowsPhoneApp
{
public static class SoundEffects
{
public static SoundEffect Hello { get; private set; }
public static SoundEffect Squawk { get; private set; }
public static SoundEffect Whistle { get; private set; }
public static byte[] HelloBuffer { get; private set; }
public static byte[] SquawkBuffer { get; private set; }
public static byte[] WhistleBuffer { get; private set; }
public static void Initialize()
{
StreamResourceInfo info;
info = App.GetResourceStream(new Uri(“Audio/hello.wav”, UriKind.Relative));
HelloBuffer = GetBytes(info.Stream);
info.Stream.Position = 0;
Hello = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(
new Uri(“Audio/squawk.wav”, UriKind.Relative));
SquawkBuffer = GetBytes(info.Stream);
info.Stream.Position = 0;
Squawk = SoundEffect.FromStream(info.Stream);
info = App.GetResourceStream(
new Uri(“Audio/whistle.wav”, UriKind.Relative));
WhistleBuffer = GetBytes(info.Stream);
info.Stream.Position = 0;
Whistle = SoundEffect.FromStream(info.Stream);
// Required for XNA Microphone API to work
Microsoft.Xna.Framework.FrameworkDispatcher.Update();
}
static byte[] GetBytes(Stream stream)
{
byte[] bytes = new byte[stream.Length];
stream.Read(SquawkBuffer, 0, (int)stream.Length);
return bytes;
}
// Make the sound louder by modifying the raw audio samples
public static void AmplifyAudio(byte[] buffer, int multiplier)
{
// Buffer is an array of bytes, but we want to examine each 2-byte value
for (int i = 0; i < buffer.Length; i += 2)
{
int value = BitConverter.ToInt16(buffer, i);
if (value > Settings.VolumeThreshold.Value)
{
// Only amplify samples that are loud enough to not
// be considered background noise
value *= Settings.VolumeMultiplier.Value;
// Make sure the multiplied value stays within bounds
if (value > short.MaxValue)
value = short.MaxValue;
else if (value < short.MinValue)
value = short.MinValue;
// Replace the two bytes with the amplified value
byte[] newValue = BitConverter.GetBytes(value);
buffer[i] = newValue[0];
buffer[i + 1] = newValue[1];
}
}
}
// Returns the average value among the first numBytes in the buffer
public static int GetAverageVolume(byte[] buffer, int numBytes)
{
long total = 0;
// Buffer is an array of bytes, but we want to examine each 2-byte value
for (int i = 0; i < numBytes; i += 2)
{
// Cast from short to int to prevent -32768 from overflowing Math.Abs:
int value = Math.Abs((int)BitConverter.ToInt16(buffer, i));
total += value;
}
return (int)(total / (numBytes / 2));
}
}
}
[/code]
Unlike in past apps, the raw audio data for each sound file is copied into a byte array exposed as a property. Listing 35.2 used these byte arrays to determine the volume over time, just like what is done for audio from the microphone.
The audio captured from the microphone can often be much softer than desired.To combat this, the AmplifyAudio method in Listing 35.3 increases the volume of the recorded microphone audio by manually multiplying the value of each sample in the buffer (if the sample is louder than a threshold, to avoid amplifying background noise). Although the volume of the played-back audio is ultimately limited by the phone’s volume setting, this technique can make the audio surprisingly loud.Of course, the more that the audio is amplified, the more distorted it may sound.
You can play music from the music library while using this app to make the parrot “sing” the song.You just have to pause the music, so the parrot gets the second of silence needed to prompt it to speak! This can be controlled via the top bar that gets displayed while adjusting the phone’s volume, shown in Figure 35.2.
Here are the constants (and read-only fields) used by this app:
[code]
public static class Constants
{
// Screen
public const int SCREEN_WIDTH = 480;
public const int SCREEN_HEIGHT = 800;
public const float SOUND_SPEED_FACTOR = .8f;
public const float ANIMATION_ADJUSTMENT = .1f;
public static readonly long MICROPHONE_BYTES_PER_SECOND =
Microphone.Default.GetSampleSizeInBytes(TimeSpan.FromSeconds(1));
public static readonly long MICROPHONE_BYTES_PER_100_MILLISECONDS =
Constants.MICROPHONE_BYTES_PER_SECOND / 10;
public const int INCLUDED_SOUND_BYTES_PER_SECOND = 141100;
public const int INCLUDED_SOUND_BYTES_PER_100_MILLISECONDS = 14110;
}
[/code]
The Settings Page
The settings page, shown in Figure 35.3, is like the settings page from the Bubble Blower app, but with two sliders instead of one. The first slider adjusts the “parrot voice volume,” which maps to the VolumeMultiplier setting. The second slider is just like the one from Bubble Blower, which maps to the VolumeThreshold setting. It is labeled as “parrot hearing sensitivity” instead of “microphone sensitivity” to be more appropriate to the theme of this app.
The XAML for Figure 35.3 is shown in Listing 35.4. The differences from Bubble Blower’s settings page are emphasized.
LISTING 35.4 SettingsPage.xaml—The Settings User Interface for Talking Parrot
[code]
<phone:PhoneApplicationPage x:Name=”Page”
x:Class=”WindowsPhoneApp.SettingsPage”
xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
xmlns:phone=”clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone”
xmlns:shell=”clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone”
xmlns:local=”clr-namespace:WindowsPhoneApp”
FontFamily=”{StaticResource PhoneFontFamilyNormal}”
FontSize=”{StaticResource PhoneFontSizeNormal}”
Foreground=”{StaticResource PhoneForegroundBrush}”
SupportedOrientations=”PortraitOrLandscape”
shell:SystemTray.IsVisible=”True”>
<Grid Background=”{StaticResource PhoneBackgroundBrush}”>
<Grid.RowDefinitions>
<RowDefinition Height=”Auto”/>
<RowDefinition Height=”*”/>
</Grid.RowDefinitions>
<!– The standard settings header: –>
<StackPanel Grid.Row=”0” Style=”{StaticResource PhoneTitlePanelStyle}”>
<TextBlock Text=”SETTINGS” Style=”{StaticResource PhoneTextTitle0Style}”/>
<TextBlock Text=”talking parrot”
Style=”{StaticResource PhoneTextTitle1Style}”/>
</StackPanel>
<!– Stacked contents inside a ScrollViewer,
for the benefit of landscape orientation: –>
<ScrollViewer Grid.Row=”1”>
<StackPanel Margin=”{StaticResource PhoneMargin}”>
<TextBlock Text=”Parrot voice volume”
Foreground=”{StaticResource PhoneSubtleBrush}”
Margin=”{StaticResource PhoneMargin}”/>
<Slider x:Name=”VolumeSlider” Minimum=”1” Maximum=”18”
Value=”{Binding Volume, Mode=TwoWay, ElementName=Page}”/>
<TextBlock Text=”Parrot hearing sensitivity”
Foreground=”{StaticResource PhoneSubtleBrush}”
Margin=”{StaticResource PhoneMargin}”/>
<Slider x:Name=”SensitivitySlider” Maximum=”1000” LargeChange=”100”
IsDirectionReversed=”True”
Value=”{Binding Threshold, Mode=TwoWay, ElementName=Page}”/>
<Button Content=”reset” Click=”ResetButton_Click”
local:Tilt.IsEnabled=”True”/>
</StackPanel>
</ScrollViewer>
</Grid>
</phone:PhoneApplicationPage>
[/code]
- Because the page is now a little too tall for the landscape orientations, the StackPanel is wrapped inside a ScrollViewer.
- The allowed range for VolumeMultiplier is 1–18.
- Although SensitivitySlider needs to be reversed to map to the underlying threshold value, VolumeSlider does not.
The code-behind for the settings page is shown in Listing 35.5.
LISTING 35.5 SettingsPage.xaml.cs—The Settings Code-Behind for Talking Parrot
[code]
using System.Windows;
using Microsoft.Phone.Controls;
namespace WindowsPhoneApp
{
public partial class SettingsPage : PhoneApplicationPage
{
public SettingsPage()
{
InitializeComponent();
}
// Simple property bound to the first slider
public int Volume
{
get { return Settings.VolumeMultiplier.Value; }
set { Settings.VolumeMultiplier.Value = value; }
}
// Simple property bound to the second slider
public int Threshold
{
get { return Settings.VolumeThreshold.Value; }
set { Settings.VolumeThreshold.Value = value; }
}
private void ResetButton_Click(object sender, RoutedEventArgs e)
{
this.VolumeSlider.Value = Settings.VolumeMultiplier.DefaultValue;
this.SensitivitySlider.Value = Settings.VolumeThreshold.DefaultValue;
}
}
}
[/code]
The Finished Product
