Capturing the Audio Stream from the Microphone Array

Article
2013-09-12

Kinect for Windows 1.5, 1.6, 1.7, 1.8

Capture the Audio Stream

The capture process works as follows:

The primary thread creates a worker thread to capture the audio data, and then waits for user input to stop the capture process.
The worker thread captures audio data in the background, resamples it, and writes it to the .wav file.
When the user inputs the stop command (the s character), the primary thread notifies the worker thread to stop capturing data. The primary thread then waits for any other user input to terminate the program.

The Primary Thread

When the capture object has been initialized, as shown by the if block that was shown at the beginning of the Preparing for Audio Capture topic, the primary thread calls the CaptureAudio method. The following code snippet shows the CaptureAudio method:

HRESULT CaptureAudio(CWASAPICapture *capturer, HANDLE waveFile, const wchar_t *waveFileName)
{
   hr = WriteWaveHeader(waveFile, capturer->GetOutputFormat(), 0);
   ...
   if (capturer->Start(waveFile))
   {
        printf_s("Capturing audio data to file %S\nPress 's' to stop capturing.\n", waveFileName);

        do
        {
           ch = _getwch();
        } while (L'S' != towupper(ch));

        printf_s("\n");

        capturer->Stop();

        // Fix up the wave file header to reflect the right amount of captured data.
        SetFilePointer(waveFile, 0, NULL, FILE_BEGIN);
        hr = WriteWaveHeader(waveFile, capturer->GetOutputFormat(), capturer->BytesCaptured());
    }
    return hr;
}

The method writes the header for the .wav file, starts the capturer, and waits for the user to enter the s character to stop the capture process. When the capture process is stopped, the .wav file header must be modified to correctly reflect the total number of bytes written to the .wav file.

Start the Capture Process

CaptureAudio calls the CWASAPICapture:Start method to start the capture process:

bool CWASAPICapture::Start(HANDLE waveFile)
{
    HRESULT hr;

    _BytesCaptured = 0;
    _CaptureFile = waveFile;

    //
    //  Now create the thread that is going to drive the capture.
    //
    _CaptureThread = CreateThread(NULL, 0, WASAPICaptureThread, this, 0, NULL);


    //
    //  We're ready to go, start capturing!
    //
    hr = _AudioClient->Start();
    
}

In this method:

The CreateThread call associates the new thread with the CWASAPICapture:WASAPICaptureThread method, which we will examine below.
The IAudioClient:Start call directs the audio client to start streaming data between the endpoint buffer and the audio engine.

Manage the Capture Process

After CWASAPICapture:Start returns, wmain waits for the user to input the stop command. When the user does enter this, wmain calls CWASAPICapture:Stop:

void CWASAPICapture::Stop()
{
    HRESULT hr;

    //
    //  Tell the capture thread to shut down, wait for the thread to complete then clean up all the stuff we 
    //  allocated in Start().
    //
    if (NULL != _ShutdownEvent)
    {
        SetEvent(_ShutdownEvent);
    }

    hr = _AudioClient->Stop();


    if (NULL != _CaptureThread)
    {
        WaitForSingleObject(_CaptureThread, INFINITE);

        CloseHandle(_CaptureThread);
        _CaptureThread = NULL;
    }
}

This method:

Raises _ShutdownEvent to notify the worker thread to stop capturing data.
Calls IAudioClient:Stop to direct the audio engine to stop streaming data.
Waits for the worker thread to signal the thread object, which indicates that the capture process is complete.
Terminates the thread.

wmain then writes the total number of bytes captured to the .wav file header, performs final cleanup, and waits for the user to hit any key to terminate the application.

WASAPICaptureThread

On the worker thread, WASAPICaptureThread calls CWASAPICapture:WASAPIDoCapture to handle the capture process:

DWORD CWASAPICapture::DoCaptureThread()
{
   bool stillPlaying = true;
   HANDLE mmcssHandle = NULL;
   DWORD mmcssTaskIndex = 0;

   HRESULT hr = CoInitializeEx(NULL, COINIT_MULTITHREADED);

   mmcssHandle = AvSetMmThreadCharacteristics(L"Audio", &mmcssTaskIndex);

   while (stillPlaying)
   {
      HRESULT hr;
      //
      //  We want to wait for half the desired latency in milliseconds.
      //
      //  That way we'll wake up half way through the processing period to pull the
      //  next set of samples from the engine.
      //
      DWORD waitResult = WaitForSingleObject(_ShutdownEvent, _EngineLatencyInMS / 2);
      switch (waitResult)
      {
          case WAIT_OBJECT_0 + 0:
          // If _ShutdownEvent has been set, we're done and should exit the main capture loop.
          stillPlaying = false;
          break;

          case WAIT_TIMEOUT:

          //  We need to retrieve the next buffer of samples from the audio capturer.
          BYTE *pData;
          UINT32 framesAvailable;
          DWORD  flags;
          bool isEmpty = false;

          // Keep fetching audio in a tight loop as long as audio device still has data.
          while (!isEmpty && (WAIT_OBJECT_0 != WaitForSingleObject(_ShutdownEvent, 0)))
          {
            hr = _CaptureClient->GetBuffer(&pData, &framesAvailable, &flags, NULL, NULL);
            if (SUCCEEDED(hr))
            {
              if ( (AUDCLNT_S_BUFFER_EMPTY == hr) || (0 == framesAvailable) )
              {
                isEmpty = true;
              }
              else
              {
                DWORD bytesAvailable = framesAvailable * _MixFrameSize;

                // Process input to resampler
                hr = ProcessResamplerInput(pData, bytesAvailable, flags);
                if (SUCCEEDED(hr))
                {
                  DWORD bytesWritten;

                  // Process output from resampler
                  hr = ProcessResamplerOutput(&bytesWritten);
                  if (SUCCEEDED(hr))
                  {
                    //  Audio capture was successful, so bump the capture buffer pointer.
                    _BytesCaptured += bytesWritten;
                  }
                }
              }

              hr = _CaptureClient->ReleaseBuffer(framesAvailable);

          }
       }

       break;
    }
  }

  AvRevertMmThreadCharacteristics(mmcssHandle);

  CoUninitialize();
  return 0;
}

The AvSetMmThreadCharacteristics function associates the worker thread with the capture task.

For each iteration, the capture loop waits until the next frame's data has been streamed.

If the primary thread raises _ShutdownEvent before the time-out ends, the capture loop terminates.
If the primary thread does not raise _ShutdownEvent, the capture loop fills the capture buffer and issues the resampling calls.
When the resampling is complete for the current data, the resampled data is written to the .wav file and the number of current collected bytes is updated.

Share via