Audio Loopback Preview

Okay, I pounded through the code and figured out how to structure everything in a modular way. What follows in a description of the code structure, sim objects, and supporting console functions. This should give you an idea of how this will work and how it can be extended.

Loopback Capture Thread:

The loopback capture thread is responsible for capturing data from the loopback device. It will pack the first 2 audio channels into a buffer that is accessible by the sim objects which process the data. Generally the first 2 channels are L and R channels respectively even in surround-sound enabled devices. So it is a safe bet to grab the first 2 channels. This is the only portion of the code that is Windows Vista and later specific. The console functions (TS commands) to control the AudioLoopbackThread C++ object are as follows:

// starts the audio loopback thread
startAudioLoopBack();
// stops the audio loopback thread
stopAudioLoopBack();
// adds a LoopBackObject sim object to be process by the AudioLoopbackThread
addAudioLoopBackObject();
// removes a LoopBackObject sim object to be process by the AudioLoopbackThread
removeAudioLoopBackObject();

Note: LoopBackObjects have a "process" method that is called from the AudioLoopbackThread. This function takes care of performing any copying of data or calculations on the data. The add/remove methods allow the objects to be added to the AudioLoopbackThread or to other objects that source data to be processed. This way the code is written to allow future data sources.

LoopBackObject:

The LoopBackObject is a SimObject derived object. This allows the object to be instantiated from the console just like any other SimObject. The object can be added or removed from the AudioLoopbackThread processing queue at any time. I use SimObjectPtr references to the object. So if the object is deleted the pointer to the object gets updated to NULL. When the processing queue is parsed by AudioLoopbackThread it checks for NULL. If the object is NULL then it is removed from the queue. This will allow the AudioLoopbackThread to be left running without interruption and objects can come and go dynamically to be processed for differing effects. The "process" method on this object calls another method named "process_unique". In LoopBackObject this is an empty virtual function. The intent is for derived objects to overload the "process_unique" function to do additional processing beyond copying the samples from the AudioLoopbackThread. Right now this object may require some adjustments to accommodate other data sources as it assumes a static buffer associated with this class. This buffer is written to by the AudioLoopbackThread. There are no additional console commands for this object.
Update:
I realized my modularity goal had not been met because of the LoopBackObject being hard coded to talk to the AudioLoopbackThread buffer. I modified the object so that it is completely buffer agnostic. When the object is registered with the AudioLoopbackThread pointers to the buffer are set. Now the LoopBackObject is completely independent of the AudioLoopbackThread.

FFTObject:

This object is the Fast Fourier Transform (FFT) object. FFTObject is derived from LoopBackObject. The sample data is processed and put into frequency bins representing the frequency ranges that we have interest in. The following object methods are available in script:

// set the frequency bands we have interest in
setAudioFreqBands();
// get the current frequency bands
getAudioFreqBands();
// get the current FFT frequency magnitude data, this data is smoothed a bit using a low pass filter
getAudioFreqOutput();

Implementation Details:

1. The AudioLoopbackThread captures data from the loopback device.
2. The AudioLoopbackThread calls the "process" function on each LoopBackObject derived object in its queue.
3. The data from the resulting calculations of the FFTObject can be retrieved by console methods.
4. The frequency ranges (bins) can be adjusted per FFTObject.
5. Start/Stopping of the AudioLoopbackThread can occur at any time using console commands.
6. Adding/Removing/Deletion of LoopBackObject derived objects can occur at any time.
7. The FFTObject does a five step process to generate its output: sums stereo data to mono, performs a "Hanning Window" function on the data (this is to help with freq isolation), calculates the FFT, separates the resulting FFT data into frequency bands, and filters the data a bit.

Conclusion:

I learned quite a bit about threading and the available resources within the T3D code base. The built in helper classes and functions made this a fairly straight forward addition to the engine. The hard part was reading through the existing code to see how things should be done. Thankfully there are plenty of good examples in the code base.

I need to go over the code and clean it up (yes, I said that last time) and do some more testing. So stay tuned for a resource containing this code.