Other Facilities

The Amazing Audio Engine provides quite a number of utilities and other bits and pieces designed to make writing audio apps easier.

Reading from Audio Files

The AEAudioFileLoaderOperation class provides an easy way to load audio files into memory. All audio formats that are supported by the Core Audio subsystem are supported, and audio is converted automatically into the audio format of your choice.

The class is an NSOperation subclass, which means that it can be run asynchronously using an NSOperationQueue. Alternatively, you can use it in a synchronous fashion by calling start directly:

AEAudioFileLoaderOperation *operation = [[AEAudioFileLoaderOperation alloc] initWithFileURL:url
targetAudioDescription:audioDescription];
[operation start];
if ( operation.error ) {
// Load failed! Clean up, report error, etc.
return;
}
_audio = operation.bufferList;
_lengthInFrames = operation.lengthInFrames;

Note that this class loads the entire audio file into memory, and doesn't support streaming of very large audio files. For that, you will need to use the ExtAudioFile services directly.

Writing to Audio Files

The AEAudioFileWriter class allows you to easily write to any audio file format supported by the system.

To use it, instantiate it using initWithAudioDescription: , passing in the audio format you wish to use. Then, begin the operation by calling beginWritingToFileAtPath:fileType:error: , passing in the path to the file you'd like to record to, and the file type to use. Common file types include kAudioFileAIFFType, kAudioFileWAVEType, kAudioFileM4AType (using AAC audio encoding), and kAudioFileCAFType.

Once the write operation has started, you use the C functions AEAudioFileWriterAddAudio and AEAudioFileWriterAddAudioSynchronously to write audio to the file. Note that you should only use AEAudioFileWriterAddAudio when writing audio from the Core Audio thread, as this is done asynchronously in a way that does not hold up the thread.

When you are finished, call finishWriting to close the file.

Managing Audio Buffers

AudioBufferList is the basic unit of audio for Core Audio, representing a small time interval of audio. This structure contains one or more pointers to an area of memory holding the audio samples: For interleaved audio, there will be one buffer holding the interleaved samples for all channels, while for non-interleaved audio there will be one buffer per channel.

The Amazing Audio Engine provides a number of utility functions for dealing with audio buffer lists:

  • AEAudioBufferListCreate will take an AudioStreamBasicDescription and a number of frames to allocate, and will allocate and initialise an audio buffer list and the corresponding memory buffers appropriately.
  • AEAudioBufferListCreateOnStack creates an AudioBufferList variable on the stack, with the number of buffers set appropriate to the given audio description
  • AEAudioBufferListCopy will copy an existing audio buffer list into a new one, allocating memory as needed.
  • AEAudioBufferListFree will free the memory pointed to by an audio buffer list, and the buffer list itself.
  • AEAudioBufferListCopyOnStack will make a copy of an existing buffer on the stack (without any memory allocation), and optionally offset its mData pointers; useful for doing offset buffer fills with utilities that write to AudioBufferLists.
  • AEAudioBufferListGetLength will take an AudioStreamBasicDescription and return the number of frames contained within the audio buffer list given the mDataByteSize values within.
  • AEAudioBufferListSetLength sets a buffer list's mDataByteSize values to correspond to the given number of frames.
  • AEAudioBufferListOffset increments a buffer list's mData pointers by the given number of frames, and decrements the mDataByteSize values accordingly.
  • AEAudioBufferListSilence clears the values in a buffer list (sets them to zero).
  • AEAudioBufferListGetStructSize returns the size of an AudioBufferList structure, for use when memcpy-ing buffer list structures.

Note: Do not use those functions above that perform memory allocation or deallocation from within the Core Audio thread, as this may cause performance problems.

Additionally, the AEAudioBufferManager class lets you perform standard ARC/retain-release memory management with AudioBufferLists.

Defining Audio Formats

Core Audio uses the AudioStreamBasicDescription type for describing kinds of audio samples. The Amazing Audio Engine provides a number of utilities for working with these types:

Improving Efficiency using Vector Operations

Vector operations offer orders of magnitude improvements in processing efficiency over performing the same operation as a large number of scalar operations.

For example, take the following code which calculates the absolute maximum value within an audio buffer:

float max = 0;
for ( int i=0; i<frames; i++ ) {
float value = fabs(((float*)audio->mBuffers[0].mData)[i]);
if ( value > max ) max = value;
}

This consists of frames address calculations, followed by frames calls to fabs, frames floating-point comparisons, and at worst case, frames assignments, followed by frames integer increments.

This can be replaced by a single vector operation, using the Accelerate framework:

float max = 0;
vDSP_maxmgv((float*)audio->mBuffers[0].mData, 1, &max, frames);

For those working with floating-point audio, this already works, but for those working in other audio formats, an extra conversion to floating-point is required.

If you are using only non-interleaved 16-bit signed integers, then this can be performed easily, using vDSP_vflt16. Otherwise, The Amazing Audio Engine provides the AEFloatConverter class to perform this operation easily with any audio format:

static const int kScratchBufferSize[4096];
AudioBufferList *scratchBufferList
= AEAudioBufferListCreate(AEAudioStreamBasicDescriptionNonInterleavedFloatStereo, kScratchBufferSize);
...
self.floatConverter = [[AEFloatConverter alloc] initWithSourceFormat:_audioController.audioDescription];
...
AEFloatConverterToFloatBufferList(THIS->_floatConverter, audio, THIS->_scratchBufferList, frames);
// Now process the floating-point audio in 'scratchBufferList'.

Thread Synchronization

Thread synchronization is notoriously difficult at the best of times, but when the timing constraints introduced by the Core Audio realtime thread are taken into account, this becomes a very tricky problem indeed.

A common solution is the use of mutexes with try-locks, so that rather than blocking on a lock, the Core Audio thread will simply fail to acquire the lock, and will abort the operation. This can work, but always runs the risk of creating audio artefacts when it stops generating audio for a time interval, which is precisely the problem that we are trying to avoid by not blocking.

All this can be avoided with The Amazing Audio Engine's messaging feature.

This utility allows the main thread to send messages to the Core Audio thread, and vice versa, without any locking required.

To send a message to the Core Audio thread, use either performAsynchronousMessageExchangeWithBlock:responseBlock: , or performSynchronousMessageExchangeWithBlock: :

[_audioController performAsynchronousMessageExchangeWithBlock:^{
// Do something on the Core Audio thread
} responseBlock:^{
// The thing above has been done, and now we're back on the main thread
}];
[_audioController performSynchronousMessageExchangeWithBlock:^{
// Do something on the Core Audio thread.
// We will block on the main thread until this has been completed
}];
// Now the Core Audio thread finished doing whatever we asked it do, and we're back.

To send messages from the Core Audio thread back to the main thread, you need to define a C callback, which takes the form defined by AEMessageQueueMessageHandler, then call AEAudioControllerSendAsynchronousMessageToMainThread , passing a reference to any parameters, with the length of the parameters in bytes.

struct _myHandler_arg_t { int arg1; int arg2; };
static void myHandler(void *userInfo, int userInfoLength) {
struct _myHandler_arg_t *arg = (struct _myHandler_arg_t*)userInfo;
NSLog(@"On main thread; args are %d and %d", arg->arg1, arg->arg2);
}
...
// From Core Audio thread
AEAudioControllerSendAsynchronousMessageToMainThread(THIS->_audioController,
myHandler,
&(struct _myHandler_arg_t) {
.arg1 = 1,
.arg2 = 2 },
sizeof(struct _myHandler_arg_t));

Whatever is passed via the 'userInfo' parameter of AEAudioControllerSendAsynchronousMessageToMainThread will be copied onto an internal buffer. A pointer to the copied item on the internal buffer will be passed to the callback you provide.

Note: This is an important distinction. The bytes pointed to by the 'userInfo' parameter value are passed by value, not by reference. To pass a pointer to an instance of an Objective-C class, you need to pass the address to the pointer to copy using the "&" operator.

This:

AEAudioControllerSendAsynchronousMessageToMainThread(THIS->_audioController,
myHandler,
&object,
sizeof(id) },

Not this:

AEAudioControllerSendAsynchronousMessageToMainThread(THIS->_audioController,
myHandler,
object,
sizeof(id) },

To access an Objective-C object pointer from the main thread handler function, you can bridge a dereferenced void** to your object type, like this:

MyObject *object = (__bridge MyObject*)*(void**)userInfo;

Receiving Time Cues

For certain applications, it's important that events take place at a precise time. NSTimer and the NSRunLoop scheduling methods simply can't do the job when it comes to millisecond-accurate timing, which is why The Amazing Audio Engine provides support for receiving time cues.

Audio receivers, channels and filters all receive and can act on audio timestamps, but there are some cases where it makes more sense to have a separate class handle the timing and synchronization.

In that case, you can implement the AEAudioTimingReceiver protocol and add your class as a timing receiver via addTimingReceiver:. The callback you provide will be called from two contexts: When input is received (AEAudioTimingContextInput), and when output is about to be generated (AEAudioTimingContextOutput). In both cases, the timing receivers will be notified before any of the audio receivers or channels are invoked, so that you can set app state that will affect the current time interval.

Scheduling Events

AEBlockScheduler is a class you can use to schedule blocks for execution at a particular time. This implements the AEAudioTimingReceiver protocol, and provides an interface for scheduling blocks with sample-level accuracy.

To use it, instantiate AEBlockScheduler, add it as a timing receiver with addTimingReceiver:, then begin scheduling events using scheduleBlock:atTime:timingContext:identifier: :

self.scheduler = [[AEBlockScheduler alloc] initWithAudioController:_audioController];
[_audioController addTimingReceiver:_scheduler];
...
[_scheduler scheduleBlock:^(const AudioTimeStamp *time, UInt32 offset) {
// We are now on the Core Audio thread at *time*, which is *offset* frames
// before the time we scheduled, *timestamp*.
}
atTime:timestamp
timingContext:AEAudioTimingContextOutput
identifier:@"my event"];

The block will be passed the current time, and the number of frames offset between the current time and the scheduled time.

The alternate scheduling method, scheduleBlock:atTime:timingContext:identifier:mainThreadResponseBlock: , allows you to provide a block that will be called on the main thread after the schedule has completed.

There are a number of utilities you can use to construct and calculate timestamps, including now, timestampWithSecondsFromNow:, hostTicksFromSeconds: and secondsFromHostTicks:.