Please note: I'm in the process of editing this text (but I've got to go to work first)...
I'm currently working on an audio system for my game XorGramana. To begin with I tried to find a C library which would provide an API for basic audio synthesis and effects processing. After seeking advice from various online communities I settled on using SDL for audio output, and LADSPA plugins for effects processing.
I am using SDL for audio output because I am already using SDL as the graphics system. I'm using LADSPA plugins for effects, firstly because I have already used them in my modular audio synthesis program Wav Composer Not Toilet (and consequently have some code to implement with). And secondly because coding audio effects is an extremely difficult subject when you're not a math's genius.
To begin with I experimented with using SDL_mixer for the audio output, this takes care of mixing sounds for me and involves relatively little work. Despite my very experimental test code sucessfully using SDL_mixer and LADSPA plugins, there are a couple of major problems with that approach.
The first is that SDL_mixer can only mix integer audio data while LADSPA plugins work with floating point data. It was very obviously a bad design to use LADSPA plugins with SDL_mixer where time would be wasted translating the data too often. A better design would require much less data translation.
The second problem arose when applying effects to the stereo audio which SDL_mixer had already mixed (ie post-mix). An SDL_mixer audio stream is a single array where the data for each audio channel (ie the left and right channels) is interleaved. Stereo LADSPA plugins do not handle interleaved audio streams so again, more proof that using SDL_mixer with LADSPA plugins was bad design - again, requiring more translation than a good design should.
To solve these problems, I found David Olofson's SDL Examples and decided to use his 'simple mixer' as a base for my mixer which would be able to handle the LADSPA plugins with less redundant data translation as required with SDL_mixer.
However I soon discovered the 'simple mixer' was not as good a place to begin as I imagined and so I investigated Olofson's DT-42 - Drum Toy - the audio mixer of which was better implemented and also included some very basic FM synthesis.
To begin I ripped the guts of the mixer code out of DT-42 to incorporate into my own. Secondly I removed the FM synthesis code from the mixer as by that stage I had decided against any synthesis beyond effects processing.
Unfortunately the mixer from DT-42 did not work with floating point data either, so my first job was to convert it to use floating point data. This was not so straightforward either and further code was removed to make this easier due to the use of bit-shifting integer data to convert it from 16 bit to 32bit (24 bit in 32bit to give the mix some 'headroom' - floating point audio data is completely different - ie its numerical range far exceeds the values used to represent audio data).
DT-42's mixer code also had a basic tracker-style sequencer. I decided to remove this also, as I had not yet decided if I wanted to use a sequencer at all, as I was considering doing without one.
SDL can read 16bit WAV audio files, so to eliminate converting these during mixing to floating point data, I converted them as soon as they loaded and then removed the integer data from memory (ie free'd them via SDL).
My code was exhibiting plenty of strange behaviour. Some sounds would cut out all others, depending upon when I loaded them. If I loaded the sound-cutting-out sound first, it would be ok, but if I loaded after other sounds it would cut all the others out. There were also pops and clicks after a sound finished playing. After plenty of head-scratching I found several artifacts from the integer data processing of DT-42's mixer which were no longer required for floating point format (ie recalculating the sound's length for different bit-depths/data types).
I've also added the ability to cross-fade one sound over another when one is playing and the second is sent to the same 'voice'. The cross-fade length (ms) is given to the WAV reading method and calculated there.
There is still a lot to do. At the moment it is only playing a single loop with a few drum sounds and some bassy tones, and a blip for player movement. I've still got to implement a method of getting the game code to interact with the audio generating code (seperate from the mixer). I want 8 squares which surround the player to alter the hits in the sequence of the loop, (ie cut out drum hits, maybe add new hits in) aswell as having an effect upon the audio filters that are processing the mix. It sounds quite good so far (to me atleast) but is too repetitive. I want to be able to specify loops for each level - and maybe this will include some simple method of varying how the objects change the sequences - so this will involve creating a sequencer again, though a very simple sequencer.