The audio system in Torque 2D
by Mike Lilligreen · in Torque 2D Professional · 02/21/2013 (7:08 am) · 6 replies
I am putting together an audio oriented toy, showing functions like alxPlay, alxPause, and alxStop. The very basic stuff for background music and single shot sound effects. No problems there. While working on this, I wanted to take the opportunity to get a deeper understanding of the audio system.
...
I think I blew a fuse in my brain trying to figure this stuff out. alxSourcef? alxSource3f? alxListeneri? ALEnum? I can't remember there ever being documentation or examples from previous versions of the engine, so I started by checking out the OpenAL documentation. A short summary from Wikipedia:
In general, I can't help but feel like the current audio system has lived well beyond it's expired by date. It would be nice to have an audio system that is a bit more intuitive and hooks into the scene/scene object systems. Examples of things I am thinking about:
Left/Right Sound Panning
Have the scene/scene window/canvas be the default listener. Add a plane sprite into the scene - moving the plane around the X axis changes the amount of energy going to each speaker depending on where the plane is in the scene. An easy setup in TorqueScript would be like this:
Distance Attenuation
A simple example would be having a player sprite walk towards a campfire. The closer the player gets to the fire, the louder the sound of burning wood becomes. Listener setup:
Source setup:
It's probably possible to do all this stuff now since OpenAL is built for it, but these examples are more to show some ideas on making the scripting of sound playing more user friendly. I see that Torque 3D has a different audio system, SFX. Is it worth putting in a feature request to integrate that system into Torque 2D?
...
I think I blew a fuse in my brain trying to figure this stuff out. alxSourcef? alxSource3f? alxListeneri? ALEnum? I can't remember there ever being documentation or examples from previous versions of the engine, so I started by checking out the OpenAL documentation. A short summary from Wikipedia:
Quote:The general functionality of OpenAL is encoded in source objects, audio buffers and a single listener. A source object contains a pointer to a buffer, the velocity, position and direction of the sound, and the intensity of the sound. The listener object contains the velocity, position and direction of the listener, and the general gain applied to all sound. Buffers contain audio data in PCM format, either 8- or 16-bit, in either monaural or stereo format. The rendering engine performs all necessary calculations as far as distance attenuation, Doppler effect, etc.Ok, so far so good. This gives me a rough idea how the system is setup and the general purpose behind functions like alxCreateSource and alxGetListenerf. How all this stuff works in Torque is still a bit of a mystery though.
In general, I can't help but feel like the current audio system has lived well beyond it's expired by date. It would be nice to have an audio system that is a bit more intuitive and hooks into the scene/scene object systems. Examples of things I am thinking about:
Left/Right Sound Panning
Have the scene/scene window/canvas be the default listener. Add a plane sprite into the scene - moving the plane around the X axis changes the amount of energy going to each speaker depending on where the plane is in the scene. An easy setup in TorqueScript would be like this:
%plane.setSoundSource(ToyAssets:jetEngineSound); %plane.setSoundPanning(true); // false being default
Distance Attenuation
A simple example would be having a player sprite walk towards a campfire. The closer the player gets to the fire, the louder the sound of burning wood becomes. Listener setup:
%player.setSoundListener(true); //This would move the listener from the default scene to a specific object
Source setup:
%campfire.setSoundSource(ToyAssets:burningWoodSound); %campfire.setSoundLooping(true); %campfire.setSoundRadius(10); // Distance in meters to go from zero to full volume
It's probably possible to do all this stuff now since OpenAL is built for it, but these examples are more to show some ideas on making the scripting of sound playing more user friendly. I see that Torque 3D has a different audio system, SFX. Is it worth putting in a feature request to integrate that system into Torque 2D?
About the author
#2
02/21/2013 (12:25 pm)
Thanks for the clarification Richard. I had a look at the audio asset and see methods like setAudioFile, setVolume, setLooping, etc, are available. I guess the point I am trying to get across is that an ImageAssset, for example, has a bunch of basic properties like cell width, then when that ImageAsset is assigned to a Sprite, I get access to a whole bunch of methods from Sprite/SpriteBase/SceneObject to manipulate that asset dynamically in script with. It would be nice to have a small set of features to manipulate AudioAssets with, to easily make stuff like campfire radii without having to clutter a scene up with a bunch of helper objects like sensors or triggers to get the same effect. 
#3
To do what you're describing with Audio Assets you can simply make a set of asset files that use the same sound file but have different properties. You would still use something like a scene object to provide positional data - though I suppose you could derive an AudioEmitter object from SceneObject for the purpose. Effectively it would be an "audio sprite." Still, putting the radius in the asset makes adjusting large numbers of these much easier - every "campfire" will use the same properties, but you sure don't want to sift through dozens of objects in dozens of level files to change them. You could then do something like:
Of course, no AudioEmitter is currently implemented, but it wouldn't be very hard to do as part of the sort of upgrade you're proposing.
The main thing to understand here is that an Asset is used by the system in a way that is reminiscent of the Datablock object that Torque engines have always used - it is only a collection of properties, not a representation of a game object. These properties need to be associated with a game object to become useful - as you pointed out.
02/21/2013 (12:40 pm)
The set_() methods are the engine-side interface that the asset fields use to set these properties when you do something like %sound.AudioFile = "./mysoundfile.wav".To do what you're describing with Audio Assets you can simply make a set of asset files that use the same sound file but have different properties. You would still use something like a scene object to provide positional data - though I suppose you could derive an AudioEmitter object from SceneObject for the purpose. Effectively it would be an "audio sprite." Still, putting the radius in the asset makes adjusting large numbers of these much easier - every "campfire" will use the same properties, but you sure don't want to sift through dozens of objects in dozens of level files to change them. You could then do something like:
%soundEmitter = new AudioEmitter(); %soundEmitter.Position = "10 2"; // place the emitter in the world %soundEmitter.AudioAsset = "Assets:burningWoodSound"; // assign the sound asset %soundEmitter.Play(); // play the sound at the position
Of course, no AudioEmitter is currently implemented, but it wouldn't be very hard to do as part of the sort of upgrade you're proposing.
The main thing to understand here is that an Asset is used by the system in a way that is reminiscent of the Datablock object that Torque engines have always used - it is only a collection of properties, not a representation of a game object. These properties need to be associated with a game object to become useful - as you pointed out.
#4
02/21/2013 (1:17 pm)
Excellent, you have a much better programmer's vocabulary than I do, so you could summarize things better. It was the fact that audio properties are tucked away inside of assets (and were tucked away inside of datablocks previously) that was bugging me and there is simply a lack of a game object (AudioEmitter) to use in a scene.
#5
02/21/2013 (9:41 pm)
Would it be fair to say then, that the current audio system is event driven? Something happens (key press, collision occurs), a function is called, and within that function is an alxPlay to fire off a sound or change the music. To have things like distance attenuation or Doppler effects an object driven system is needed to give an audio source/listener position and/or velocity within the scene. Such a system should simply be in addition to the event one.
#6
The only thing that is holding the system back from using the features of OpenAL that you're wanting is that they are not implemented in the engine at the moment. Technically, when the campfire AudioEmitter that we have discussed is loaded it would immediately begin playing its associated sound, which would loop. Because we have discussed attaching the listener to the player object and enabling the use of OpenAL's 3D sound capabilities the player object will "hear" the sound when it is within range without relying on some sort of callback or other aid - though technically the "event" of loading is what started the sound playing in the first place.
I have been very verbose, but what I'm getting at is that this feature set that we're discussing should be of low-intermediate difficulty. The features exist in OpenAL but were not exposed to the engine. The features need to be exposed and then a new scene object needs to be introduced. Also, some minor tweaking to the AudioAsset class and a change in the way the listener is handled are in order. Perhaps having two listeners would be appropriate; one for ambient sounds like background music and one that follows the camera/player for positional audio.
So, I wouldn't say it would be "simple" to add these features, but it shouldn't pose that much difficulty either.
02/21/2013 (10:28 pm)
<shrug> I suppose you could call it "event driven." Sounds are played by calls to alxPlay(). Whether this happens as a result of an input event or because you loaded a level up and started it's background music is up to you and in the strictest sense everything that is done is an "event."The only thing that is holding the system back from using the features of OpenAL that you're wanting is that they are not implemented in the engine at the moment. Technically, when the campfire AudioEmitter that we have discussed is loaded it would immediately begin playing its associated sound, which would loop. Because we have discussed attaching the listener to the player object and enabling the use of OpenAL's 3D sound capabilities the player object will "hear" the sound when it is within range without relying on some sort of callback or other aid - though technically the "event" of loading is what started the sound playing in the first place.
I have been very verbose, but what I'm getting at is that this feature set that we're discussing should be of low-intermediate difficulty. The features exist in OpenAL but were not exposed to the engine. The features need to be exposed and then a new scene object needs to be introduced. Also, some minor tweaking to the AudioAsset class and a change in the way the listener is handled are in order. Perhaps having two listeners would be appropriate; one for ambient sounds like background music and one that follows the camera/player for positional audio.
So, I wouldn't say it would be "simple" to add these features, but it shouldn't pose that much difficulty either.
Torque Owner Richard Ranft
Roostertail Games
Not all of the API has been exposed, however, so some work would need to be done. All in all I think that sticking with T2D's current audio system would be preferable to porting the SFX system - primarily because it's less work, but also because it's just simpler. Simpler means less error-prone.
Your %campfire example is over-complex. The parameters are set on the Asset. So the first two lines are superfluous - you'd just call alxPlay("ToyAssets:burningWoodSound") - and the third should be a field on the asset as well, so that whole thing would be dealt with by the call to alxPlay(). Your asset would resemble this:
<AudioAsset AssetName="burningWoodSound" VolumeChannel="1" AssetAutoUnload="0" Looping="1" Radius="10" AudioFile="burningWoodSound.wav" />The Radius field would have to be added to audio Assets, but along with the described changes to assign the listener to a mobile object to make things like radius relevant this shouldn't be a huge undertaking.