Side Scrolling Camera Audio Problem
by Matt Giffels · in Torque 3D Beginner · 07/11/2014 (11:06 pm) · 16 replies
Hey everyone,
I am trying to setup a side-scrolling camera and so far I've gotten the camera to position itself correctly and follow the player. I am having two problems with the implementation I came across on this forum though:
1. When an object moves between the player and the camera the camera snaps forward to avoid the object. I'd rather just have the camera stay stationary in the Z axis and only follow vertically and horizontally.
2. As the player moves left or right the sound effects (player shooting gun for example) move off into the distance as if the camera had never actually followed the player. To me it's like the audio is only being played through whatever camera started out attached to the player and not being played through the camera that is actually following the player. Sorry if this is confusing.
Other than the method given below I have tried moving the camera manually by utilizing the schedule function and having the camera's position updated every few ticks but that is making the camera movement stutter and look bad.
Here is the current implementation I'm using to get the camera into a side scrolling format, this code all lives in onClientEnterGame:
Any ideas?
I am trying to setup a side-scrolling camera and so far I've gotten the camera to position itself correctly and follow the player. I am having two problems with the implementation I came across on this forum though:
1. When an object moves between the player and the camera the camera snaps forward to avoid the object. I'd rather just have the camera stay stationary in the Z axis and only follow vertically and horizontally.
2. As the player moves left or right the sound effects (player shooting gun for example) move off into the distance as if the camera had never actually followed the player. To me it's like the audio is only being played through whatever camera started out attached to the player and not being played through the camera that is actually following the player. Sorry if this is confusing.
Other than the method given below I have tried moving the camera manually by utilizing the schedule function and having the camera's position updated every few ticks but that is making the camera movement stutter and look bad.
Here is the current implementation I'm using to get the camera into a side scrolling format, this code all lives in onClientEnterGame:
%client.setCameraObject(%client.camera); %client.player.mountObject(%client.camera); %client.setFirstPerson(false); %client.camera.setOrbitObject(%client.player, "0 0 " @ -mDegToRad(90), 15, 25, 25, false, "0 0 1", true);
Any ideas?
#2
07/12/2014 (9:07 am)
Our camera snapping problem is now solved! Thanks for your help on that. Still stumped on the audio problem though. It's as if whatever receives sound effects is not moving along with the camera. I'm having trouble determining how the engine knows where to hear sound effects from in world space.
#3
I've noticed that if I don't call setCameraObject the camera stays snapped behind the player in third person mode which I assume means the camera is attached to the player's Eye node. I feel like I'm missing a core concept here with how the camera works. I'd rather not have the camera actually attached to the player's skeletal structure so we can do other camera effects more easily.
When I call setCameraObject it seems to remove the camera from the player's skeletal structure but the audio seems to still come from where the player spawned rather than where the camera is positioned. Any help with getting this setup properly is greatly appreciated.
07/12/2014 (11:56 am)
Here is the side scrolling camera code in it's current form:localClientConnection.camera.setOrbitObject(localClientConnection.player, "0 0 " @ -mDegToRad(90), 15, 25, 25, true, "0 0 2", true); localClientConnection.setCameraObject(LocalClientConnection.camera); localClientConnection.setControlObject(LocalClientConnection.player); serverConnection.setFirstPerson(false);
I've noticed that if I don't call setCameraObject the camera stays snapped behind the player in third person mode which I assume means the camera is attached to the player's Eye node. I feel like I'm missing a core concept here with how the camera works. I'd rather not have the camera actually attached to the player's skeletal structure so we can do other camera effects more easily.
When I call setCameraObject it seems to remove the camera from the player's skeletal structure but the audio seems to still come from where the player spawned rather than where the camera is positioned. Any help with getting this setup properly is greatly appreciated.
#4
Anyway - the engine knows where the player hears sounds from by using a "listener" object. I think the sound effects are emitted from dynamically created objects that expire on end (and might be stationary). This should be somewhere in the player class code....
07/12/2014 (1:28 pm)
I'm no expert on the audio system, but it is probably something to do with the listener assigned to the player object. Perhaps when the call to setCameraObject() is made it detaches the listener? Or perhaps the sound emitter itself is not attached to the player?Anyway - the engine knows where the player hears sounds from by using a "listener" object. I think the sound effects are emitted from dynamically created objects that expire on end (and might be stationary). This should be somewhere in the player class code....
#5
07/12/2014 (2:04 pm)
After doing some more digging around it is starting to look like I should set the Camera as the control object and then control the player's movement using a similar setup to the ai bot tutorial. Thanks for the pointers on setCameraObject though. That lead me down the right path. 
#6
07/12/2014 (2:56 pm)
I believe i just changed all the sound datablocks to 2d sounds. To avoid the camera snapping when a object comes between the player and the camera i did the following: i disabled collision on any objects that may come between the player and the camera.
#7
07/12/2014 (10:00 pm)
I tried the camera code out and encountered this problem. I haven't had much of a chance to dig further, but we could open up some discussion on it. 
#8
07/13/2014 (11:00 am)
Ugh. The camera was never designed to be anything other than FPS or free observer. All sorts of ugly crap rears its head whenever you try to do anything beyond that. It really needs a redesign :(.
#9
07/13/2014 (11:53 am)
Well, now we're probably going to get a chance to look at AFX's camera setup - maybe it handles this better.... 
#10
It sets the cameras controlling client to the current connection, and within that function the following occurs:
This causes the current sound FX listener to be set to the camera. This would be corrected by the following:
Except the control object is already set to the player, so nothing happens when that command is called. Now you've got your camera is place, and the current listener in the sound system is that camera.
The bug appears to be in two parts. First of all there should only still be 1 camera, so why does it think the camera is back at the spawn position? And second, should the SFX listener really be updated based on the last call to setControllingClient in shapebase? I would think setting it from within GameConnection to the object you're currently controlling would make more sense.
This leads to questions such as "if you can control the camera position, should the sound get louder/quieter as you zoom in/out or should it be based on your players position". I think based on players position makes the most sense personally. When you switch to free look you aren't controlling the player anymore, you're controlling the camera, so then that should be the SFX listener.
I'll investigate more on this later. Worse case scenario I'll just make a function to set the current SFX listener from script. Could be useful in other cases as well.
07/14/2014 (8:43 am)
I've identified how/where the bug is occurring, but I haven't rigged up a fix yet. It's happening in ShapeBase::setControllingClient. When you call:localClientConnection.setCameraObject(LocalClientConnection.camera);
It sets the cameras controlling client to the current connection, and within that function the following occurs:
if( gSFX3DWorld->getListener() == this && !client && getControllingClient() && getControllingClient()->isConnectionToServer() )
{
gSFX3DWorld->setListener( NULL );
}
else if( client && client->isConnectionToServer() && !getControllingObject() )
{
gSFX3DWorld->setListener( this );
}
}This causes the current sound FX listener to be set to the camera. This would be corrected by the following:
localClientConnection.setControlObject(LocalClientConnection.player);
Except the control object is already set to the player, so nothing happens when that command is called. Now you've got your camera is place, and the current listener in the sound system is that camera.
The bug appears to be in two parts. First of all there should only still be 1 camera, so why does it think the camera is back at the spawn position? And second, should the SFX listener really be updated based on the last call to setControllingClient in shapebase? I would think setting it from within GameConnection to the object you're currently controlling would make more sense.
This leads to questions such as "if you can control the camera position, should the sound get louder/quieter as you zoom in/out or should it be based on your players position". I think based on players position makes the most sense personally. When you switch to free look you aren't controlling the player anymore, you're controlling the camera, so then that should be the SFX listener.
I'll investigate more on this later. Worse case scenario I'll just make a function to set the current SFX listener from script. Could be useful in other cases as well.
#11
07/14/2014 (11:26 am)
Thank you for finding the bug! Our sound is still not working correctly so this gives us something to work with. If you are able to come up with a fix though that's even better.
#12
07/14/2014 (11:42 am)
Perhaps an independent way to set what the listener is attached to, some way to override the default would be nice.
#13
07/14/2014 (2:12 pm)
What would be the best way to do that? Is that particular part of the sound system exposed to script already?
#14
07/14/2014 (2:47 pm)
I don't know for sure, but I think that the listener is not really directly exposed to script....
#15
Only 1 out of the 3 cases actually checks for a controlling camera to get it's reference from. The fix is as simple as changing:
To:
@ Matt
Open up T3D/sfx/sfx3DWorld.cpp and replace the SFX3DObject::getEarTransform function with this code:
gist.github.com/andr3wmac/1c4f6e027f1d8f8c2f55
07/14/2014 (2:56 pm)
I tracked it down and fixed it. The problem was coming from the SFX3DObject::getEarTransform( MatrixF& transform ) function. If you look at the code here: github.com/GarageGames/Torque3D/blob/8142a3e9645e881cd8e21434809289855e57a0b4/En...Only 1 out of the 3 cases actually checks for a controlling camera to get it's reference from. The fix is as simple as changing:
transform = mObject->getTransform();
return;To:
GameConnection* connection = dynamic_cast<GameConnection *>(NetConnection::getConnectionToServer());
if ( !connection || !connection->getControlCameraTransform( 0.0f, &transform ) )
transform = mObject->getTransform();
return;@ Matt
Open up T3D/sfx/sfx3DWorld.cpp and replace the SFX3DObject::getEarTransform function with this code:
gist.github.com/andr3wmac/1c4f6e027f1d8f8c2f55
#16
07/14/2014 (3:13 pm)
That has fixed the audio problem perfectly! Thank you so much for the quick turnaround on this fix. My first experience on this forum has definitely been a pleasant one. 
Torque Owner Daniel Buckmaster
T3D Steering Committee
Or, at least, replace the condition so it looks like if(false). Because there's an else clause as well.
That audio issue is bizarre. Hopefully someone can shed some light on that...