Theora Video/Audio Sync Fix
by Jackie Hayes · in Torque Game Engine · 11/05/2005 (3:41 pm) · 17 replies
OK. I have been digging at this for the past few days and think I have fixed the problem. What seems to be happening is that two vars are not being reset in MagicalTrever whenever the video ends or is stopped. The fix is actually pritty simple it just took me some time to dig it out. The problem is in theoraPlayer.cc.
In theoraPlayer.h add
at the end of struct MagicalTrevor.
In theoraPlayer.cc add the following
right after TheoraTexture::MagicalTrevor::~MagicalTrevor()
Now update these sections that are in bold.
I have tested this with the samples that were provided by GG and also with a 3.42 minute music video. Everything works fine now so far as I can tell. You can start, stop and replay with everything now being in sync. It also works great as a texture using my modified version of fxRenderObject. I am working on the audio now to make the sound source eminate from the object and not the general non-positional alxPlay().
I have this in Head V1.4 10/27/05 with John's great TLK. I am going to port Theora over to 1.3 tonight since most of my updates are in the previous version to see how it works.
personal note....
I have been using Torque since 2003 and usually keep kind of silent unless there is someone that needs a little help that I might can offer. I know I have personally got allot from these forums and want to give something back. I have also posted a few questions and have even emailed certain people that work for GG only to be ignored. I am certainly no top notch programmer like the people at GG and a few others here but I can catch on pritty quick. I know the people at GG are very busy and probably to busy to respond to everyone. Just don't forget us little guys....
In theoraPlayer.h add
void resetVars();
at the end of struct MagicalTrevor.
In theoraPlayer.cc add the following
[b]//***JH
void TheoraTexture::MagicalTrevor::resetVars()
{
mBufListHead = NULL;
mLastBufferID = -1;
}
//***JH[/b]right after TheoraTexture::MagicalTrevor::~MagicalTrevor()
Now update these sections that are in bold.
void TheoraTexture::stop()
{
mPlaying = false;
if(mPlayThread)
{
delete mPlayThread;
if(mHasVorbis)
{
alxStop(mVorbisHandle);
}
mPlayThread = NULL;
mVorbisHandle = NULL;
mVorbisBuffer = NULL;
[b]mMagicalTrevor.resetVars(); //<<***JH[/b]
}
}
************************
bool TheoraTexture::playLoop()
{
bool fMoreVideo = true;
bool fMoreAudio = mHasVorbis;
// timing variables
F64 dVBuffTime = 0;
F64 dLastFrame = 0;
mPlaying = true;
mCurrentTime = 0.f;
mStartTick = Platform::getRealMilliseconds();
bool isAudioActive = false;
while(mPlaying)
{
if(fMoreAudio)
fMoreAudio = readyAudio();
if(fMoreVideo)
fMoreVideo = readyVideo(dLastFrame, dVBuffTime);
if(!fMoreVideo && !fMoreAudio)
{
break; // if we have no more audio to buffer, and no more video frames to display, we are done
}
// if we haven't started yet, start it! :)
if(!isAudioActive && mHasVorbis)
{
alxPlay(mVorbisHandle);
isAudioActive = true;
}
// if we're set for the next frame, sleep
/*S32 t = (int)((double) 1000 * (dVBuffTime - getTheoraTime()));
if(t>0)
Platform::sleep(t); */
U32 safety = 0;
while((dVBuffTime - getTheoraTime()) >= 0.f && safety < 500)
{
safety++;
Platform::sleep(2);
}
// time to draw the frame!
drawFrame();
// keep track of the last frame time
dLastFrame = getTheoraTime();
}
mPlaying = false;
mVorbisHandle = NULL;
mVorbisBuffer = NULL;
[b]mMagicalTrevor.resetVars(); //<<***JH[/b]
return false;
}I have tested this with the samples that were provided by GG and also with a 3.42 minute music video. Everything works fine now so far as I can tell. You can start, stop and replay with everything now being in sync. It also works great as a texture using my modified version of fxRenderObject. I am working on the audio now to make the sound source eminate from the object and not the general non-positional alxPlay().
I have this in Head V1.4 10/27/05 with John's great TLK. I am going to port Theora over to 1.3 tonight since most of my updates are in the previous version to see how it works.
personal note....
I have been using Torque since 2003 and usually keep kind of silent unless there is someone that needs a little help that I might can offer. I know I have personally got allot from these forums and want to give something back. I have also posted a few questions and have even emailed certain people that work for GG only to be ignored. I am certainly no top notch programmer like the people at GG and a few others here but I can catch on pritty quick. I know the people at GG are very busy and probably to busy to respond to everyone. Just don't forget us little guys....
#2
As far as e-mail goes - please, understand that some people at GG get over a hundred e-mails a day. It's often hard enough to keep track of the essential business communication, much less correspondence from outside people. In your case, your e-mail arrived the same week that I had midterms, as well as the usual 40+ hours of work, contract work, etc. I assure you that I didn't ignore your e-mail out of spite - I don't know who else you correspond with, but I spend most of my days working hard, not wining and dining with "big fish" (whoever they might be...). :)
Reply inbound - next time, couldn't you just try sending another one making sure it didn't get stuck in a spam filter or dropped by a mail relay, instead of bringing it up in public as if I personally attacked you? :)
11/05/2005 (6:47 pm)
Good fix - in fact, I put the exact same one in last week. :) Just haven't pushed it out to public yet. I'm glad to see we reached the same conclusion independently; makes me more confident that the solution is a complete one.As far as e-mail goes - please, understand that some people at GG get over a hundred e-mails a day. It's often hard enough to keep track of the essential business communication, much less correspondence from outside people. In your case, your e-mail arrived the same week that I had midterms, as well as the usual 40+ hours of work, contract work, etc. I assure you that I didn't ignore your e-mail out of spite - I don't know who else you correspond with, but I spend most of my days working hard, not wining and dining with "big fish" (whoever they might be...). :)
Reply inbound - next time, couldn't you just try sending another one making sure it didn't get stuck in a spam filter or dropped by a mail relay, instead of bringing it up in public as if I personally attacked you? :)
#3
ps..I'll try and reserve my personal notes from now on...;-)
11/05/2005 (7:08 pm)
Ben....NO...NO... In no way did I mean that statement toward you. I was just stateing things that I personally have had happen to me on these forums. The only point I was trying to make was that if anyone reads this post and was haveing similar problems that I have had don't give up. There are allot of other people around here to help. I know you guys are very busy, as we all are. Again...I certainly did not mean ANYTHING in the wrong way. If I offended you I am truly sorry. That was not my intent.ps..I'll try and reserve my personal notes from now on...;-)
#4
11/06/2005 (2:30 am)
Thanks for posting that Jackie - one less item on my list of things to do.
#5
11/08/2005 (8:34 pm)
I finally got this working in 3D(positional sound). The sound now eminates from the movie object. The audio now fades away when you move away from the movie object and will pan as you move around the object. I am going to do some cleanup tomorrow and update this post if anyone is still interestd.
#6
11/08/2005 (9:08 pm)
Very cool, would love to see that.
#8
11/10/2005 (9:27 pm)
Awesome! Thanks for posting this, I had been having the same problem and hadn't had a chance to work on it yet!
#9
Thanks
01/13/2006 (11:56 am)
Does anyone know if Jackie Hayes ever did post his work on how he got 3D(positional sound) to work with regards to theora objects. If so, where? If not, does anyone have any good pointers as to where to start.Thanks
#10
Yeah, I'm wondering the same thing if Jackie ever posted the resource for positional 3D audio with Theora video? It'll be great if he did, and if someone can point us to it!
thanks
05/26/2006 (8:31 pm)
Hi,Yeah, I'm wondering the same thing if Jackie ever posted the resource for positional 3D audio with Theora video? It'll be great if he did, and if someone can point us to it!
thanks
#11
Any suggestions?
10/27/2006 (8:46 am)
At first this resource looked like it would have worked for me. The issue I am having is that my ogg plays fine the first time I run it. However, if I call a subsequent setFile my video plays in slow motion without sound. After reviewing the source I discovered that your fix was already applied to my version. I am using TGE 1.4 (w/ Torque Lighting Kit). I saw that James Chambers had a similar issue but no one ever replied to it. This issue can be found here: www.garagegames.com/mg/forums/result.thread.php?qt=41185Any suggestions?
#12
in theoraPlayer.h
and in theoraPlayer.cc
The variable mLastBufferID was orignally signed. A buffId is unsigned. This can lead to errors when comparisons are made between the two. As there is no value for mLastBufferID that is guaranteed to be an invalid bufferId, I had to track when it contains a valid number seperately.
11/14/2006 (11:41 pm)
Have noticed a few more synch issues. The only thing I could trace it back to was a signed/unsigned mismatch in some of the MagicalTrevor variables. I made the following updates and the problem seems to have gone away.in theoraPlayer.h
FreeListChunker<BufInf> mBuffPool;
BufInf *mBufListHead;
// ALint mLastBufferID;
[b] ALuint mLastBufferID; // MPD - Buffer ID's should be unsigned. Having it signed can cause issues
bool mLastBufferIDValid; // MPD - Cannot determine by value if mLastBufferID is valid so need to track seperately
[/b]
void *mMutex;and in theoraPlayer.cc
TheoraTexture::MagicalTrevor::MagicalTrevor()
{
mBufListHead = NULL;
// mLastBufferID = -1;
[b] mLastBufferIDValid = false; // MPD - Cannot use a 'magic' number for mLastBufferID so track seperately
[/b]
mMutex = Mutex::createMutex();
}
const F64 TheoraTexture::MagicalTrevor::advanceTime(const ALuint buffId)
{
MutexHandle handle;
handle.lock(mMutex);
// We basically find the last entry on the list that references
// this buffId, then count how much time it + all its followers
// contains, and return that amount. Then the list is truncated.
// Skip if we just saw this one... we'd better not go
// through all the buffers in one advanceTime call, that would
// confuse the hell out of this code.
// if(mLastBufferID == buffId)
[b] if (mLastBufferIDValid && mLastBufferID == buffId) // MPD [/b]
return 0.f;
mLastBufferID = buffId; [b]mLastBufferIDValid = true; // MPD[/b]
// Ok, find last occurence of buffId.
BufInf **walk = &mBufListHead;
....
void TheoraTexture::MagicalTrevor::reset()
{
MutexHandle handle;
handle.lock(mMutex);
// Since we're mostly touched by the thread, let's make this a mutex
// operation.
mBuffPool.freeBlocks();
mBufListHead = NULL;
// mLastBufferID = -1;
[b] mLastBufferIDValid = false; // MPD - Cannot use a 'magic' number for mLastBufferID so track seperately[/b]
}The variable mLastBufferID was orignally signed. A buffId is unsigned. This can lead to errors when comparisons are made between the two. As there is no value for mLastBufferID that is guaranteed to be an invalid bufferId, I had to track when it contains a valid number seperately.
#13
11/15/2006 (12:39 am)
Hi Guys, been a while since I read this post. Actually there is more than meets the eye with the Theora implementation. At the end of the day, it has to do more with threading issues than with anything else. I've left theora for a while now since we've been working with DirectShow, but my implementaiton inside of Torque is quite generalized, and I will be bringing my changes over to theora. I'll be releasing it as a resource, and be asking Garage Games if they want to itnegrate it into their head.
#14
11/15/2006 (2:35 am)
Cool. Thanks
#15
Though, it is strange the only ogg video file that runs well in it's full glory is the areTorquePopular.ogg that came with the demo. It can play the video and sound right. While the soundTest.ogg from the demo even had problems with audio (no audio). I also tried my own OGG videos with different combinations of encoding (from the most bare, to the top quality encoding) all of them didn't fare well.
03/07/2007 (10:23 pm)
I concur. There are lots of threading issues happening especially when the video file has an audio stream with it. In my case, what often happens is there is no sound and the video is almost at the point of not advancing a frame.Though, it is strange the only ogg video file that runs well in it's full glory is the areTorquePopular.ogg that came with the demo. It can play the video and sound right. While the soundTest.ogg from the demo even had problems with audio (no audio). I also tried my own OGG videos with different combinations of encoding (from the most bare, to the top quality encoding) all of them didn't fare well.
#16
I'm still having a problem with the sound not being there but I can live with that. What I can't really live with is the fact that my video jumps. It doesn't do it in windows media player or mplayer. They play just fine with sound and everything's smooth and beautiful.
In game the video jumps and the sound is there, but so quiet it can't be heard.
EDIT: I'm also seeing a white flash on the video when it starts. This flash isn't there when I play it in in windows media player, mplayer, or even GraphEdit which I just now downloaded to explore what it does. Does this have something to do with the initialization of the surface? I post as I'm looking into it, but if somebody else knows it'll probably make things easier.
06/13/2007 (10:44 am)
I'm needing a theora control but not needing to stop and replay at all so I don't have to worry about any sync issues.I'm still having a problem with the sound not being there but I can live with that. What I can't really live with is the fact that my video jumps. It doesn't do it in windows media player or mplayer. They play just fine with sound and everything's smooth and beautiful.
In game the video jumps and the sound is there, but so quiet it can't be heard.
EDIT: I'm also seeing a white flash on the video when it starts. This flash isn't there when I play it in in windows media player, mplayer, or even GraphEdit which I just now downloaded to explore what it does. Does this have something to do with the initialization of the surface? I post as I'm looking into it, but if somebody else knows it'll probably make things easier.
#17
08/01/2007 (12:39 pm)
It is only working with no sound movies here
Torque 3D Owner Brandon Maness
The positional audio would be an awesome resource BTW
Thanks again for sharing your hours of digging!
B--