Tearing in Theora
by Jon Wilsdon · in Torque Game Engine Advanced · 04/03/2007 (10:00 am) · 5 replies
The problem:
We have a lot of pre-rendered cutscenes (clips range from 45 seconds to 3 minutes or so) in our two games and we get a lot of tearing during playback. We have approximately 2 months of production left in the current phase of our project and I am hoping to eliminate all of the tearing in that time. I have little experience with video playback (besides knowing that it is hard to get right and having written two resources; Theora in TGE 1.3 and Theora in TSE). So hopefully if I describe our process and some things I am thinking about trying, someone in the community or at Garage Games can help me figure out what course of action will most likely be the best.
Our process:
We get the videos at a resolution of 720x405 as an uncompressed avi. Those files are obviously huge, on average a few gigs each. We then use After Effects to add black bars to the top and bottom to resize the image to 720x432. That step is necessary for ffmpeg2theora to convert the file without adding green lines. We then export the video as an uncompressed avi, open up that uncompressed avi in TMPG and re-save it as an uncompressed avi. If we don't re-save it out of TMPG the video will sometimes either have a greenish tint, or have some other visual problems. We finally have an uncompressed avi that is good for compressing to theora and so we run it through ffmpeg2theora.exe with outputting a visual quality of 7.
The result:
The result is decent for the most part. The theora files get compressed down to an average of 14MB or so, done from an average uncompressed avi size of over 1 GB. However, in almost every video we get at least a little bit of tearing whenever there is either a quick camera cut, or some moderately fast movement. In the videos where there are lots of camera cuts and a lot of fast movement we get a lot of tearing. I suppose it is worth mentioning that we are playing the video from the guiTheoraCtrl having used setCanvas to set it to take up the full screen.
Our ideas about what to try:
A test I just thought of while writing up this long post is to play the video in a gui that is 720x432 instead of fullscreen. If it plays without tearing then I suppose it is the stretching of the video size that is the culprit, although I would not know what to do about it since we need to play the video fullscreen.
We can play with the encoding settings in ffmpeg2theora. We have tried this in the past, a long time ago (while still using TGE I believe), and we couldn't eliminate the tearing. However, we are now using TGEA, and newer versions of ffmpeg2theora have come out. I believe we were using 0.15 originally, and maybe 0.16 at some point. 0.18 was just recently released. This idea is one that we will definitely be playing with soon as we are getting the final versions of the videos shortly.
Put it on a texture in the world. What I mean by this is to put the video on a GuiTextureCanvas (or write something from fxRenderObject) and place it in the world. The canvas can be the same size as the video, just viewed up close (as in the post by David Barr in this thread). It might eliminate the tearing, but would it be fuzzier or suffer from a loss of quality by viewing a smaller texture so close?
Move YUV->RGB decoding to pixel shader. I would imagine that if done correctly there would be a speed up in playback from implementing this. What I don't know is if it would be something that is likely to reduce the tearing. I also would have to get a better understanding of how to write a shader in TGEA... which is never a bad thing, it is just more time consuming.
Conclusion:
If you have any guesses as to what would be most likely to eliminate the tearing, comments or any other suggestions for what to try we would appreciate hearing them!
We have a lot of pre-rendered cutscenes (clips range from 45 seconds to 3 minutes or so) in our two games and we get a lot of tearing during playback. We have approximately 2 months of production left in the current phase of our project and I am hoping to eliminate all of the tearing in that time. I have little experience with video playback (besides knowing that it is hard to get right and having written two resources; Theora in TGE 1.3 and Theora in TSE). So hopefully if I describe our process and some things I am thinking about trying, someone in the community or at Garage Games can help me figure out what course of action will most likely be the best.
Our process:
We get the videos at a resolution of 720x405 as an uncompressed avi. Those files are obviously huge, on average a few gigs each. We then use After Effects to add black bars to the top and bottom to resize the image to 720x432. That step is necessary for ffmpeg2theora to convert the file without adding green lines. We then export the video as an uncompressed avi, open up that uncompressed avi in TMPG and re-save it as an uncompressed avi. If we don't re-save it out of TMPG the video will sometimes either have a greenish tint, or have some other visual problems. We finally have an uncompressed avi that is good for compressing to theora and so we run it through ffmpeg2theora.exe with outputting a visual quality of 7.
The result:
The result is decent for the most part. The theora files get compressed down to an average of 14MB or so, done from an average uncompressed avi size of over 1 GB. However, in almost every video we get at least a little bit of tearing whenever there is either a quick camera cut, or some moderately fast movement. In the videos where there are lots of camera cuts and a lot of fast movement we get a lot of tearing. I suppose it is worth mentioning that we are playing the video from the guiTheoraCtrl having used setCanvas to set it to take up the full screen.
Our ideas about what to try:
A test I just thought of while writing up this long post is to play the video in a gui that is 720x432 instead of fullscreen. If it plays without tearing then I suppose it is the stretching of the video size that is the culprit, although I would not know what to do about it since we need to play the video fullscreen.
We can play with the encoding settings in ffmpeg2theora. We have tried this in the past, a long time ago (while still using TGE I believe), and we couldn't eliminate the tearing. However, we are now using TGEA, and newer versions of ffmpeg2theora have come out. I believe we were using 0.15 originally, and maybe 0.16 at some point. 0.18 was just recently released. This idea is one that we will definitely be playing with soon as we are getting the final versions of the videos shortly.
Put it on a texture in the world. What I mean by this is to put the video on a GuiTextureCanvas (or write something from fxRenderObject) and place it in the world. The canvas can be the same size as the video, just viewed up close (as in the post by David Barr in this thread). It might eliminate the tearing, but would it be fuzzier or suffer from a loss of quality by viewing a smaller texture so close?
Move YUV->RGB decoding to pixel shader. I would imagine that if done correctly there would be a speed up in playback from implementing this. What I don't know is if it would be something that is likely to reduce the tearing. I also would have to get a better understanding of how to write a shader in TGEA... which is never a bad thing, it is just more time consuming.
Conclusion:
If you have any guesses as to what would be most likely to eliminate the tearing, comments or any other suggestions for what to try we would appreciate hearing them!
#2
04/03/2007 (10:47 pm)
Theora works well because it has great compression, however that my be why your getting tearing. i wonder if it possible to use Xvid or one of the other open source codecs. Xvid gives you much more control over your compression rates. But tearing could be from a lot, as a rule of thumb always treat video files as you would a single frame or picture file. Does that help a little, i would try a different compression setting.
#3
I got the feeling they are, just had to ask :)
04/04/2007 (5:32 am)
Are the videos working fine if viewed outside of tge-a in a normal video player?I got the feeling they are, just had to ask :)
#4
@Bobby: Yeah, I am always impressed with the compression with Theora. Unfortunately it does mean that it requires more processing power to decompress it. I am hoping that maybe some of the motion settings in ffmpeg2theora or some combination with a different video quality will help out. I will be sure to post the results of any tests I do with different settings.
@Niklas: Yep. I meant to put that in the original post, but I hit the maximum post size limit so I had to cut it. They play fine in VLC windowed and fullscreen with no tearing. It is a good thing to check. :)
04/04/2007 (7:31 am)
@Manoel: We have changed disableVerticalSync. Unfortunately it had no effect on the tearing in Theora videos... and added a sort of tearing onto 3D Objects with straight lines when a camera is moving past them. I am pretty sure we have tried all of the combinations with enabling and disabling vsync in Windows and in TGEA, but I will double check that today. If I don't mention it again, it is because it didn't make any difference. :)@Bobby: Yeah, I am always impressed with the compression with Theora. Unfortunately it does mean that it requires more processing power to decompress it. I am hoping that maybe some of the motion settings in ffmpeg2theora or some combination with a different video quality will help out. I will be sure to post the results of any tests I do with different settings.
@Niklas: Yep. I meant to put that in the original post, but I hit the maximum post size limit so I had to cut it. They play fine in VLC windowed and fullscreen with no tearing. It is a good thing to check. :)
#5
04/04/2007 (11:29 am)
When you created the video, did you set it to NTSC. If so, then you may be experiencing the field delay as it combines the fields. A lot of video applications don't experience this as frequently because they are optimized to do one thing very well. You can often see it in transitional segments or cuts for compressed scenes. De-interlace your video and see if it helps smooth things out a bit. 
Associate Manoel Neto
Default Studio Name