How big of a RAW has anyone completed?
by Mark Dynna · in Torque Game Engine Advanced · 06/06/2006 (11:05 pm) · 15 replies
I am working on a game with TSE and we have some fairly ambitious plans for the size of terrain we want. I recently tried converting our RAW (over 500 MB) file and got an out of memory error. I was wondering how big of a RAW file everyone has successfully converted?
#2
06/08/2006 (5:47 am)
The treedepth makes a massive difference to the file size. I've had huge multi-megabyte files for 1024 x 1024 terrains. If you're having probs you can reduce the depth and see if it helps.
#3
06/08/2006 (7:40 am)
And what if I'm not even getting that far. We have a 16,385x16,385 file that is over 500MB and when I try to convert it I get a memory error and TSE crashes. Can it even handle something this big?
#4
You probably also mean your terrain is 16385x16385.
06/08/2006 (8:43 am)
Which parameters do you use for GenerateChunkFromRAW16?You probably also mean your terrain is 16385x16385.
#5
Square Size = 2.0,
vertScale = 1/256 (however I just learned from our artist that the scale is much larger than +/- 128m)
Error metric = 2.0
Tree Depth = 3
I don't even think it is getting to the point where it *tries* to convert the file, it seems to crash as soon as it tries to load it.
06/08/2006 (8:53 am)
Yes I do mean 16,385. Last time I tried I used:Square Size = 2.0,
vertScale = 1/256 (however I just learned from our artist that the scale is much larger than +/- 128m)
Error metric = 2.0
Tree Depth = 3
I don't even think it is getting to the point where it *tries* to convert the file, it seems to crash as soon as it tries to load it.
#6
And Of course (if it crash when it try to load it), increase your tree depth. (try 6 +)
06/08/2006 (9:13 am)
Try to increase your error metrics : try 3.0 to 6.0And Of course (if it crash when it try to load it), increase your tree depth. (try 6 +)
#7
"Torque Memory Error"
"Error allocating memory. Shutting down."
Vincent, didn't you write that HTC utility? I noticed you mentioned something about increasing TSE's memory (or was it file?) limit to handle up to 4GB. Am I hitting that limit here?
06/08/2006 (10:55 am)
I tried running it with an error metric of 10.0 and a Tree Depth of 8 and every time I get this error:"Torque Memory Error"
"Error allocating memory. Shutting down."
Vincent, didn't you write that HTC utility? I noticed you mentioned something about increasing TSE's memory (or was it file?) limit to handle up to 4GB. Am I hitting that limit here?
#8
06/08/2006 (11:45 am)
Ok, I ran it in the VS debugger and tracked it down. In platformMemory.cpp there is a call dRealMalloc (this probably gets called a billion times) but the call to it made from within the generateChunkFile... console function returns a NULL pointer. After that it throws the Memory Allocation Error. I think the file is too big, and I don't have enough memory.
#9
06/08/2006 (11:47 am)
Quote:The treedepth makes a massive difference to the file size. I've had huge multi-megabyte files for 1024 x 1024 terrains. If you're having probs you can reduce the depth and see if it helps.Reduce the Tree Depth of the function call or reduce the tree depth of the image itself? Is this something that needs to be done by editing the RAW image?
#10
A solution to your problem could be to take a pixel over two for your terrain to make it 8193x8193 and to generate with Em : 6.0 and Td : 7. and increase square size to 4.
I Had the same problem with platformMemory.cpp but it was the my createJpeg function which crashes TSE.
06/08/2006 (12:11 pm)
I Work mainly with 8192x8192 files, because TSE can't handle JPEG more than 8192x8192 pictures. for this terrain size I use an error metrics of 6.0 and a tree depth of 7.A solution to your problem could be to take a pixel over two for your terrain to make it 8193x8193 and to generate with Em : 6.0 and Td : 7. and increase square size to 4.
I Had the same problem with platformMemory.cpp but it was the my createJpeg function which crashes TSE.
#11
06/08/2006 (11:35 pm)
Ok, I'm making some progress. We have an 8193 RAW file that actually starts to convert. If I'm getting lots of Min/Max exceeded warnings and lerpedHeight Overflow errors, what should I be adjusting?
#12
By the way I didn't figure this out, someone else graciously pointed it out to me.
Tim
06/09/2006 (8:40 am)
Mark, check out my post at http://www.garagegames.com/mg/forums/result.thread.php?qt=45598 about using multiple Atlas terrain chunks in a single mission. That might make your situation simpler if you can chop your terrain into 4 or even 16 separate pieces.By the way I didn't figure this out, someone else graciously pointed it out to me.
Tim
#13
06/12/2006 (12:06 pm)
Just an FYI note here. I was pushing TSE to see how big I could make the terrain and I recently converted a 2048x2048 heightmap using a Square Size of 8, and vertScale of 1/32. It would not convert to CHU with a Tree Depth less than 10 (!), and then it created a 482MB CHU file. I tried loading this up in TSE and it crashed.
#14
07/02/2006 (12:03 pm)
I've tested with up to 8192 square raw files. For that I don't think you'd want a tree depth of more than 8 or so - with a tree depth of 10 you not only have over a million chunks, but each one is for only a tiny, tiny area of terrain - very inefficient!
#15
07/02/2006 (8:22 pm)
Our TQT for GDC was over 4 gigs in size. That was a long week.
Torque Owner Stefan Lundmark