Tackling the tge terrain

#21 My point is that if the only "assistance" you intend to offer is "buy TGEA", then kindly butt out. For some of us, TGEA is NOT the solution and won't be until it is cross platform (and I'm not holding my breath).

#22 The entire purpose of my monolog regarding the two different engine target markets was to try to provide insight in the minds of those wanting to update the TGE terrain (and the MK) so that you as developers would understand the impact of decisions that you make.

You personally are on a non-windows environment, and that gives you a set of restrictions/opportunties.

However, you aren't the only developer (and in fact, are a late comer) to this thread...and others have completely different restrictions/opportunities, but if they don't consider the long term impact of decisions like "I have to have shaders to make my terrains look better", then they may not be making good long term decisions for their product.

It may very well be that you've already done the target market research and cost/risk/benefit analysis for this type of additional development, but I can just about guarantee that most haven't. But when you come into a thread late, with a pre-conceived notion of:

--GG's intent ("TGE-A isn't cross platform"--when in fact it's designed for more cross platform capability than any other product we've ever made)
--GG's business practices ("If GG would allow a developer to sell an OpenGL layer for TGEA for profit, we also would likely not be having this conversation."--Have you even bothered to offer your services? Have you ever thought about why Alex was an interesting employee to hire? Can you read our minds, and have hacked access to our business plans?)
--GG's attitude ("marketing drone", when I was actually trying to teach the business aspect of game development)

You want to know the root reason why I came into this discussion with a "teach the underlying business concepts first" approach instead of right into tech?

Because not once has anyone here described what they actually feel are issues with the stock TGE terrain, they just state "problems with legacy terrain", and "it doesn't work for me", and other various generic statements.

That implies (no insult intended to anyone) a low level lack of analysis of target market, game genre, product development lifecycle and risk/cost/benefit analysis, and quite a bit more that goes into a well informed decision to re-write a major system within an engine (any engine). Based on those implied areas of need, I began discussing the design and market targets of the engines themselves.

#23 Personally, I feel that a terrain overhaul is long over due.

If TGE's terrain was a light bulb, it would be long burned out. If the employees aren't willing to make an effort to improve then it is right to assume they are unable to do so.

So this is where GG should make a choice, get a new light bulb or sit in the dark.

#24 Re: Porting Atlas to TGE

You're probably better writing something from scratch. Atlas is heavily dependent on Shader technology, and that would kill the broad spectrum hardware support that TGE offers you.

Now, I have to ask, what stage of development are you in now? Have you actually tried to make TGE terrain work? Or have you just looked at the "listed" 2k size limitation and said "that's too small!" I'm trying to save you some effort here, and I know how precious time is in an "indie" project. Build the max size TGE terrain, stick a player and some buildings there and run around. It really is quite large. There are other "tricks" you can pull like slowing down everyone's movement rate, and that will make the terrain appear larger.

Word of advice: You attract more bees with honey. Generally, insulting someone into helping you doesn't work.

#25

Quote:
Personally, I feel that a terrain overhaul is long over due.

If TGE's terrain was a light bulb, it would be long burned out. If the employees aren't willing to make an effort to improve then it is right to assume they are unable to do so.

So this is where GG should make a choice, get a new light bulb or sit in the dark.

Ok, ignoring the glib metaphor for a moment. What exactly is wrong with TGE terrain? Lets get down to real technical "brass tacks" here.

#26 [disclaimer]GG is by no means perfect (documentation for example), so please no one think i am taking sides...[/disclaimer]


I dont want to sound like an arse here, but if cash flow or Multi platform support is really that much of a problem wouldn't you be better of using one of the many GPL engines (like Ogre or CrystalSpace) that are out there?
You could use as many copys of the product as you wanted and as long as you released any change to the code you made you would be legal.


Being cross platform is important, but when you compare the Windows to Mac/Linux user ratio i can see why its not on GG urgent to do list.

I mean ID's next engine is only just about to support Mac OS

When it comes down to it, we all must realise that TGE has a limited life span that is based of very old technology (DX 8?) and if GG are clever they would be pooling their resources into TGEA because it is their Flagship product and is the future of the company where PC's are concerned.

Sure OpenGL would be a great addition to the engine (and much needed) but i think getting the engine working with all the TGE features that are either not available or yet fully implemented is more important.

How about making TGEA DX10?
That is something that will need addressing asap. because eventually Vista will be king of the hill.
Most new PC's are already shipped with it now and sooner or later TGEA will start to look like a legacy product, rather than a Next gen engine.

Would we rather see GG close it doors forever because it spends all its time supporting products for years longer than it should have, or flourish with the industry and grow alongside current and relivent technology's?.

I own TGE but i upgraded to TSE because i realised that TGE was quickly heading towards a dead end.
This was a harsh reality check and i wasted my money, but i had to make the jump so i wasted less time in the future learning the new engine.

It must be reinforced that GG could make the jump easier by providing an upgrade path though.

#27 The following may be completely wrong, but it's my take on it - and maybe some TGE licensees can really think about it. Caution, novella ahead!

Look, TGE is old tech. It's designed for spectacular compatibility with older hardware, cross-platform development, and other functionality.

When Tribes originally was released, its engine's terrain was completely revolutionary, and, I would wager, required immense amounts of coding voodoo to happen. As this engine became TGE, many things were cleaned up and documented; however, the really remarkable and revolutionary-for-its-time terrain still remained a bit of an enigma.

The GG folk realized that it needed an update and they tried to fix up the whole system. Instead, after who-knows-how-long, they abandoned it and created Atlas. Unfortunately, Atlas is only in TGEA, and at the moment TGEA hasn't been ported over to other platforms. I'm sure the hooks are in place which will make this act of porting to Linux or Mac OS X much easier, but it's not there yet. OK.

So, every few months, someone decides they're going to rewrite or "fix" TGE's terrain. I've heard that siren's call a couple of times, but a week's worth of digging through files has shown me that it would require someone much more knowledgeable about C++ than me and with a lot of time on their hands to fix it.

Consider this, then:
1.) Is massive terrain so important to your game? I'd love to have huge terrain as well, but at the moment cross-platform compatibility and compatibility with more hardware is more important to me and my target market, so I'll live without massive terrain. If things go well, maybe I can consider hiring a full-time developer whose only responsibility would be to spend a year or so hacking in a new terrain system to TGE, or adding paging or other features. But, by that point, I'd hope TGEA was stable and cross-platform, so this could be a moot point.

2.) If massive terrain is a make-or-break thing for you, then perhaps TGE - an engine designed to be able to support hardware from, like, 7 years ago - isn't the best option. And, since TGEA doesn't fit your needs (and I agree, it's not ready for production without lots of work on a programmer's part but, hey, it's not a drag-and-drop solution), perhaps you should be looking at entirely different engines. No one has ever publicly released a successful fix for what many perceive to be issues with and updates to TGE terrain - maximum size, tiling, paging, etc. Re-read that sentence again. This discussion has been going on for years and it has never been resolved to a satisfactory point, or if it has, it has never been released. People like me that have said that have generally been told "well THIS time we NEED it so we're going to team up and DO it, and if you're not helping, gtfo!" Great. I'm not holding my breath.

You're not going to make Oblivion on TGE. Things like the modernization kit help put a fresh coat of paint on TGE, but the terrain runs too deeply to be easily "fixed." It's best to just assume that modern-style cutting-edge, huge, seamless, high-res terrain isn't going to happen in TGE and react accordingly.

You either need to rethink your game design, rethink your cross-platform requirements, or move to a different engine. (Sorry, BJ, I know we were talking about this in IRC, and your metaphor was nice and all, but that's the truth.)

#28 /puts marketing drone hat back on

Quote:
own TGE but i upgraded to TSE because i realised that TGE was quickly heading towards a dead end.
This was a harsh reality check and i wasted my money, but i had to make the jump so i wasted less time in the future learning the new engine.

TGE is not heading towards a dead end if you are using the engine for an appropriate target market.

The absolute, without a doubt, proven by market research largest segment of the gaming market is "soccer moms" and "retired grandpas".

Soccer mom's have a large amount of disposable income, but little to no interest in buying the latest and greatest gaming rig. They have nice laptops with probably integrated graphics, and/or a run of the mill dell system that is a couple of years old, and they won't touch it until it stops working, and then they may (or may not) buy a different one. This market demographic spends money on games--specifically casual, but in some cases they are getting into more "real games", but again, they aren't going to update their hardware because you added shaders or need 512m video cards. TGE is a perfect engine for games targetting that market segment.

Retired grandpa's are even more amazing--these guys in fact have huge disposable income (from smart retirement focused investments to reverse mortgages to simply not having all that much to spend money on in the big picture), and TGE stock is probably not a good engine choice for this segment--these guys have the latest hardware, and in many cases have better gaming rigs than you. as developers do. TGE-A would be a pretty good engine decision, given it's legs, and the fact that it sacrifices back-generation hardware compatibility for current/future tech optimization and growth. The problem is--no one is really sure what kind of games this market segment is leaning towards yet, but given personal anecdote: my father (who is a grandpa) plays WoW to the exclusion of anything else.

So the question is--which segment do you think best fits your game idea, and even more importantly: which aspects of the terrain system doesn't meet the needs of a game for a particular segment?

When it comes down to it, a very large portion of the TGE/TGE-A consumer market (you guys) are indies or hobbyists. Given budgets, time available, resources available, and general skill sets, what makes sense for your game development--writing a new terrain engine, or making an informed decision on your target market segment and building up your tools and systems if needed to meet that market segment's hardware and game expectations?

#29 One final thought:

Nintendo just announced this week that they are smoothing the cost aspect of putting indie games onto the Wii to as low as they can. Other than the price concerns with receiving an ESRB rating for your game (a requirement for WiiWare and downloadability on the console), as things stand right now in the industry, the latest announcement makes the Wii one of the most accessable consoles for indies (I don't want to get into the XNA/Torque X vs Wii discussion here as it's a good conversation, but not on point, so please don't bring it up here at least!).

Now, here's my question--does anyone know the graphics and resource (art assets specifically) limitations of the Wii? Hint: It doesn't have shader capability, and it certainly isn't going to handle massive terrains.

#30

Quote:Now, here's my question--does anyone know the graphics and resource (art assets specifically) limitations of the Wii? Hint: It doesn't have shader capability, and it certainly isn't going to handle massive terrains.

And I think we all know which console's the runaway best seller.

It's such a cliche, but at the end of the day the games that endure are the ones that are the most fun, regardless of core tech. You can put a pig in a pretty dress, but it's still a pig.

(No comment on how putting a beautiful woman in a pretty dress = even more beautiful, but, c'mon, what I'm reading in this thread is that no one can use the pretty dress to begin with for whatever reason. Ow, metaphors... my brain...)

#31 Final final thought (yeah I know!)

Quote:
So, every few months, someone decides they're going to rewrite or "fix" TGE's terrain. I've heard that siren's call a couple of times, but a week's worth of digging through files has shown me that it would require someone much more knowledgeable about C++ than me and with a lot of time on their hands to fix it.

I was one of these guys--two and a half years ago, I walked in to TGE with the exact same assumptions that many in this thread have: TGE terrain is outdated, I must have massive terrains, etc. etc.

I spent 3 full months (at the time I was an indie with solid self-funding) working with at the time "modern" attempts, ideas, designs, and in one case a 5,500 line patch file to implement paged terrain blocks. I wasn't a Torque expert by any means, and my c++ was relatively rusty, but I had every intent of "fixing the out dated TGE terrain". I won't say I failed miserably, but I didn't finish the implementation either.

As it turns out, a couple of relatively minor design changes, as well as learning, and then applying market analysis to my project design made me realize that all the "nay sayers" and "marketing drones" were actually pretty well informed.

#32 @mark
It's problems are the same as it's limits.

a. Although repeating terrain can be a godsend in some situations is others people would rather have a large non repeating terrain. There isn't an option to choose and are forced to accept the repeating terrain.
b. The inability to create realistic mountains because of that size creates yet another crutch.
c. Th terrain texture quality and such can be quite trivial and I would prefer a little more quality.

On paper these problems are not as impacting as they are when you hit a brick wall because of them while in development. If fixing problems such as these is such a difficult task that should show how much of a mess the code is. At this point things are not looking too good for TGE terrain.

#33 Ok first off to all please can we keep this objective as a thread, the intention was to canvas ideas, support and information from all and everyone that would hold an interest in a discussion about TGE terrain.

@Stephen - I'm working on a project with Jason Lee and although he didn't go into detail he did mention further up in the thread some of the features that we'd looked at in other terrain engines which we'd like to make use of, most prominant of which is the size but also additional "beautifiers" - we're currently at the starting phase of developing an MMORPG game - by the start I mean we're looking at story line designs, concept artwork and of course the whole technological headache that actually coding and running an MMO will be.

I'm the Lead programmer on the project and have worked in IT for nearly 10yrs now so I'm fully aware of the processes and pitfalls of the entire development lifecycle and certainly fully aware of the analysis, risks an business decisions (the popular and unpopular) that have to made in this kind of industry. So I can assure you we're not about to blindly leap into hacking about with the terrain without a lot forethought and planning.

As an outsider to garagegames staff it looks to me as though TGE is a dying product - yes it's loved, yes it still works but it's an old engine these days and has certainly been a successful one, at some point in the future it will be put to bed and active development will cease on that product - as with every software application. As a business it's not feasible to continually tinker, tweak and update what is essentially an old system... Also, like any good development team work has begun on a replacement before the older one is brought to it's knees and no longer viable.

This software lifecycle leaves a period where the new is lacking in certain areas - bugs, features, tools and support/knowledge available and isn't therefore as productive an option as the older more stable engine but does offer some of the nicer and newer things people want. Unfortunately this appears to me to be the stage we're at with TGE and TGEA - which then leads to some decisions, Do you go with the older stable software - accept the loss of functionality for the moment and review an upgrade later or do you go with the newer and work with the bugs, await the tools (in this instance for us its things like AFX). The purpose of this thread was really then a look at a third option.... whether to look at TGE and see if it can be brought more up to date in the areas we need and how much work that would involve. For us another thing we'll have to look at is shaders - which again we will have to balance of between TGE... TGE+MK and TGEA.

Usually in this situation I work with the vendor to understand their roadmap for each of the products - is there such a roadmap for TGE / TGEA so that we can assess which is going to be the most viable option for us? Unfortunately without access to the TGEA forums there's not too much I can glean from how stable, usuable it is and again without source I can't determine how much work would be involved in adding OpenGl support, so any pointers on either of those would be useful too.

#34 We're starting to get some where with new design conditions, but still extremely vague:

Quote:
Although repeating terrain can be a godsend in some situations is others people would rather have a large non repeating terrain. There isn't an option to choose and are forced to accept the repeating terrain.

Can you define "large"? "Accept the repeating terrain" is (no insult intended) not true--it's a boolean and a couple of minor code changes to turn off repeating terrain.

Currently, the system (TGE terrain) is highly optimized (which is why some people look at it and say it's "poor" or "hacky"--there is a huge difference between bad code and optimized code, but not from first glance)--for example, it uses an assembly language real time texture blender. Is that hard to figure out? Absolutely is! Is it "bad code"? Not at all--it's an optimization given a set of design requirements.

It's also optimized for 256x256 height grid points--again, for optimization. Your next well-optimized choice would be to move to 16kx16k (since any increase in size means another byte to store data, may as well go all the way), and this will require not only reverse engineering the optimizations, but then analyzing the rest of the code to reimplement the new design condition, and then re-optimize.

Your other option would be to keep the 256x256 setup, and go with supporting multiple terrain blocks per mission. Base level implementation of this is rather trivial actually (search/replace on a lot of single instance accesing of the terrainData object), but then we have to factor in boundary conditions, cross-block collision, rendering, editing, and a host of other issues.

Quote:
The inability to create realistic mountains because of that size creates yet another crutch.

that's actually a limitation of heightmaps in general, not necessarily terrain "size". Whenever you are using a procedural renderer and collision algorithm with a heightmap configured set of altitude points, steep surfaces and resultant texture stretching are an issue. DTS/DIF mountains are a possible solution, or you can convert your heightmap data to an internal poly representation--which is exactly what Atlas does. Unfortunately, this brings in a completely different set of follow-on challenges, one of which is a huge complaint from the TGE-A/Atlas user set: how do you do dynamic/real time editing on a set of poly based lowest level of detail (chunked lod, remember), and also be able to decimate that poly set up through the LOD levels to integrate properly with the rest of the renderer? It's done in a pre-processing stage in the Atlas pipeline, but are you willing to give up real time editing for a poly based solution that gives you more accurate geography?

Quote:
Th terrain texture quality and such can be quite trivial and I would prefer a little more quality.

Also a side effect of blended texture renderers vs direct texture "painting". Atlas goes this route as well, utilizing clipmapping (also known as MegaTexturing) and a choice of blended textures or unique textures depending on your needs. There are already resources for increasing the number of textures from 6 to 8 (which is actually a minor GUI/Editor bug, not a terrain system issue), and from there moving to 16 textures, but that moves you from the optimized assembly language blender to a c++ renderer, at quite a bit of performance loss.

In the big picture, what level of terrain capability are you looking for? EQ/Planetside level? EQ2/WoW level? Vanguard level?

Until we define exactly what it is you want, you can't define how to get there. And until we account for all of the costs and risks associated, we can't even begin to decide if it's a "can't get there from here on this path" situation, or a "it's going to be uphill a long way, but once it's done it meets needs" scenario.

#35 Edit: Sorry didn't realize this became so active, the previous isn't needed to re-hash... I'll try to continue to refine our needs...

If you want an example GuildWars is one, but it's not as important as saying... We are using instanced maps, as well as, a few massively shared zones similar in scope to say WOW. Texturing is really key in most games that aren't built entirely around dif maps, like most FPS games are. In those cases, terrain is secondary.

As i mentioned, creating a cave or tunnel, even in the shallow sense is important to me and there are ways to make it with the current terrain using zone portals, but in many cases it would be nice to just walk into a cave without having to load the map.

Really, texturing is important to a game that takes place outside, so tackling multi-texturing, detailed splating are pretty optimal. As well as creating maps that can reach at least a true 2k or 4k limit.

#36 www.garagegames.com/index.php?sec=mg&mod=forums&page=result.thread&qt=2471
I found this post and this actually is very very close to what I wanted (it'll take a little longer but idc).
Most people in this thread sound like they want a certain level of scale or LOTR proportions and the ability to create bigger and more detailed terrain is what is needed to do that. The multiple terrain blocks thread has quenched my thirst.

#37 @Brian: good find, but I have to be up front: Bryan Turner's project is exactly what I was working with, and ultimately even that solution didn't work and cover all the things I thought I needed/wanted to do.

It's just not a trivial solution in the long run, which almost by definition is why it's such a "hot" (both in frequency and content) topic.

#38 It wont cover everything but it's an improvement for the situation Greg and I are in if it works.

#39 I've played Guild Wars pretty extensively, and from a pure size/scale perspective honestly cannot think of a single area in the game (pre expansions anyway, haven't played for a long time) that cannot be done (again scale/size wise) with stock TGE terrain.

Texturing possibly, but a big part of it is artist skill and understanding of "painterly" techniques when it comes to a weighted texture blending system. Increasing number of textures per terrain will help, but has quickly diminishing returns when it comes to performance.

Multiple detail textures is another area to explore, but this is guaranteed to not be trivial due to the level of optimization currently in both the TGE terrain system itself, and the assembly language blender.

Quote:
As i mentioned, creating a cave or tunnel, even in the shallow sense is important to me and there are ways to make it with the current terrain using zone portals, but in many cases it would be nice to just walk into a cave without having to load the map.

You aren't going to get that with a heightmap stored terrain data set without multiple terrain blocks, and inverting one over the other.

What many people do with this need is to create an interior (dif) that is the cave, populate it with dts "props" (stalagtites, etc), and then place it into an appropriate area of the terrain by using the "cut hole" capability.

Quote:
As well as creating maps that can reach at least a true 2k or 4k limit.

A lot of scale is exactly that--scale. There is a project (can't remember the name off the top of my head) where the player is a toy solider, and the entire mission area is a kid's bedroom. What he did was play with the ratios between the player's size, the physics with which the player can move through the mission, and tuned the mission's perspective for increased size.

For what it's worth, the "stock" ratio of player model size to movement speed to terrain grid size is actually a 2kmx2km block to the player's point of view. And that's a lot of space for a single mission.

#40 Correct me if I'm wrong, but isn't that 2k with 8 square size? Which is really a matter of stretching a 256 height map 8x's?

And you're right, you can get fairly decent texturing with proper texture creating and blending many of the textures all around, but, those texture details have to be sooo incredibly finite that it makes for a very troublesome set of details. In most instance i have to make a 2k x 2k texture and shrink it down to 256, loosing a really large amount of the detail i spent creating. If by chance those 256 texture weren't stretched so much, they would look very nice. Again though this stretching does limit the texture types one can use as well and the ratio gets even worse when you use scaling of avatars and interiors.

I agree that in scale you can create Guild Wars maps, but by stretching (that may not be the right word) the textures and height maps 8x's, which really kills the overall quality far to much. Let's address one thing at a time... Talk more about multi-detail support?