TCP Project - UI - Discussion

#1 I'll bite. I figure a small contribution is better than none at all.

[EDIT: I think it might be best to discuss this type of issue first, since it could require reworking of much of the existing code. Might as well address the structural issues first.]

Can we discuss layout issues to begin with? The current layout mechanism is OK, but it does have flaws. The most notable is the relative sizing option.

While it works OK for a gui that only increases or decreases in size, it totally falls apart if the parent container is resized back and forth from large to small. The problem stems from the fact that each time the component is resized, it calculates position and size based on current values. This introduces errors that eventually shrink the child component into lala land (especially if the parent gets really small at any point).

I see three ways to correct this off-hand:

==========

1) Patch up the current method by having each component keep the ratios of the extent (w or h) and position (x or y) of the component to the extent of the parent (w or h) at the time they are assembled. Then all future placements and sizings can be calculated using that factor and the parent extent, eliminating error.

This would work OK, although it would not be exact. There will still be some possible minor overlapping when the parent is reduced or gapping when the parent is expanded, but if you are not lining up components perfectly side-by-side, that will not be an issue.

It would require changes to all code that has to do with sizing, and each component would need to maintain four new floating point values (the ratios), but the changes would be simple to implement. It could be done in a backward compatible way, so that all of the existing guis will still work fine. It is also fairly quick to resolve the size/position changes, so it intruduces little slowdown.

==========

2) Patch up the current bounds system to use symbols similar to that used in the T1 PrestoPack gui code, and throw out the sizing properties entirely.

The Presto system allowed simple math, percent values, keywords such as LEFT, RIGHT, TOP, BOTTOM, CENTER, WIDTH and HEIGHT, and references to other gui components in the same container. This allowed constructs such as:

-> (position) 'LEFT+5 33%' - position the component 5 pixels from the left edge of the parent container and 1/3 of the way down the parent vertically. 1/3 is defined by subtracting the size of the component from the extent of the parent and dividing the result into thirds, putting 1/3 of the remaining space above the component and 2/3 below.

-> (position) 'SomeOtherComponent-1 SomeOtherComponent' - position the component directly to the left of SomeOtherComponent and keep the top edges aligned. When SomeOtherComponent is moved, this component moves to stay in that same position relative to SomeOtherComponent.

-> (extent) 'WIDTH 16' - resize to width of parent with a height of 16 pixels. Together with a position of 'LEFT BOTTOM', this keeps the component stretched across the very bottom of the component and 16 pixels high.

While this system was really designed for laying out HUD components on the play screen, it would work nicely for any gui. It is precisely descriptive and gives the designer an immense ammout of control. The drawback is that it will require more changes to the existing code, and may not be very familiar to many. This option could be implemented in a backward-compatible way.

It also increases the amount of processing required, since (for example) a component (A) that is sized based on the size of another component (B) will need to trigger that component to re-evaluate its (B's) size before it (A) can finish evaluating its own size. This can lead to some cascading evaluation. Also, there is the possibility of two components being circularly dependent on each other. While Presto ignored this issue we would be better served to address it. Simple to address however.

==========

3) Actually implement layouts. Java layouts come to mind (BorderLayout, GridBagLayout, etc.). This would be very flexible, leave room for extension, and be accurate in laying out components in relationship to each other while still allowing very abstract placement settings.

It would be professional, and (at least with the BorderLayout) be somewhat familiar and easy to use for developers. Option #2 could actually be implemented as a layout type in this scheme as well if this option is implemented properly.

The drawback here is that it will require very extensive changes to the codebase already extant. In addition, it would be problematic to make it backward compatible.

==========

Personally, I would prefer the second option. It offers an immense amount of control and it could easily be used to simulate the layouts in option #3.

Any comments? Other ideas? Am I over-thinking the issue?

#2 While we're at it allow some things do be fixed in pixel-extent (like gifs and borders) and others to be fixed in screen extent (like text). So you can vary your text size independent of your screen resolution.

I would vote for a real layout engine. They just aren't that expensive so long as users mostly adjust window sizes by hand, and you cache the layout of hidden windows.

By "real" layout engine I probably means simple struts and springs. make struts and springs visible in the gui-gui-editor-editor and let designer set weights on the springs and the rest is almost trivial. There are a number of damping mechanisms available to prevent negative feedback loops.

#3 Hi guys - sorry for the delay in getting started on this. In writing the skinnable user interface controls - I learned a great many things about the existing 1.1.1 user interface code. I am attempting to compile a list of core questions that we all need to answer before continuing.

For example, do we have the freedom to replace the UI system in 1.1.1 with an improved system that might not be backward compatible? The answer to that one single question will affect the direction we take in implementing the enhanced user interface. There are many other questions of equal importance that need to be answered before we can continue.

In the next day ro two I will get those core questions assembled and posted. In the meanwhile, go ahead and keep throwing out ideas for UI enhancements. The layout ideas are a perfect example.

#4 I'm not sure If this is the right place to ask.
I was wondering if anyone else noticed this little annoying pixel or two wide strip.

It Appears:
on the right and bottom sides on the main screen and
some other places, Only in some of the video modes(1024
by 768 for example) and on more then one of my systems.

Is it the textures supplied with the demo or something else?


Edit:My first attempt made no sence.

#5 Not that I've noticed...you have a Trinatron monitor?

#6 Viewsonic, It does it on other computer's too. Very odd and a bit annoying after a long time. All my computers are running win98 and Geforce cards. Like I said only at some resolutions and only really noticable at the main page. I'm not sure what is causing it.

If you get bored try flipping through the resolutions and run your mouse along the right and bottom. When I do that I get mouse pointer artifacts at some settings.

This could be all a hardware issue but I wanted to see if anyone else has noticed it.

Thanks,

Bruce

#7 Justin, it would be great if you can do it as compatible as you can and try using code that its already in the engine, this way it will be easier to keep updating from cvs without having to break lots of stuff...

#8 I think you can take it as a given that any layout system that cannot duplicate the current one (albeit through some compatibility layer) is insufficiently flexible.

Under no circumstances use the same procedure names to do new things (that's basic API design hygiene) and rewrite the old ones to load the old window-description, add the new layout info, and display.

Or you can leave it so the old ones just work in their own independent fashion.

IMHO TCP cannot get in the business of requiring people to rewrite their code, though of course GarageGames can.

#9 I have the same problem that Bruce is describing, and I wouldn't be surprised if everyone else has it but is not noticing it. Just for kicks, color background.jpg entirely black and go to fullscreen 1024x768 mode and run your mouse along the bottom and right, and you'll see what he means.

#10 It is a bug in the stretching code... I fixed it with a hack but I have not comfirmed that it works in all circumstances....basically what I did was when its resized dont display the last pixel on the bottom or right...thus hiding the artifact by not drawing it. Here is the fix if you want to try it out yourself:

Go to engine/dlg/dlg.cc and go to the void dglDrawBitmapStretchSR function and in it go to the lines:

F32 texRight  = F32(srcRect.point.x + srcRect.extent.x) / F32(texture->texWidth);

F32 texBottom = F32(srcRect.point.y + srcRect.extent.y) / F32(texture->texHeight);

And change them too:

F32 texRight  = F32(srcRect.point.x + srcRect.extent.x-1) / F32(texture->texWidth);

F32 texBottom = F32(srcRect.point.y + srcRect.extent.y-1) / F32(texture->texHeight);

-Tim aka Spock

#11 Thanks Tim, I'll give it a try.

#12 I forgot to tell you this won't fix the problem with the main menu background because it is not a normal bitmap but a chunkedbitmap....it uses a different stretch function.

-Tim aka Spock

#13 The issue is common to Torque and T2 (at least the last time I checked).

#14 I just made a gui, and right now, its almost impossible to place controls in a natural manner.

I'd expect normally, that controls either placed themselves relative to the top left of thier container (dialog or whatever) OR as an relative size (a percentage basically so that the container and the object remain the same as the container is sized).

Ive really no understanding of how it works now, other than it doesnt (for me :))

Lets get something going on this, its driving me nuts!

Phil.

#15 I have now implemented option #1 from my post above. When using relative sizing, the resized components act properly now.

Not that I think pursuing another option is a bad idea (as I said above I actually have other preferences when it comes to a solution), but this is immediate and, well, done now. I'll post it as a code snippit. As a short term solution, it at leasts corrects the deficiency/bug in the current relative sizing.

Meanwhile... does anyone have any comments on my 'essay' above?

#16 I would like a relative positioning that didnt resize. My bitmaps look horrible in high resolutions cause they are all streched since I made them for 640x480.

-Tim aka Spock

#17 That would be supported by my option #2 above...

#18 Ok, based on whats been discussed so far and the ideas I have collected in building the Myrmidon user interface, here is a list of proposed features for the new user interface system. Let's discuss!

Backwards Compatible After implementing the new user interface for Myrmidon, I realized (the hard way) that trying to augment the existing Torque controls with so much new and different functionality was very time consuming and error prone. With the type of new features we are suggesting for the new user interface system, my suggestion would be to implement a new "Canvas" that can run side-by-side with the existing system. There are pro's and con's to this approach but after going through the effort once already, I really feel like that's a good direction to explore first.

Compound Controls One thing I found when implementing the Myrmidon user interface was that many of the base Torque controls failed to re-use code in other controls and instead duplicated a lot of effort. An example of this would be the minimize, maximize, and close buttons on a frame window control those should just be regular buttons but the base Torque frame window control duplicates all the button code and manages its own max, min, and close images. The new user interface controls should be true compound controls that re-use other controls as necessary.

Control Profiles There are aspects of the existing Torque control profile system that are appealing but for the most part, they are a pain in the rear. For example, if I want to change the font on a particular control I need to create a new profile and assign that one control a new profile. This makes real-time control manipulation difficult, especially with complex controls. Every control should have its own concept of font, colors, etc that are populated from the loaded skin. With a skinned system in place, the need for these profiles diminishes greatly because most of the information is described with the skin itself.

Editors Of course, it almost goes without saying that a UI editor is needed to facilitate rapid development of a user interface with whatever new system we build.

Layout Management It is clear that the layout features of the existing Torque 1.1.1 controls leave something to be desired. Personally, I think HTML is a great template to follow for layout behavior and management but there are others as well. We must decide on the best approach during the design phase of this project. Another aspect of Layouts would be introducing FX like scrolling, animated positions, fades, wipes, etc.

Internationalization Do we want to support multiple languages with these new user interface controls? If this is the case, we need to think about supporting external string tables, for example.

Multi-Res Support Now that graphics cards can run 3D games at 1600x1200, there is a real issue for many user interfaces in games today. At those high resolutions, the UI text and icons are often too small to read or use properly - forcing the gamer to switch to a lower resolution. This is inherently a problem with using bitmaps to render the user interface elements. That said, I still believe there is a solution out there to this problem and that this is a great opportunity to make Torque shine.

Scripted Controls One of the biggest deficiencies in the current Torque controls is the lack of support for scripted controls. In a fun experiment, I implemented our latest HUD control for inventory management all in script. This means I had to go into the base Torque controls and add a whole bunch of scripting events (like onMouseDown) and expose a handful of user interface methods on the Canvas and base GuiControl class. The experiment was a success and I proved that complex user interface controls (even with drag/drop support) can be developed in script only - as long as the engine exposes enough methods and events. I propose that the new user interface system be targeted at building user interface controls in script as much as possible. The engine would have the Canvas and the common controls like a window, button, or scroll bar but let scripts do the rest.

Skins All controls are rendered using external images, called "skins". A set of bitmaps collectively are called a "user interface skin". The ability to "skin" the user interface is extremely powerful for an engine like Torque because every game ideally wants a unique look to their user interface. It also bodes well for the "mod" community if your game supports extensibility. User interface skins are, of course, not a new concept and can be found in many popular programs like WinAmp, Media Player, WindowBlinds, and now Windows XP.

#19 Good points.

Another few things came to me recently.

1) Transitions. The movement of one interface to another, to do a really professional job, we should look at how menu's should change between one and another. That implies stuff like fading, movement (paths) etc.

I like the idea of scripting the interface, its a pretty obvious area that SHOULD be scripted, which is strange, because I think there are lots of area's in game which ARE scripted which shouldnt be :))

Having the default gui control with some more built in behaviour (transitions, movements etc) would help.

I agree about the skins.

Ive been down the HTML route myself, I personally hate it with a passion. If we are using a realtime layout, why would we want something that can be mis-represented? the only problem with the current layout system is that it doesnt actually work :) thats not to say its not the right direction, just a confused implementation.

I'd say we move towards something more like flash than like html.

I agree about the profiles, there are better methods.

Phil.

#20 TSRR is already using a component that move/resizes automatically. We are using it for slide-out tabbed settings screens, but it does quite a bit more. If I can find the time to make the source look acceptable, I'll post it. Doesn't do transitions though, unless you count shrinking to nothing or sliding off screen or under another component...