How well will T3D support multiple languages?
by ysun · in Torque 3D Professional · 06/22/2009 (5:49 pm) · 8 replies
Both TGE and TGEA didn't support multiple languages well though they used utf-8 encoding on characters. So how well will T3D support multiple languages? There are several ways to make multiple languages support, I will post 2 popular samples, hope GG can add multiple languages support in T3D V1.
1. FontGen style, used in Doom, Quake, RtCW, MOHAA, COD, SH, GTA, etc.
Tools: fontgen, fontrast, etc. Fontrast has better quality and supports non-english characters, it contains 2 parts, font image(s) and a lattice file.
The lattice file contains the unicode number and the coordinates of the characters on the font image(s).
Fontrast Website: http://hexforge.com/fontrast.shtml
2. Unicode Font style, used in COH, ETQW, etc.
Fonts: Microsoft Yahei, Arial Unicode MS, etc. Arial Unicode MS supports much more characters but it's commercial. COH and ETQW use this font in the games.
Arial Unicode MS Website: http://www.ascenderfonts.com/font/arial-unicode.aspx
Which way will GG use in T3D? The first way is font customizable but need a few steps to generate the font image(s) and a lattice file; the second way is limited to one font style but can be used directly.
1. FontGen style, used in Doom, Quake, RtCW, MOHAA, COD, SH, GTA, etc.
Tools: fontgen, fontrast, etc. Fontrast has better quality and supports non-english characters, it contains 2 parts, font image(s) and a lattice file.
The lattice file contains the unicode number and the coordinates of the characters on the font image(s).
Fontrast Website: http://hexforge.com/fontrast.shtml
2. Unicode Font style, used in COH, ETQW, etc.
Fonts: Microsoft Yahei, Arial Unicode MS, etc. Arial Unicode MS supports much more characters but it's commercial. COH and ETQW use this font in the games.
Arial Unicode MS Website: http://www.ascenderfonts.com/font/arial-unicode.aspx
Which way will GG use in T3D? The first way is font customizable but need a few steps to generate the font image(s) and a lattice file; the second way is limited to one font style but can be used directly.
About the author
Wanted by the police
#2
The font system basically does both of what you describe above. Individual glyphs are rendered to bitmaps and then used as textures. Bitmap data can come from user-generated glyph sheets like the one you've included above. The default operation, however, is to render glyphs from platform-installed fonts. So, as long as the specified font has the relevant Unicode characters mapped, Torque will be able to render Unicode text in the given script appropriately.
Leaving text rendering aside for a moment (which will most probably not be a problem in whatever language setting you put it in), I see more of an issue with text input. Torque's input system does not handle surrogate characters in most cases. This can also be seen in western language settings. In German, for example, trying to input an eacute character will not work as Torque will try to immediately act on the first keypress (the acute) and won't wait for the second keypress which only then forms a valid character.
I hope to address this in the near future and make Torque detect and buffer up characters that belong to compounds.
//Edit for clarification: for game input, handling of surrogate characters isn't needed or even desirable. For the game, you want the keypresses to register individually. It's solely text input in the GUI that is affected by this.
06/23/2009 (7:48 am)
There are a number places where due to the transition from ANSI to Unicode the UTF-8 handling in Torque is shaky (surrogate characters will cause problems) but the font and text rendering system generally should handle UTF-16 and thus internationalized text correctly.The font system basically does both of what you describe above. Individual glyphs are rendered to bitmaps and then used as textures. Bitmap data can come from user-generated glyph sheets like the one you've included above. The default operation, however, is to render glyphs from platform-installed fonts. So, as long as the specified font has the relevant Unicode characters mapped, Torque will be able to render Unicode text in the given script appropriately.
Leaving text rendering aside for a moment (which will most probably not be a problem in whatever language setting you put it in), I see more of an issue with text input. Torque's input system does not handle surrogate characters in most cases. This can also be seen in western language settings. In German, for example, trying to input an eacute character will not work as Torque will try to immediately act on the first keypress (the acute) and won't wait for the second keypress which only then forms a valid character.
I hope to address this in the near future and make Torque detect and buffer up characters that belong to compounds.
//Edit for clarification: for game input, handling of surrogate characters isn't needed or even desirable. For the game, you want the keypresses to register individually. It's solely text input in the GUI that is affected by this.
#3
I remember a while ago, in the TGE forums, someone hooked up the input system to IME and got it to properly accept chinese input. It might be worth doing a careful search for that (specially now that the private forums are open to everyone).
06/23/2009 (8:09 am)
Rene is right. The text rendering system fully supports unicode, so asian characters and even right-to-left languages are supported. But the input system needs changes to cope with even simple diacritics.I remember a while ago, in the TGE forums, someone hooked up the input system to IME and got it to properly accept chinese input. It might be worth doing a careful search for that (specially now that the private forums are open to everyone).
#4
@Manoel, I remember there was a post as you mentioned. I hope that issue could be resolved by GG in T3D officially.
06/30/2009 (11:06 am)
@Rene, you are quite right that we need game input. As far as I know the games on Steam platform has made the international input well enough, for GG's reference.@Manoel, I remember there was a post as you mentioned. I hope that issue could be resolved by GG in T3D officially.
#5
Thanks.
08/03/2009 (4:41 pm)
Is there any information that GG would make it out or not in the next releases? It's quite important to the international users.Thanks.
#6
The input system will likely not see any more significant changes for Torque 3D 1.0 but this will likely get addressed in the follow-up release.
I don't think the change required for the input system to properly deal with multi-key input is a drastic one. Basically, the code only needs to check whether a keypress corresponds to part of a surrogate character and then buffer up instead of immediately firing events.
Other than that, only the keycode mapping currently needs some looking into.
The rest, I think, should hold up pretty well in non-western locales.
08/03/2009 (4:56 pm)
The input system will likely not see any more significant changes for Torque 3D 1.0 but this will likely get addressed in the follow-up release.
I don't think the change required for the input system to properly deal with multi-key input is a drastic one. Basically, the code only needs to check whether a keypress corresponds to part of a surrogate character and then buffer up instead of immediately firing events.
Other than that, only the keycode mapping currently needs some looking into.
The rest, I think, should hold up pretty well in non-western locales.
#7
http://2war.freechal.com/Community/Pic_View.aspx?SeqNum=788
http://2war.freechal.com/Community/Pic_View.aspx?SeqNum=792
08/05/2009 (2:53 pm)
@ Rene, thanks for the reply. I saw the news about Torque Powered Games just now, there is a game made by a korean company, they seemed have realized the input feature(with the screenshots below), hope GG can added this feature to T3D in an earlier version after V1.0. That would help a lot to the indie studios which didn't have a coder. Thanks.http://2war.freechal.com/Community/Pic_View.aspx?SeqNum=788
http://2war.freechal.com/Community/Pic_View.aspx?SeqNum=792
#8
The real problem here is handling dead-keys, i.e. the multi-key inputs I mentioned above. Here more than one keypress is required to build up a valid input that can then be converted into Unicode. It depends on the keyboard layout whether this is used or not.
01/23/2010 (6:57 pm)
This is an old thread but just to clear this up: I was mistaken that this has anything to do with surrogate characters. Torque's input system will correctly map a key into whatever Unicode string it happens to generate.The real problem here is handling dead-keys, i.e. the multi-key inputs I mentioned above. Here more than one keypress is required to build up a valid input that can then be converted into Unicode. It depends on the keyboard layout whether this is used or not.
Torque Owner ysun
FPSCHINA.COM