1.4CR2 unicode not work correct

#1 Did you define TORQUE_UNICODE?

It is known to work at least to the extent that what shows up in the IME window is what appears in the console text box.

#2 Thanks for your reply,But I had added the word 'UNICODE' to the preprocessor definitions in the C/C++ General section and added 'IMM32.LIB' to the object/library modules in the 'Link' section. Clean and re-compile,the same error.

#3 And TORQUE_UNICODE is also defined?

What IME are you using?

#4 TORQUE_UNICODE is defined In the file torqueconfig.h,all IME I using has the same error.

#5 On the official TGE CVS check out, clean recompile, on Windows XP, using "Chinese (Simplified) - Microsoft Pinyin IME 3.0", I can properly input chinese characters. I input with the IME (typing ANSI characters "hongkong" followed by a space), and two chinese characters are placed into the console text box. For some reason the current font we have in there isn't generating unicode characters, but internally it is storing the exact correct characters that I entered with the IME, without the space as the IME accepts that as part of its input.

Checking the unicode codepoints in a font viewer, I can confirm that the characters that are stored match what I input.

We've tested with a fair number of other IMEs, and Torque does correctly get the input. It can't render RTL languages properly, yet, but hopefully we'll take care of that in an upcoming release.

Just to add - if there is an issue, I definitely want to resolve it! Maybe I'm missing something in my repro procedure, or you're using a different IME? I know some older ones have slightly broken behavior.

#6 Hello,
Do we have any progress on this matter?
I have the latest TGE (official 1.4) and input still won't work correctly.
In more detail:

Japanese Input:
Same problem as the Chinese input. When you type some characters, these characters output both on the torque as latin characters and they still get through the IME. If then I hit ENTER on the IME they will get in the Torque, resulting into having what I've inputed in both latin characters and Japanese.

Greek Input:
Characters input and display fine except for letters with accent. In Greek input in order to put the accent on the top of the letter you press the ; key before a vowel. Somehow Torque will not work properly and will put the ; character (actually ' in the Greek font) before the vowel.

For both Japanese and Greek input I use the native Windows XP IME, which is what I'm expecting from every player to have.
One solution could be to search in the code and make corrections for each of the input but the problem seems to be more general to me. Torque seems to override the native Windows input and use it's own, that's why this is happening. Any other program would just use the native windows input (whatever it is) and it would work normally. Does this make any sense?

I would be glad if you could give me any update on this, since I'd rather have a general fix for every input, rather than messing up the code, creating serious input bugs ^^;

Thank you

PS: Add to this that even when I hit ENTER on the Japanese input, the characters seems to pass from the IME to Torque into a strange messed order. (like typing "Hello there" in English and getting "the Herello"...)

#7 Hi George,

Status on our end is still the same as described in my last post... Roger never followed up with any more feedback, and since we can't get the problem to happen on my end, there's very little we can do. :(

Have you tried the latest in CVS?

I'd love to get this resolved; I'll need some debugging help from your end to make that happen. :)

Thanks,
Ben

#8 Hi,
thanks for replying so fast.
I'd be glad to help to get this resolved, just ask me whatever you need.
I put two self-explaining pictures that I hope they can give you some clues.
*** Images removed - PM me if you need them ***

{{ Mind that when I put that Greek accented character into Torque by copy, paste
from notepad I'd get a square, therefore it's not only an input problem. I also have
a problem of getting these characters displayed. I messed up a bit with some .cs
files and changed into "Arial Unicode MS" and the CHINESEBIG5 encoding into
UNICODE but that wouldn't work. }}
Got an improvement on this. If I edit the .gui file (outside Torque) I get diplayed what
I want (including the Greek accented letters). But further than that I also
want to be able to do that from within the engine anyway (chat capabilities).

As for the Torque version I've got the one that you can download from the website,
which I guess is the official 1.4.
Is there out a newer one?
If yes what should be the CVS checkout settings to get it?
Thank you

#9 Can you please tell me what exact IME you are using, so that I can make sure I'm testing against the right one? Based on the D3D examples, it seems that some IMEs vary from the spec in strange ways, and require workarounds, so I want to make sure I'm not testing with an IME that's broken in a different way than yours. ;)

Thank you for following up! I am really glad we are getting rid of bugs! :)

#10 Hello,
Happy to see your reply.
I'm on vacation now. I'll get back 2 weeks later so I'll check and let you know then.

#11 Hello,
I'm back so I can give you some information now.
I've Windows XP Pro + Service Pack 2 (English) installed and I'm updating them regurly.
For both Greek & Japanese I'm using the Windows build-in IME.
That's Microsoft IME Standard 2002 v8.1 and Microsoft Natural Input 2002 v8.1 for Japanese.
For Greek it just say Greek...
Let me know what else you need.
Thank you

#12 Ah, great, thanks for getting back to this! Hmm, ok, I'll confirm that we're using the same one on our end and see if we can reproduce. Can you also record a short journal with -jSave that reroduces the problem and e-mail it to me?

#13 I am getting the same issue of squares instead of accented characters when using localisation files. The accents show up in the original text and the .lso compiled file fine but are squares in the engine itself.

EDIT 1
As a follow up - I exported the font image strips and there is a small square where the accented letters should be - if I draw the correct letter into the strip and import back again it works fine (not that I want to do that for all the font strips :S )

EDIT 2
Success - I need to save the original text files containing the translated localisation files as UTF8 format before I run the through langc and then all the accented characters work fine.

#14 I found the same problem! the key I press is show in the console box. After I press the space key, the chinese word also show in the console box, but it is reverse.

#15 @David: Glad you got it working. Future versions of our tools will have better checking that things are in the right format.

@Fish: Please tell me what IME you are using, and send me a journal of the problem as well (see previous post for instructions).

#16 Thanks ben!
the error just look like this image. I use Microsoft pingyin IME.

#17 I have found the reason!
Because I use chinese windows OS. I try in the w2k professional english version, it work in gear.
I think TGE develope team should test in chinese windows OS!

#18 Using TGE 1.4 (with TLK). I encounter the "reverse display" problem using IME MS Pinyin 3.0). I see just the Chinese characters (no keys) but order is reversed.

I wasn't able to input Chinese characters into the console text initially, but was able to do so after I modify the associated GUI profile (GuiTextEditProfile) to use a chinese fonttype ( fonttype = "SimHei").

screenshots

1. IME input

2. Reversed output in console

Anyone?

#19 The screenshots are not showing up for me...

If anyone has a copy of the chinese version of windows they want to send me, along with detailed instructions on how to use it (since I don't speak chinese), I can see about getting QA to use it.

What version of windows are you guys using? XP does not care about the fonttype insofar as I was able to determine.

#20 Oops, the images links expired after X downloads. Have reloaded at a permanent location.
See edited prev post.

Ben, I'm using an English version of XP. The 2 simplified chinese characters stand for one,two (just type yier into the IME).

BTW, thanks for explaining BOM here