Game fails to run with double-byte characters in directory path name
by Aisle 5 Games, Inc. · in Torque Game Builder · 01/16/2010 (5:05 pm) · 2 replies
We've localized our game for Japanese and it works unless the game is installed in a path containing Japanese characters. Then it fails to start and displays an error.
console.log shows this:
--
setModPaths: invalid mod path directory name: 'common'
Missing file: C:/Aisle5Games/?/common/main.cs!
setModPaths: invalid mod path directory name: 'game'
Missing file: C:/Aisle5Games/?/game/main.cs!
--
The ? in the path name is a Japanese character, so I think I've traced the problem to WinFileio.cc:
StringTableEntry Platform::getCurrentDirectory()
{
char cwd_buf[2048];
GetCurrentDirectoryA(2047, cwd_buf);
forwardslash(cwd_buf);
return StringTable->insert(cwd_buf);
}
where GetCurrentDirectoryA is the Ansi version of the C++ function. There is also a call to SetCurrentDirectoryA and the use of "char", "backslash()", "forwardslash()", etc, all of which are used with single-byte characters.
As an added bonus, if the "User" part of "C:/Documents and Settings/User/ApplicationData/GameName..." is in Japanese, the game fails to save.
How can this be rewritten to handle Japanese (or any double-byte) characters in the path names?
I've seen similar posts asking for help on this with no useful (if any) replies.
console.log shows this:
--
setModPaths: invalid mod path directory name: 'common'
Missing file: C:/Aisle5Games/?/common/main.cs!
setModPaths: invalid mod path directory name: 'game'
Missing file: C:/Aisle5Games/?/game/main.cs!
--
The ? in the path name is a Japanese character, so I think I've traced the problem to WinFileio.cc:
StringTableEntry Platform::getCurrentDirectory()
{
char cwd_buf[2048];
GetCurrentDirectoryA(2047, cwd_buf);
forwardslash(cwd_buf);
return StringTable->insert(cwd_buf);
}
where GetCurrentDirectoryA is the Ansi version of the C++ function. There is also a call to SetCurrentDirectoryA and the use of "char", "backslash()", "forwardslash()", etc, all of which are used with single-byte characters.
As an added bonus, if the "User" part of "C:/Documents and Settings/User/ApplicationData/GameName..." is in Japanese, the game fails to save.
How can this be rewritten to handle Japanese (or any double-byte) characters in the path names?
I've seen similar posts asking for help on this with no useful (if any) replies.
#2
01/19/2010 (1:50 pm)
Thanks a bunch for this thread. We've implemented a number of unicode fixes ourselves, not sure if this was part of them. I'll get back to you if I find anything else. 
Aisle 5 Games, Inc.
Default Studio Name
Unicode bug? Non-English characters in install path fails
They fixed Platform::getWorkingDirectory which has since been supplanted by Platform::getCurrentDirectory().
I implemented the changes to Platform::getCurrentDirectory() suggested by the author and... the game starts in folders with double-byte characters!
amaranthia also reported in 2008 the 2nd part of my complaint here Won't write to appdata for some users
I modified winUser.cc getUserDataDirectory() and getUserHomeDirectory() following the same pattern above and the game now recognizes Windows User names in Japanese.
Here are my changes:
winFileio.cc
#ifdef UNICODE // ...please see original post for MVJ comments... StringTableEntry Platform::getCurrentDirectory() { static StringTableEntry cwdUTF8 = NULL; if (!cwdUTF8) { // Microsoft Windows 2000/XP/2003/Vista/CE and future, are native UTF-16 char cwd_buf_UTF16[(MAX_PATH+1)*2]; // max windows path length 260 TCHAR GetCurrentDirectoryW((MAX_PATH+1), (UTF16*)cwd_buf_UTF16); // W variant is the UTF-16 function char cwd_buf_UTF8[(MAX_PATH+1)*4]; convertUTF16toUTF8((UTF16 *)cwd_buf_UTF16, (UTF8 *)cwd_buf_UTF8, sizeof(cwd_buf_UTF8)); forwardslash(cwd_buf_UTF8); // works transparently on UTF-8 encoded text (probably not on UTF-16) cwdUTF8 = StringTable->insert(cwd_buf_UTF8); } return cwdUTF8; } #else StringTableEntry Platform::getCurrentDirectory() { static StringTableEntry cwd = NULL; if (!cwd) { char cwd_buf[2048]; GetCurrentDirectoryA(2047, cwd_buf); forwardslash(cwd_buf); cwd = StringTable->insert(cwd_buf); } return cwd; } #endifand in WinUser.cc
#include "core/unicode.h" . . . #ifdef UNICODE const char *Platform::getUserDataDirectory() { static StringTableEntry uddUTF8 = NULL; if (!uddUTF8) { // Microsoft Windows 2000/XP/2003/Vista/CE and future, are native UTF-16 char udd_buf_UTF16[(MAX_PATH+1)*2]; // max windows path length 260 TCHAR if(! SHGetSpecialFolderPathW( NULL, (UTF16*)udd_buf_UTF16, CSIDL_APPDATA, true ) ) return ""; char udd_buf_UTF8[(MAX_PATH+1)*4]; convertUTF16toUTF8((UTF16 *)udd_buf_UTF16, (UTF8 *)udd_buf_UTF8, sizeof(udd_buf_UTF8)); char *ptr = udd_buf_UTF8; while(*ptr) { if(*ptr == '\') *ptr = '/'; ++ptr; } uddUTF8 = StringTable->insert(udd_buf_UTF8); } return uddUTF8; } #else const char *Platform::getUserDataDirectory() { char szBuffer[512]; if(! SHGetSpecialFolderPathA( NULL, LPSTR( szBuffer ), CSIDL_APPDATA, true ) ) return ""; char *ptr = szBuffer; while(*ptr) { if(*ptr == '\') *ptr = '/'; ++ptr; } return StringTable->insert(szBuffer); } #endif #ifdef UNICODE const char *Platform::getUserHomeDirectory() { static StringTableEntry uhdUTF8 = NULL; if (!uhdUTF8) { // Microsoft Windows 2000/XP/2003/Vista/CE and future, are native UTF-16 char uhd_buf_UTF16[(MAX_PATH+1)*2]; // max windows path length 260 TCHAR if(! SHGetSpecialFolderPathW( NULL, (UTF16*)uhd_buf_UTF16, CSIDL_PERSONAL, false ) ) if(! SHGetSpecialFolderPathW( NULL, (UTF16*)uhd_buf_UTF16, CSIDL_COMMON_DOCUMENTS, false ) ) return ""; char uhd_buf_UTF8[(MAX_PATH+1)*4]; convertUTF16toUTF8((UTF16 *)uhd_buf_UTF16, (UTF8 *)uhd_buf_UTF8, sizeof(uhd_buf_UTF8)); char *ptr = uhd_buf_UTF8; while(*ptr) { if(*ptr == '\') *ptr = '/'; ++ptr; } uhdUTF8 = StringTable->insert(uhd_buf_UTF8); } return uhdUTF8; } #else const char *Platform::getUserHomeDirectory() { char szBuffer[512]; if(! SHGetSpecialFolderPathA( NULL, LPSTR( szBuffer ), CSIDL_PERSONAL, false ) ) if(! SHGetSpecialFolderPathA( NULL, LPSTR( szBuffer ), CSIDL_COMMON_DOCUMENTS, false ) ) return ""; char *ptr = szBuffer; while(*ptr) { if(*ptr == '\') *ptr = '/'; ++ptr; } return StringTable->insert(szBuffer); } #endifHopefully, I didn't break anything in the process. Please comment if this breaks your code....