Game Development Community

How to recompile scan.l ? (to add new string concat operator)

by Orion Elenzil · in Torque Game Engine · 12/04/2007 (10:46 am) · 9 replies

Hi All,
i'm thinking of adding another script concatenating operator like SPC, @, TAB, and NL,
but can't find where those guys are implemented.

.. oh man,
i think it's in console/gram.y,
which forget it.

#1
12/04/2007 (10:56 am)
Yeah - it's in the lexer - look in CMDscan.l.
#2
12/04/2007 (1:44 pm)
Thanks Andy!

next Q,
where does one get [the right version of] lex to recompile scan.l ?
judging from this comment at the top of scan.cc: /* lex -a -P console\yylex.c -p CMD -o console\scan.cc console\scan.l */, scan.cc was generated via DOS, not linux.

i tried that command line on a linux box, lex version 2.5.4,
but it says that -a is an unknown option.

also, while i'm at it,
does adding the following line look like it'll do what i want it to do ?
"SPC"       {CMDlval.i = ' '; return '@'; }
[b]"QUOTE"     {CMDlval.i = '"'; return '@'; }[/b] <---- the new line

i think the other possibilities would be
[code]
"QUOTE" {CMDlval.i = '\"'; return '@'; }
"QUOTE" {CMDlval.i = '\\"'; return '@'; }
[code]
#3
12/04/2007 (3:12 pm)
I think TGE includes flex in the bin directory.

I don't have the TGE stuff handy, but in T2 we have a generateCompiler.bat file to regenerate the appropriate files. It looks like this [you'll have to modify paths etc to fit, but you get the idea]:

echo Generating CMDgram.c and CMDgram.y with prefix CMD
..\..\..\bin\bison\bison.exe -o CMDgram.c CMDgram.y --defines -p CMD
echo Renaming CMDgram.c to CMDgram.cpp
move /Y CMDgram.c cmdgram.cpp

..\..\..\bin\flex\flex -PCMD -oCMDscan.cpp CMDscan.l

I think your first one should be correct because " is just a single char.
#4
12/04/2007 (3:37 pm)
Ah - i see things changed a bit from 1.3 -> 1.4 in this department, and i'm based on 1.3.

in 1.3, the files don't have the "CMD" prefix,
and also flex wasn't in the bin directory.
(it is in 1.4)

so this should be do-able.

thanks again.
#5
12/04/2007 (3:57 pm)
I think you want to escape the quote:
"QUOTE" {CMDlval.i = '\"'; return '@'; }
#6
12/05/2007 (1:25 pm)
Erg,
the bison & flex utilities from 1.4 create pretty radically different versions of scan.cc and gram.cc than the scan.cc & gram.cc included in 1.3.

i got the resulting scan.cc to compile, but not link.

i also tried bringing over the CMDgram etc files from 1.4,
but it's pretty much a festival of cryptic compile errors after that.

any chance the flex/bison in use for TGE 1.3 are around somewhere ?
#7
12/05/2007 (1:26 pm)
Also i think "engine/console/bison.simple" may be needed to recompile gram.y
#8
12/05/2007 (2:32 pm)
The scan/gram.cc from 1.3 used a really, really, really old version of flex and yacc. It's also worth noting that the scan/gram.cc in 1.3 and below was hacked by hand in places. Thus, it's probably impossible to actually find anything that will get you a working scan/gram.cc for 1.3 without merging in 1.4's console.

Your best bet is to merge the console dir up to 1.4 and maybe some platform stuff if needed. When it comes to the scan/gram.cc files, don't bother merging the .cc files directly, just merge and recompile the grammar. It's not really that bad unless you've done something retarded like modify the scan/gram.cc by hand ;)

T.
#9
12/05/2007 (3:28 pm)
Heh.

seeing as how none of 1.3's scan.cc, gram.h, and gram.cc contain the string "SPC",
hacking them up seems prohibitive to me.

also, i think my world can live without the "QUOTE" string concatting operator.
- this has just been a lunchtime project so far,
but merging over to 1.4's console is definitely out of that scope.

many thanks for the illumination, folks !