Game Development Community

dev|Pro Game Development Curriculum

Komodo Edit TorqueScript Plugin (Mac/PC/Linux)

by Thomas -elfprince13- Dickerson · 02/01/2012 (11:57 am) · 10 comments

Anyone who's developed for Torque on OS X knows how limited our options have been as far as a good programmer's text-editor for composing Torque Script files. Komodo Edit is a fantastic (and free, though it's IDE cousin is not) editor based on Scintilla, Python, and Mozilla. It supports both extensible syntax highlighting, extensible linting (syntax checking) and extensible code-sense (autocomplete). After struggling to understand the User-Defined Language architecture - it's limited to Cascading Style Sheet, Client-Side Languages, Server-Side Languages, and Markup type languages, and the state machine for the lexer evidently begins in Markup mode, and to get the SDK working on both Lion (PPC only binaries) and Vista (VirtualStore is the worst thing I've ever encountered in what's supposed to be a useable operating system), I have produced a working plugin. At the moment it only supports the UDL/syntax-highlighting portion of the features mentioned above, but I plan to keep hacking away at it. Also, for whatever reason SSLs have one additional color available as compared to CSL.

If you just want to download it, click here and then drag-and-drop the downloaded file into the main editing window of Komodo Edit (or IDE), approve the installation, and restart. Then change the file associations and coloring to fit your needs and have fun.


Here we have examples of two different color schemes:

img546.imageshack.us/img546/2209/screenshot20120201at228.png
img59.imageshack.us/img59/2595/screenshot20120201at258.png
A couple things to note:
  • Komodo Edit supports C# development, so you'll have to change the file-extension/language association for *.cs in the preferences. You can use this as an opportunity to add associations for *.gui and *.mis as well
  • The default color-scheme doesn't do anything special for variables or named identifiers, but you can change this to suit your fancy
  • I developed this under Komodo 6.1, but version 7 was released in the last couple days, so I'll be upgrading to that and testing it out there as soon as possible
  • For whatever reason, variable names with only a single character aren't lexed as variable tokens. Also, the array[index] syntax can confuse the syntax highlighting in certain scenarios. If you want to take a look at, ponder, and comment on the regex/state-machine code which is used for the lexing, I've embedded it below. It's derived from the example code the Komodo developers provide for JS and PHP lexers, and incorporates TorqueScript specific features which I've extracted
  • Tagged (single quoted) strings right now are color coded as regex tokens, since TS doesn't support regex syntax, by default. If you've added that somehow to your installation, let me know and it might motivate me to color it using the markup-tag colors instead.
  • I've also added rudimentary auto-indent/dedent code.



[edit]

The forum software breaks code with backslashes. Click here if you want to take a look at the lexer code.

About the author

C.S. PhD student at Brown University. Project lead for FreeBuild. Administrator, Cemetech tech community. Webmaster for the Village2Village Projects and the Vermont Sustainable Heating Initiative.


#1
02/01/2012 (10:36 pm)
The IDE uses Comic Sans? Ok, maybe not actually, but it looks pretty darn close...
#2
02/01/2012 (11:14 pm)
Monaco. Like Courier, but it doesn't hurt your eyes ;)

Also, this afternoon/evening I've been working on the code-completion portion of the plugin, which requires something that can parse Torquescript. Being the lazy fiend that I am, I found a byacc distribution that claims to output Python (the plugin language) and grabbed a copy CMDgram.y and ast.h. 4 hours later...


%{
 /* Reserved Word Definitions */
import astnodes
from astnodes import *
debug = 0                
        
%}
%token rwDEFINE rwENDDEF rwDECLARE rwDECLARESINGLETON
%token rwBREAK rwELSE rwCONTINUE rwGLOBAL
%token rwIF rwNIL rwRETURN rwWHILE rwDO
%token rwENDIF rwENDWHILE rwENDFOR rwDEFAULT
%token rwFOR rwDATABLOCK rwSWITCH rwCASE rwSWITCHSTR
%token rwCASEOR rwPACKAGE rwNAMESPACE rwCLASS
%token rwASSERT
%token ILLEGAL_TOKEN
%{
 /* Constants and Identifier Definitions */
%}
%token CHRCONST
%token INTCONST
%token TTAG
%token VAR
%token IDENT
%token TYPE
%token DOCBLOCK
%token STRATOM
%token TAGATOM
%token FLTCONST

%{
 /* Operator Definitions */
%}
%token '+' '-' '*' '/' '<' '>' '=' '.' '|' '&' '%'
%token '(' ')' ',' ':' ';' '{' '}' '^' '~' '!' '@'
%token opINTNAME opINTNAMER
%token opMINUSMINUS opPLUSPLUS
%token STMT_SEP
%token opSHL opSHR opPLASN opMIASN opMLASN opDVASN opMODASN opANDASN
%token opXORASN opORASN opSLASN opSRASN opCAT
%token opEQ opNE opGE opLE opAND opOR opSTREQ
%token opCOLONCOLON

%left '['
%right opMODASN opANDASN opXORASN opPLASN opMIASN opMLASN opDVASN opMDASN opNDASN opNTASN opORASN opSLASN opSRASN '='
%left '?' ':'
%left opOR
%left opAND
%left '|'
%left '^'
%left '&'
%left opEQ opNE
%left '<' opLE '>' opGE
%left '@' opCAT opSTREQ opSTRNE
%left opSHL opSHR
%left '+' '-'
%left '*' '/' '%'
%right '!' '~' opPLUSPLUS opMINUSMINUS UNARY
%left '.'
%left opINTNAME opINTNAMER

%%

start:
decl_list {
        pass
};

decl_list:
{
        $$ = None
} | decl_list decl {
        if not astnodes.gStatementList:
                astnodes.gStatementList = []
        else:
                astnodes.gStatementList.append()
};

decl:
stmt {
        $$ = 
} | fn_decl_stmt{
        $$ = 
} | package_decl
{
        $$ = ;
};

package_decl:
rwPACKAGE IDENT '{' fn_decl_list '}' ';' {
        $$ = 
        for walk in ():
                walk.set_package()
};

fn_decl_list:
fn_decl_stmt {
        $$ = []
} | fn_decl_list fn_decl_stmt {
        $$ = 
        ().append()
};

statement_list:
{
        $$ = None
} | statement_list stmt {
        if not :
                $$ = []
        else:
                ().append()
                $$ = ;
};

stmt
   : if_stmt
   | while_stmt
   | for_stmt
   | datablock_decl
   | switch_stmt
   | rwBREAK ';' {
        $$ = BreakStmtNode()
} | rwCONTINUE ';' {
        $$ = ContinueStmtNode()
} | rwRETURN ';' {
        $$ = ReturnStmtNode(None)
} | rwRETURN expr ';' {
        $$ = ReturnStmtNode()
} | expression_stmt ';' {
        $$ = ;
} | TTAG '=' expr ';' {
        $$ = TTagSetStmtNode(, , None)
} | TTAG '=' expr ',' expr ';' {
        $$ = TTagSetStmtNode(, , )
} | DOCBLOCK {
        $$ = StrConstNode(, False, True)
};

fn_decl_stmt:
rwDEFINE IDENT '(' var_list_decl ')' '{' statement_list '}' {
        $$ = FunctionDeclStmtNode(, None, , )
} | rwDEFINE IDENT opCOLONCOLON IDENT '(' var_list_decl ')' '{' statement_list '}' {
        $$ = FunctionDeclStmtNode(, , , )
};

var_list_decl:
{
        $$ = []
} | var_list {
        $$ = 
};

var_list:
VAR {
        $$ = [VarNode(, None)]
} | var_list ',' VAR{
        $$ = 
        ().append(VarNode(, None))
};

//------------------------
//  ...ETC...
//-----------------------


The entirety is here if anyone is curious.

This compiles without error to Python, and I can import the resulting module after I rename it to ytab.py (from y.tab.py which is un-importable because of the name) but since I have yet to hook up a lexer to it, I can't be sure it works. Also, I think the compiled version has some problems with the way it abuses scope, so I may need to carpet-bomb the function generation code in the byacc source with lists of globals and recompile that as well. But for now, it's sleep time.
#3
02/01/2012 (11:30 pm)
Gah. I'd honestly rather be able to embed iFrames from Pastebin than deal with the vagaries of this forum software. Every phpBB, SMF, and Invision installation in the world can display code just fine, so why is a website by-and-for programmers incapable of doing so correctly?

This has been a problem for literally the entire time I can remember posting on GarageGames, will someone from the web team not spend the 5 minutes it should take to fix the problem?
#4
02/02/2012 (5:15 am)
Nice work. Seeing Python in flex code is a bit odd, but it's good to know that's out there!
#5
02/02/2012 (7:45 am)
Nice, i will be nice to move from JEdit and it half working plugin TIDE when this is finished
#6
02/02/2012 (12:37 pm)
Thanks Glenn.

Daniel: It was damn hard to find. For reference, I'm using perl-byacc 1.8.2, which has an undocumented Python output flag, took me about 30 minutes to track it down while I was googling about different parser options for Python.
#7
02/04/2012 (6:23 pm)
I never knew this, but with TorqueScript you can put a type specifier before the field declarations within an object, or datablock declaration. Weird, huh? Anyway, the parser needs to know what types are available to the console, in order to distinguish between them and a normal identifier, so here's some code to dump a python dictionary matching type names to their numeric IDs to the console. I put mine in console/dynamicTypes.cpp

ConsoleFunction(dumpConsoleTypes, void, 1, 1, "dumpConsoleTypes")
{
    Con::printf("gConsoleTypeTable = {");
    ConsoleBaseType *walk = ConsoleBaseType::getListHead();
    while( walk != NULL )
    {
        Con::printf("    '%s' : %d,", walk->getTypeName(), walk->getTypeID());        
        walk = walk->getListNext();
    }
    Con::printf("}");
}
#8
04/10/2013 (9:54 am)
Old tread this is, but some questions here, I have used Komodo Edit 7 for some time doing the .css, .php and other web oriented stuff but never really created a Torque project in it so... How do I get a Komodo Torque project like in Torsion done?

Sorry for vamping a old thread :o)
#9
04/10/2013 (9:57 am)
I have no project management in the plugin yet. Create a project using the Torque Toolbox, and then edit it from Komodo. Incidentally, Komodo 8 is out.
#10
04/10/2013 (11:44 am)
Okay thanks for answering. I actually edit in Kommodo and compile/run it i torsion at this moment. I drop the pluin above in the editor and everything installed perfectly.

I really like the dark color theme in Komodo 7 so I use it when doing some long crunch hours in order to spare my eyes :o)