Using the library project as a static .lib
by Stephen McCloskey · in Torque Game Engine · 02/08/2003 (1:07 am) · 11 replies
I'm trying to use the library project as a static .lib and link into from my program. I got the thing to compile and link, but the program is choking up somewhere in the interpreter after the main.cs file is loaded. The game just enters the main loop without ever executing any of the .cs instructions. Perhaps this is some esoteric pointer bug in the interpreter code.
Has anyone encountered this before? Has anyone ever successfully used the library project instead of just compiling every bloody last line of code into their executable like the sample?
On another note, I notice that the debug executable created from linking to the library is about a meg. smaller than the debug executable created from sample project. Either I am doing something horribly wrong, or there is over a meg. of unused code floating around in the sample project. Note that I'm using the same preprocessors on both. Does anyone know of the performance ramifications of this?
Thanks,
Stephen
Has anyone encountered this before? Has anyone ever successfully used the library project instead of just compiling every bloody last line of code into their executable like the sample?
On another note, I notice that the debug executable created from linking to the library is about a meg. smaller than the debug executable created from sample project. Either I am doing something horribly wrong, or there is over a meg. of unused code floating around in the sample project. Note that I'm using the same preprocessors on both. Does anyone know of the performance ramifications of this?
Thanks,
Stephen
About the author
#2
02/09/2003 (7:05 pm)
I'm not giving up that easily. Does anyone have any idea how to fix this? I really believe that there is a lot of dead code in the demo app. Also, compiling everything into the .exe is also just a bad design. I am genuinely interested in doing the work here so that the rest of the community can benefit.
#3
You are pretty much on your own if you look at the past forum posts about this same thing.
The way it is now is actually more robust against cheating and other hacks that would be much easier to do and harder to without a monolothic build.
There is a lot of dead code and Tribes2 cruft in Torque. Getting rid of that code is a worth cause, making it linkable is not so much.
02/09/2003 (8:47 pm)
This is a religous debate that has sprung up on a regular basis. There are pros and cons for the way it is done. Most of the cons are theoretical at best all the pros are theoretical at worst. I none of the games that shipped using this engine have suffered in any practical way because of the way the build is done.You are pretty much on your own if you look at the past forum posts about this same thing.
The way it is now is actually more robust against cheating and other hacks that would be much easier to do and harder to without a monolothic build.
There is a lot of dead code and Tribes2 cruft in Torque. Getting rid of that code is a worth cause, making it linkable is not so much.
#4
Ummm.. what do you mean? Are you proposing linking against libs or using DLLs?
How is that?
02/10/2003 (4:25 am)
Quote:Also, compiling everything into the .exe is also just a bad design.
Ummm.. what do you mean? Are you proposing linking against libs or using DLLs?
Quote:The way it is now is actually more robust against cheating and other hacks that would be much easier to do and harder to without a monolothic build.
How is that?
#5
02/10/2003 (10:48 pm)
You completely misunderstood what I was saying. I'm not saying that the engine would be any better or worse if it were split into dlls. In fact I am not even talking about dlls. I'm talking about static libraries that you use during link-time. Yes, there is a lot of unused code in the engine. The easiest way to get rid of that unused code is to make the library project that actually ships with the game work! If you compile the code into a static library (not a dll) and then link into it when you build the bloody code, then most of that dead code will not be in your exe. I'm perfectly happy to have one big fat exe. I'm not happy to have an exe that has a meg of dead code in it. All I want to do is fix the project that shipped with the SDK...
#6
The first way saves a lot in the way of debugging and maintenance, the second way just saves a small bit of download time and disk space.
02/11/2003 (5:58 am)
It seems to me that the 'best' way to fix a dead code problem is to remove the dead code, not do a bunch of work so that the linker will keep it out of your final executable.The first way saves a lot in the way of debugging and maintenance, the second way just saves a small bit of download time and disk space.
#7
Seems there are enough people skilled in software design that a refactoring group could be formed. Identify the core functionality the codebase should provide, strip out what is not needed, ship.
I think what needs to happen is GG's must move from being a self funded game 'project' into being a self funded game 'company'.
A PROJECT has lots of contributors pulling in multiple directions donating this cool bit and that. The result 'Marble Blast.'
A COMPANY has a CEO to set direction, a CTO to lead the technical development, and a COO to ensure company operations support the stated mission.
It's the difference between being Slackware Linux and Red Hat, Inc.
02/11/2003 (7:04 am)
It's called refactoring.Seems there are enough people skilled in software design that a refactoring group could be formed. Identify the core functionality the codebase should provide, strip out what is not needed, ship.
I think what needs to happen is GG's must move from being a self funded game 'project' into being a self funded game 'company'.
A PROJECT has lots of contributors pulling in multiple directions donating this cool bit and that. The result 'Marble Blast.'
A COMPANY has a CEO to set direction, a CTO to lead the technical development, and a COO to ensure company operations support the stated mission.
It's the difference between being Slackware Linux and Red Hat, Inc.
#8
it was created solely for the purpose of building the various content pipeline tools that need access to some, not all, of the engine's functionality.
Now, nothing is stopping you or anyone else (some community members have done it, and use it in their own private projects) from creating your own, and using it.
Again, the library project as it is in the SDK is not meant for building games.
02/11/2003 (8:30 am)
The lib project is incomplete (ie it misses files) for a very simple reason : it was created solely for the purpose of building the various content pipeline tools that need access to some, not all, of the engine's functionality.
Now, nothing is stopping you or anyone else (some community members have done it, and use it in their own private projects) from creating your own, and using it.
Again, the library project as it is in the SDK is not meant for building games.
#9
There is probably a way to get it to run, but I just can't figure out how. My current theory is that there is some sort of pointer referencing issue in the interpreter, but I would love to be proven wrong.
If you know of anyone who got this to work, please let us know and provide the appropriate instructions. Several people are interested in getting the library to work.
02/13/2003 (10:57 am)
I figured out that the library project needed more files the first time I linked it into my client project. It was a brainless process to add them all. However, when you actually run the resulting executable, the interpreter code completely chokes and will not run. Note this is a runtime error not a compile time error. The problem is that the interpreter is extremely difficult to debug when it starts processing individual instructions.There is probably a way to get it to run, but I just can't figure out how. My current theory is that there is some sort of pointer referencing issue in the interpreter, but I would love to be proven wrong.
If you know of anyone who got this to work, please let us know and provide the appropriate instructions. Several people are interested in getting the library to work.
#10
I got this working but it's a bit of a hack at the moment. Let me preface this by saying I've only had the kit since Sunday and have only had a few hours here and there to mess around with it.
It also disturbed me that I had this nice "lib" sitting here and then had to recompile it all in the game itself.
So here's the scoop. The reason that the exe balks when you create a lib and use it, is because the dynamic objects it needs to run are missing. Why is this? Well in the lib itself anytime a function or data type is created it's done by using one of the IMPLEMENT macros. These will make a new dynamic object and place it into the system name table for resolution.
This is all well and good except for one thing... since your exe itself doesn't access these directly the linker thinks that you don't need to pull them in. Soooo they never get added to your exe and thus never get created and added to the engine. Thus you end up with none of the functions you need to actually execute script code. (The first runtime error you will run into is the fact that the "eval" function doesn't exist and eval is what's used to actually evaluate the first script file main.cs).
So in a hack and slash attempt to get a simple project consisting of main.cc and the torque lib I went nuts by dropping in a simple ForceLink global bool in each module where I knew the engine required the dynamic objects. I'll put a list at the end of this message. By simply dropping in a global bool in each of the modules and referencing it from main.cc I was able to "force" those modules to get loaded.
By doing this I was able to create a test.exe that is 500k smaller than the usual build (granted this is still a debug build) that runs the 3 default missions just fine.
I'm sorry if this doesn't explain everything as well as it could but no one had replied to you in some time so I thought I'd get back to you.
I have to agree with some others that this project layout is heavily in need of refactoring but with a little massaging you can load only the dynamic objects you need by forcing refs to those object modules and not others.
If you need more info let me know. I'm hoping after more than a couple days of owning this thing that I'll have a better solution to things like this.
I did this using the last release version (I haven't sync'd to the head yet). So your mileage may vary.
Files needed to be forced into the exe for basic functionality of the demo missions:
C:\torque\engine\audio\audioDataBlock.cc
C:\torque\engine\console\consoleFunctions.cc
C:\torque\engine\console\scriptObject.cc
C:\torque\engine\game\aiPlayer.cc
C:\torque\engine\game\camera.cc
C:\torque\engine\game\gameTSCtrl.cc
C:\torque\engine\game\guiNoMouseCtrl.cc
C:\torque\engine\game\item.cc
C:\torque\engine\game\missionArea.cc
C:\torque\engine\game\missionMarker.cc
C:\torque\engine\game\player.cc
C:\torque\engine\game\projectile.cc
C:\torque\engine\game\trigger.cc
C:\torque\engine\game\tsStatic.cc
C:\torque\engine\gui\guiChunkedBitmapCtrl.cc
C:\torque\engine\gui\guiConsoleEditCtrl.cc
C:\torque\engine\gui\guiControlListPopup.cc
C:\torque\engine\gui\guiFrameCtrl.cc
C:\torque\engine\gui\guiInputCtrl.cc
C:\torque\engine\gui\guiMessageVectorCtrl.cc
C:\torque\engine\sim\simPath.cc
C:\torque\engine\terrain\sky.cc
C:\torque\engine\terrain\sun.cc
C:\torque\engine\ts\tsShapeConstruct.cc
02/25/2003 (2:21 pm)
Stephen,I got this working but it's a bit of a hack at the moment. Let me preface this by saying I've only had the kit since Sunday and have only had a few hours here and there to mess around with it.
It also disturbed me that I had this nice "lib" sitting here and then had to recompile it all in the game itself.
So here's the scoop. The reason that the exe balks when you create a lib and use it, is because the dynamic objects it needs to run are missing. Why is this? Well in the lib itself anytime a function or data type is created it's done by using one of the IMPLEMENT macros. These will make a new dynamic object and place it into the system name table for resolution.
This is all well and good except for one thing... since your exe itself doesn't access these directly the linker thinks that you don't need to pull them in. Soooo they never get added to your exe and thus never get created and added to the engine. Thus you end up with none of the functions you need to actually execute script code. (The first runtime error you will run into is the fact that the "eval" function doesn't exist and eval is what's used to actually evaluate the first script file main.cs).
So in a hack and slash attempt to get a simple project consisting of main.cc and the torque lib I went nuts by dropping in a simple ForceLink global bool in each module where I knew the engine required the dynamic objects. I'll put a list at the end of this message. By simply dropping in a global bool in each of the modules and referencing it from main.cc I was able to "force" those modules to get loaded.
By doing this I was able to create a test.exe that is 500k smaller than the usual build (granted this is still a debug build) that runs the 3 default missions just fine.
I'm sorry if this doesn't explain everything as well as it could but no one had replied to you in some time so I thought I'd get back to you.
I have to agree with some others that this project layout is heavily in need of refactoring but with a little massaging you can load only the dynamic objects you need by forcing refs to those object modules and not others.
If you need more info let me know. I'm hoping after more than a couple days of owning this thing that I'll have a better solution to things like this.
I did this using the last release version (I haven't sync'd to the head yet). So your mileage may vary.
Files needed to be forced into the exe for basic functionality of the demo missions:
C:\torque\engine\audio\audioDataBlock.cc
C:\torque\engine\console\consoleFunctions.cc
C:\torque\engine\console\scriptObject.cc
C:\torque\engine\game\aiPlayer.cc
C:\torque\engine\game\camera.cc
C:\torque\engine\game\gameTSCtrl.cc
C:\torque\engine\game\guiNoMouseCtrl.cc
C:\torque\engine\game\item.cc
C:\torque\engine\game\missionArea.cc
C:\torque\engine\game\missionMarker.cc
C:\torque\engine\game\player.cc
C:\torque\engine\game\projectile.cc
C:\torque\engine\game\trigger.cc
C:\torque\engine\game\tsStatic.cc
C:\torque\engine\gui\guiChunkedBitmapCtrl.cc
C:\torque\engine\gui\guiConsoleEditCtrl.cc
C:\torque\engine\gui\guiControlListPopup.cc
C:\torque\engine\gui\guiFrameCtrl.cc
C:\torque\engine\gui\guiInputCtrl.cc
C:\torque\engine\gui\guiMessageVectorCtrl.cc
C:\torque\engine\sim\simPath.cc
C:\torque\engine\terrain\sky.cc
C:\torque\engine\terrain\sun.cc
C:\torque\engine\ts\tsShapeConstruct.cc
#11
For Marble Blast, we took the torque example, built our game (basically added just one main source file), and then, near the end of the project, slashed out all the code for sections we didn't use (mainly terrain). We then used UPX to slim the exe size down to about 1.15 megs.
As we move forward with the Torque Engine, we'll definitely look at making the modules more seperable and making it easier for people to cut out unused code.
02/25/2003 (3:24 pm)
Just for the GG $.02 - we definitely recommend new users to Torque using the default Torque Example project, and then adding and removing files. As was mentioned above, the Torque Lib project is there only for the purpose of linking against the 3DS Max exporter.For Marble Blast, we took the torque example, built our game (basically added just one main source file), and then, near the end of the project, slashed out all the code for sections we didn't use (mainly terrain). We then used UPX to slim the exe size down to about 1.15 megs.
As we move forward with the Torque Engine, we'll definitely look at making the modules more seperable and making it easier for people to cut out unused code.
Associate Tom Spilman
Sickhead Games