Building plug-in system, looking for opinions

#1 The only plugin system I have written a plugin for was OBSE. You might take a look at that code to get some ideas. I think they used interface classes between classes in the plugin and classes in the engine. I don't know exactly how it hooked things together at runtime though. At least it would be a "working" example of a plugin "system".

You might also look at how Python does plugins. Python is written in C, but the entire "system" is highly object oriented. So its dynamic DLL (pyd) stuff might be interesting to look at.

#2 Coming from a T3D perspective, this is similar to something I was hoping to work on when I got the time. The main difference between this implementation idea, and the implementation I was thinking of; is that with what I was considering doing, the engine it self would be inside namespaces, and the components of the engine would be plugins as well. For example, physics as it's own plugin, rendering implementations as plugins, you get the idea.

Personally, I would put the core required parts of the engine (as in absolute core, not rendering/physics and the like) in it's own base DLL, and then have other components of plugins and the engine itself expand as libraries that interface with the core. I would also make absolutely sure that the implementations could be swapped if need be, and not hard coded to one specific implementation. If I wanted to plug in another rendering engine, I should be able to do that. Bitsquid's blog has a lot of really nice stuff on decoupling the engine.

One thing I would make absolutely sure to do however is make sure each plugin and component of the engine that will be a DLL has it's own namespace. One problem I've noticed with T3D is that the majority of the code from what I've seen isn't in namespaces. This will cause a lot of issues down the line involving plugins. Make absolutely sure that you namespace the various components of the engine.


Resources that would be useful:
bitsquid.blogspot.com/2011/01/managing-coupling.html
bitsquid.blogspot.com/2011/02/managing-decoupling-part-2-polling.html
bitsquid.blogspot.com/2011/02/some-systems-need-to-manipulate-objects.html
bitsquid.blogspot.com/2011/09/managing-decoupling-part-4-id-lookup.html

#3 I'm trying to avoid redesigning the entire engine. In my original post I mentioned a "game" branch but my current work in progress is actually in the plug-in branch - in case you'd like to see where I'm going with it.

My goal is to simply be able to extend console classes - mainly SimObject derivatives. Say for example that I was going to extend the Sprite class, I would like to make this a plug-in. Right now I have a manager and a plug-in interface class, but actually creating the SimObject child class is making me scratch my head.

#4 @Richard,
I am just not sure how to do that without bringing in large segments of the engine into the plugin. I was thinking there might be a way to create stub classes that interact with the real classes once linked into the engine. Otherwise you will be bringing one class in and have to compile and link every class that class touches.

Is there a standard way to stub class to the real class without losing significant functionality?

I did quick search and found this:
stackoverflow.com/questions/43322/whats-safe-for-a-c-plug-in-system
It sounds promising.

#5 Yes, my plug-in interface class does this, and the manager has access to it. But anything that descends from an engine-side class needs that heirarchy - I get unresolved everything in the link phase.

My options seem to be exporting a large amount of the engine (tweaks all over the place) or making a copy of the engine project with these exports to use for buiding the plug-in - from the plug-in side everything should be fine since the .dll should resolve at run-time with the live engine code. I'm still experimenting, though.

I'm leaning toward the second option at the moment - I just don't feel good modifying so much of the "real" engine like that. Exporting stuff really shouldn't have any impact on performance, but that's a lot of code to touch and there are probably subtle issues that I'm not aware of.

#6 @Richard,
I read through the link and the links inside that link. It talked about how C++ is unsuitable plugin systems due to language limitations. The sum total of the conversation seemed to indicate a plugin system would need to use abstract classes and do the plugin connection using C. One issue they talked about with C++ was brittle base class issues. Apparently you can extend the base class in the plugin, but if it changes in the host app then it will break the plugin classes. I don't know the technical aspects on this, but there is some good info in that link.

Here is a link inside that was really good:
www.drdobbs.com/cpp/building-your-own-plugin-framework-part/204202899?cid=RSSfee...

#7 Yeah, that pretty much aligns with other articles I've read on the topic.

However, given that the engine is in C++ and also given that a plug-in system would be handy for a number of reasons I'm left with building a plug-in system in C++.

As far as changing base classes in the engine breaking the .dll - well, you can't even honestly expect a .dll built with a different compiler to function with your executable reliably because of the lack of an ABI specification. So any change to the engine would probably require a new version of the plug-in (though maybe not if no interfaces or member variables changed). This is a good reason to lean toward building it directly into the engine - if I created a separate codebase to use as the basis for plug-in projects I'd have to maintain both. But that's really about my only reason for leaning this direction - as I said in my last post, I feel a little uncomfortable modifying about 2/3 of the engine to accommodate this feature.

#8 The OBSE plugin system allows for plugins to be used that were compiled against older versions of the API. The exception to this is the plugin cannot use new features it was not compiled with. This might include a different style of parameters.

So the ABI for OBSE is stable enough that even older plugins will still work with newer versions of the OBSE program.

It would be painful to get third party DLL code and have it only work with the version of Torque it was compiled with. So the third party vendor would need to provide many different versions. Then again, Python is this way so maybe it is workable.

#9 The whole thing is "painful" - lol. I'll look into OBSE and see if I can find a non-intrusive way to get it into the engine.

I also took a little detour this weekend and started looking at building T2D in Ubuntu, but I'm new to the environment and spent most of my time looking up how to get compilers configured, etc....

#10 Well I have written a couple of dll:s and made some plugin systems over the years. I'm not sure if I have understood the problem 100%

Usually all interface functions are declared as external C, so the DLL can be written and compiled in any language (supporting external C declaration). The way I did it was to simply provide the core header file and just design the DLL from there. Yes, if you make a change in the core, you need to update the DLL as well. I implemented a version system that could verify if a DLL was able to run the core version or not. Even to handle different versions if needed.

The way Unix and later on even Windows solved this, are with the COM architecture or DCOM for remote extension. For the caller there is no difference between COM and DCOM. This could be an interesting feature if you want the extension to stay on the server side. COM solves the problem with different OS and Endians for the various COM and DCOM objects. The interface is written in ATL to ensure all systems passes parameters in a standardized way that all can understand. For example DirectX in Windows is built with COM technology, you ask a class factory for a specific interface and version. This will ensure that older versions will be able to run on the never versions, hence you can run a DirectX9 game on a DirectX11 installation.

If this only seems like a lot of ranting to you, then I have probably not captured the actual problem and are describing something else (or you can have a look at COM and ATL to see what I'm talking about)... :P

#11 No, that sounds about like how my thought process went - I'm not sure how much shoe-horning it would take to get COM into the mix. I also want it to work on Mac and under iOS - and maybe Linux if my other experiments pan out - so I'm not totally sure on what path to take.

As far as external C functions, that's where my real dilemma lies - hence my question - in order to inherit from an engine-side class in the dll I need the engine-side class exported. The thing is, that's a ton of exports and pandemic modification of the engine.

#12 You could go the XPCOM route and that is cross platform.

@Max,
I think you stated much of what was used to solve this issue for Windows platforms. Not sure if COM/DCOM was used with *nix platforms though. Wasn't CORBA another approach?

@Richard,
I am not sure how to do this, but you might try something like this:

#include <T2D_interface.h>

class SimObject;  // dummy place holder?  not sure if this will work right

class FakeSimObject 
{
// constructor calls code to "connect" to a real simobject
FakeSimObject();
// a bunch of stub function variables like so
void (*someCall)(some vars);
};

// then you would instantiate a real simobject in the engine after the dll is imported via the FakeSimObject constructor
FakeSimObject::FakeSimObject()
{
  // interface function retrieves a SimObject
  SimObject* newSOPtr = interface::createSimObject();
  // connect function
  someCall = newSOPtr->someCall; // matching binary call?
}

What I don't understand is how the actual calls would be made to construct the object. Those interface functions would need to be populated against function pointers in the DLL and would need to be static calls I think. Also, the FakeSimObject might be better to be a struct rather than a class.

Yes, this looks like a pain in the arse, but this is pretty much what I have seen done. Maybe not exactly this way, though. Also, in this case the class to fake class calls might be finicky. Perhaps a separate C call layer would be needed. Maybe the component code in T3D can help with this.

Can the scripting engine code be used for some of this? Like have it wrap the object calls, but maybe not expose to script. Then have this interface be queried and C calls be built into the interface.h?

Another avenue that might give some insight is checking out what the SWIG folks have done. You might be able to use it to make a C to C++ interface for something like this. It does have the capability to be extended. It is basically solving the same issue, but added C/C++ to a scripting language like a plugin. At least for the Python stuff it is. It might be doing this for the C#/.NET interfaces as well.

#13 @Richard: There are a few issues that I see for getting plugins running in T2D


1.) C++ has no standard ABI, so different platforms and even different compilers can implement the binary interface differently. This leads to name mangling type issues at the very least. Also, there is no defined standard for how inheritance is implemented. Some might use VTables, etc.

2.) T2D builds as a standalone executable project, so there is nothing that plugins can link against.

The first isn't the worst issue to work around. Projects have done it many times. Take a look at this for a system that tries to make it easy to support different os/compilers for a c++ plugin system.

The 2nd issue is where we have most of our work ahead of us. Some good news is that I have a feeling T2D is going to turn into a shared library (dll) project to support editors. If that happens, then you can just have plugins link against the dll (.lib) for the engine and now you are most of the way there.

Once the engine is a shared module project, you just have to mark some classes for dll export (ConsoleObject/SimObject/etc). Just one tag added per class.

Plugins would use the engine headers, and link against the lib of the shared module, and now you can properly compile a plugin which can subclass something like SimObject.

Now, your host/plugin-manager still needs to be able to construct those new objects which you can't just new up across the plugin dll. A popular method for doing this is to have plugins implement a common factory constructor method exported as C. The factory can just return a void*, and implemented to just "return new PluginObject()". Your manager can then cast that to your plugin interface class and it should be wired up.

Of course, you can't call any custom methods on your plugin object, as the host has no idea what it really is. The good news is that T2D already has a solution for this with TorqueScript. Plugins can register their custom methods with the script interface, and now scripts can actually call those new methods that the host has no idea about.

I whipped up a quick example of how the structure would look if T2D was a dll, and a host running both that and a plugin that subclasses an object defined in the engine dll.

Find it here: https://docs.google.com/file/d/0B1Objq22-weMTldhZzZtWW9zOWc/edit?usp=sharing


Let me know if you have any questions.

#14 The best possible sulation to this is to look at competing engines.

There is a lot of open source engines that you can look at to see how they solved this very problem.

#15 @William,
Thanks for that explanation. That sounds like it may be a better approach than abstract classes.

Yes, you are right T2D will need to be a DLL, so, etc no matter what. I was actually going to work on that so I can turn T2D into a Python module. So I think that effort will benefit a lot of uses. I like the idea of doing what T3D did and make it so T2D can be a browser plugin as well.

#16 @William - That about sums up my take on how to do it, except for the "make the engine a dll" part. Like I was thinking, any way you do it there is no avoiding massive engine changes.

#17 @Richard,
You know, all this talk about plugins is making me rethink how I have done Python for T3D, and will do Python for T2D. If I make a dll that actually loads the dll T3D normally compiles to and the dll T2D will compile to, then I can make it so there are no dependencies on the python dlls in the code I write. There will be some minor modifications for getting into the interpreter, making function calls faster and the like, but it will not be tied to Python. Then I can add the interface code to the Github repo without introducing any Python dependencies. As it is now, even if I run T3D dll without explicitly importing into Python it still loads the Python dlls. This re-engineering of the interface will help T3D and T2D be "external interpreter ready". Also, this might be worth while to look at doing this like a plugin like you are are pursuing. I would like to continue this discussion with you over the next few months.

#18 Let me wave my hands a bit, since I have no experience in this area, but I do find the project interesting.

Have you looked into using/extending AbstractClassRep?

Almost every large C++ project gets to the point where it has a home-grown type system for reflection, connecting with scripts, etc. T2D is no exception. It has the AbstractClassRep et al.

If there turns out to be a way to go through this system, it may not be the shortest path, but it may be robust and it may reuse as much as possible.

#19 @Richard & Demolish : I'm pretty sure that T2D is going to get turned into a dynamic module instead of an executable. Editors are the hot item right now, and this is almost a must have to make the WYSIWYG editors work.

I wouldn't go down the path of doing this yourself yet, as you'll more then likely just be duplicating some work.

@Charlie, AbstractClassRep will have to be in the mix of course, but the main hurdle at the moment is getting a lib/obj that a plugin can link against to pull in base engine classes.

@Richard: It's been awhile, and not sure if this is cross platform but it might get you going short term. In Visual Studio, if you define anything as dll export in an exe project, it will also generate a lib file from that. You should be able to have your plugin link against that lib to subclass a base class in the engine.

#20

Quote:I wouldn't go down the path of doing this yourself yet, as you'll more then likely just be duplicating some work.

This is a problem. We can't be waiting for someone else to do anything anymore. There is only so much GG dev time to go around. They have also expressed needing help. So how would this duplicate work if someone else is saving someone else time that they can use to focus on other areas? Like the multiple rendering target issue. I can't help with the rendering stuff, but I sure can help tweak T2D to behave like T3D when it comes to acting like a DLL, and a plugin for a browser.

Edit:
I apologize if that was too strong. But I feel like we are the devs for Torque now and GG is merely lending their wisdom to help use out. That is how I see myself.