Asynchronous MySQL++ for TGEA 1.8.x
by Tony Richards · 04/05/2009 (8:10 am) · 37 comments
This resource is a sequel to my Event Driven Database resource, except written specifically for TGEA 1.8.x. I've continued using mysql++ in this example, but it should not be terribly difficult to modify for use with other databases, or even with other service oriented architectures.
This resource ended up being a lot simpler than what I had originally anticipated, primarily because TGEA now supports script callbacks from threads other than the main render thread.
I started by copying the ProjectsTemplate folder into ProjectsMySQLDemo.
Then I renamed Template.sln to MySQLDemo.sln, and Template.vcproj to MySQLDemo.vcproj. When you open MySQLDemo.sln, you'll have to remove the Template project and add the MySQLDemo project (and rename it as necessary).
Next, download mysql++. I downloaded the source and placed it in ProjectsMySQLDemo/mysql++, and then I added the mysqlpp project to my solution.
In the mysqlpp project properites, under C/C++, Code Generation, make sure you change Runtime Library to match Torque (which is normally Multi-threaded and Multi-threaded Debug).
Next, make sure you add the MySQL header location to C/C++, General, Additional Include Directories. You'll need to do this to both mysqlpp and MySQLDemo projects. Also, add the location of the mysql++ headers to the MySQLDemo project.
Mine looked like this:
It's best if you go ahead and make these adjustments to Debug, Release and Optimized configurations while it's fresh on your mind.
Also, right-click on MySQLDemo and click on Project Dependencies... and add mysqlpp project as a dependency (actually, add all of the projects as a dependency to the MySQLDemo project).
Next, in main.cpp include "Database.hpp" and add Database::init();.
Now, add these four files:
Database.hpp:
Database.cpp
RecordSet.hpp
And RecordSet.cpp
You might want to do conditional compiles so that this only gets compiled when you're compiling your dedicated server.
And finally, in scriptAndAssets/server/init.cs, either in initServer() or initDedicated() (depending on if you want this only for dedicated servers or if you want this for all servers)
And then in that same directory, add database.cs file:
So there you go. Torque + mysql++, including multi-threading asynchronous database requests and a scriptable record set.
Enjoy!
This resource ended up being a lot simpler than what I had originally anticipated, primarily because TGEA now supports script callbacks from threads other than the main render thread.
I started by copying the ProjectsTemplate folder into ProjectsMySQLDemo.
Then I renamed Template.sln to MySQLDemo.sln, and Template.vcproj to MySQLDemo.vcproj. When you open MySQLDemo.sln, you'll have to remove the Template project and add the MySQLDemo project (and rename it as necessary).
Next, download mysql++. I downloaded the source and placed it in ProjectsMySQLDemo/mysql++, and then I added the mysqlpp project to my solution.
In the mysqlpp project properites, under C/C++, Code Generation, make sure you change Runtime Library to match Torque (which is normally Multi-threaded and Multi-threaded Debug).
Next, make sure you add the MySQL header location to C/C++, General, Additional Include Directories. You'll need to do this to both mysqlpp and MySQLDemo projects. Also, add the location of the mysql++ headers to the MySQLDemo project.
Mine looked like this:
../../../mysql++/lib C:/Program Files/MySQL/MySQL Server 5.0/include
It's best if you go ahead and make these adjustments to Debug, Release and Optimized configurations while it's fresh on your mind.
Also, right-click on MySQLDemo and click on Project Dependencies... and add mysqlpp project as a dependency (actually, add all of the projects as a dependency to the MySQLDemo project).
Next, in main.cpp include "Database.hpp" and add Database::init();.
...
#include "Database.hpp"
...
S32 TorqueMain(S32 argc, const char **argv)
...
// Initialize the subsystems.
StandardMainLoop::init();
// Handle any command line args.
if(!StandardMainLoop::handleCommandLine(argc, argv))
{
Platform::AlertOK("Error", "Failed to initialize game, shutting down.");
return 1;
}
Database::init();
// Main loop
while(StandardMainLoop::doMainLoop());
// Clean everything up.
StandardMainLoop::shutdown();
...Now, add these four files:
Database.hpp:
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
// Zen Worlds
//
// Copyright (C) 2001 - 2009 Tony Richards
//
// This software is provided 'as-is', without any express or implied
// warranty. In no event will the authors be held liable for any damages
// arising from the use of this software.
//
// Permission is granted to anyone to use this software for any purpose,
// including commercial applications, and to alter it and redistribute it
// freely, subject to the following restrictions:
//
// 1. The origin of this software must not be misrepresented; you must not
// claim that you wrote the original software. If you use this software
// in a product, an acknowledgment in the product documentation would be
// appreciated but is not required.
// 2. Altered source versions must be plainly marked as such, and must not be
// misrepresented as being the original software.
// 3. This notice may not be removed or altered from any source distribution.
//
// Tony Richards trichards@indiezen.com
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
#ifndef ZEN_WORLDS_ZGARAGE_DATABASE_HPP_INCLUDED
#define ZEN_WORLDS_ZGARAGE_DATABASE_HPP_INCLUDED
#include "core/util/tVector.h"
#include "platform/event.h"
#include "platform/threads/thread.h"
#include <mysql++.h>
#include "RecordSet.hpp"
struct DatabaseRequest
{
const char* pSQLString;
const char* pQueryCallback;
RecordSet* pRecordSet;
};
struct DatabaseReply
{
const char* pQueryCallback;
RecordSet* pRecordSet;
};
class Database
{
/// @name 'Structors
/// @{
public:
Database();
virtual ~Database();
/// @}
/// @name Static methods
/// @{
public:
static void init();
/// @}
/// @name Implementation
/// @{
public:
/// Place an event in Game's event queue.
virtual void postEvent(DatabaseRequest* _pEvent);
/// Static thread startup method.
/// @note Don't call this; the thread lifetime
/// is automatically handled.
static void processThread(void *udata);
/// @}
/// @name Additional implementation
/// @{
protected:
/// Main process loop
void processLoop();
/// Process a single event
void processEvent(DatabaseRequest* event);
/// Process a generic query
void processRequest(DatabaseRequest* _pRequest);
/// @}
/// @name Member Variables
/// @{
private:
/// Events are stored here by any thread, for processing by the main thread.
Vector<DatabaseRequest*> m_eventQueue1, m_eventQueue2, *m_pEventQueue;
volatile bool m_shuttingDown;
Mutex* m_pEventQueueMutex;
Semaphore* m_pNotEmptyEvent;
Thread* m_processThread;
mysqlpp::Connection m_con;
/// @}
}; // class Database
extern Database* gDatabase;
#endif // ZEN_WORLDS_ZGARAGE_DATABASE_HPP_INCLUDEDDatabase.cpp
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
// Zen Worlds
//
// Copyright (C) 2001 - 2009 Tony Richards
//
// This software is provided 'as-is', without any express or implied
// warranty. In no event will the authors be held liable for any damages
// arising from the use of this software.
//
// Permission is granted to anyone to use this software for any purpose,
// including commercial applications, and to alter it and redistribute it
// freely, subject to the following restrictions:
//
// 1. The origin of this software must not be misrepresented; you must not
// claim that you wrote the original software. If you use this software
// in a product, an acknowledgment in the product documentation would be
// appreciated but is not required.
// 2. Altered source versions must be plainly marked as such, and must not be
// misrepresented as being the original software.
// 3. This notice may not be removed or altered from any source distribution.
//
// Tony Richards trichards@indiezen.com
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
#include "Database.hpp"
#include "platform/platform.h"
#include "platform/threads/mutex.h"
#include "platform/threads/semaphore.h"
#include "util/messaging/eventManager.h"
#include "console/console.h"
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
Database *gDatabase = NULL;
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
Database::Database()
: m_pEventQueue(&m_eventQueue1)
, m_pEventQueueMutex(new Mutex)
, m_pNotEmptyEvent(new Semaphore)
, m_shuttingDown(false)
, m_con(false)
{
m_processThread = new Thread((ThreadRunFunction)processThread, this, 1);
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
Database::~Database()
{
// TODO: Shutdown gracefully. For now everything just quieces correctly
// anyway, but it'd be nice to do this correctly.
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::init()
{
gDatabase = new Database();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::postEvent(DatabaseRequest* _pEvent)
{
// Only one thread can post at a time.
Mutex::lockMutex(m_pEventQueueMutex);
m_pEventQueue->push_back(_pEvent);
// Assert the condition variable
m_pNotEmptyEvent->release();
Mutex::unlockMutex(m_pEventQueueMutex);
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::processEvent(DatabaseRequest* _pEvent)
{
try
{
// First, always ping to attempt to reconnect if we've dropped
if (!m_con.connected())
{
Con::printf("Lost database connection... reconnecting");
if (!m_con.connect(
Con::getVariable("Database::database"),
Con::getVariable("Database::host"),
Con::getVariable("Database::user"),
Con::getVariable("Database::password")))
{
Con::printf("Unable to reconnect with connect()");
}
}
processRequest(_pEvent);
}
catch(...)
{
Con::printf("Database thread threw an unhandled exception.");
}
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void Database::processRequest(DatabaseRequest* _pRequest)
{
// Create the query
mysqlpp::Query query = m_con.query();
query << _pRequest->pSQLString;
// Free up the memory for this query
delete _pRequest->pSQLString;
// Execute the query
mysqlpp::UseQueryResult res = query.use();
if (res)
{
try
{
mysqlpp::Row row;
// Set the field count
_pRequest->pRecordSet->setFieldCount(res.num_fields());
// Loop through all of the columns and set them in the recordset
for(int x = 0; x < res.num_fields(); x++)
{
_pRequest->pRecordSet->setFieldName(x, res.field_name(x).c_str());
}
// Loop through all of the rows and set the record field values
while (row = res.fetch_row())
{
_pRequest->pRecordSet->addRecord();
for(int x = 0; x < res.num_fields(); x++)
{
_pRequest->pRecordSet->setFieldValue(x, row.at(x).c_str());
}
}
}
catch(...)
{
// Hard error
Con::errorf("DatabaseInterface::processEvent exception!");
}
}
else
{
// Soft error
Con::errorf("DatabaseInterface::processEvent error: %s", query.error());
}
// Since TGEA supports executef() from threads other than the main thread, this
// is safe.
Con::executef(_pRequest->pQueryCallback, _pRequest->pRecordSet->getIdString());
delete _pRequest->pQueryCallback;
// This will be done in processLoop, so don't do it here.
//delete _pRequest;
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::processLoop()
{
if (!m_con.connect(
Con::getVariable("Database::database"),
Con::getVariable("Database::host"),
Con::getVariable("Database::user"),
Con::getVariable("Database::password")))
{
Con::printf("Error connecting to database!");
// TODO: Loop a few times before giving up?
return;
}
while(!m_shuttingDown)
{
// Wait for the event to be signalled
m_pNotEmptyEvent->acquire(true);
m_pEventQueueMutex->lock(true);
// swap event queue pointers
Vector<DatabaseRequest*> &fullEventQueue = *m_pEventQueue;
if(m_pEventQueue == &m_eventQueue1)
m_pEventQueue = &m_eventQueue2;
else
m_pEventQueue = &m_eventQueue1;
m_pEventQueueMutex->unlock();
// Keep track of the original size
const int size = fullEventQueue.size();
// Walk the event queue in fifo order, processing the events, then clear the queue.
for(int i=0; i < size; i++)
{
processEvent(fullEventQueue[i]);
dFree(fullEventQueue[i]);
}
fullEventQueue.clear();
}
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::processThread( void *udata )
{
Database* pThis = (Database *)udata;
pThis->processLoop();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~RecordSet.hpp
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
// Zen Worlds
//
// Copyright (C) 2001 - 2009 Tony Richards
//
// This software is provided 'as-is', without any express or implied
// warranty. In no event will the authors be held liable for any damages
// arising from the use of this software.
//
// Permission is granted to anyone to use this software for any purpose,
// including commercial applications, and to alter it and redistribute it
// freely, subject to the following restrictions:
//
// 1. The origin of this software must not be misrepresented; you must not
// claim that you wrote the original software. If you use this software
// in a product, an acknowledgment in the product documentation would be
// appreciated but is not required.
// 2. Altered source versions must be plainly marked as such, and must not be
// misrepresented as being the original software.
// 3. This notice may not be removed or altered from any source distribution.
//
// Tony Richards trichards@indiezen.com
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
#ifndef ZEN_WORLDS_ZGARAGE_RECORD_SET_HPP_INCLUDED
#define ZEN_WORLDS_ZGARAGE_RECORD_SET_HPP_INCLUDED
#include "console/simBase.h"
#include <vector>
#include <map>
#include <string>
#include <list>
class RecordSet
: public SimObject
{
/// @name Types
/// @{
private:
typedef SimObject Parent;
/// @}
/// @name 'Structors
/// @{
public:
RecordSet();
virtual ~RecordSet();
/// @}
/// @name SimObject implementation
/// @{
public:
virtual bool onAdd();
virtual void onRemove();
static void initPersistFields();
/// @}
/// @name Getter / Setter methods
/// @{
public:
void setName(const char* _pName);
const char* getName();
void setFieldCount(S32 fieldCount);
S32 getFieldCount();
void setFieldName(S32 index, const char* pFieldName);
const char* getFieldName(S32 index);
void setFieldValue(S32 index, const char* pValue);
const char* getFieldValue(S32 index);
const char* getFieldValue(const char* pFieldName);
/// @}
/// @name Record traversal
/// @{
public:
void addRecord();
bool nextRecord();
/// @}
/// @name Member variables
/// @{
private:
typedef std::vector<std::string> record_type;
typedef std::list<record_type> record_set_type;
S32 m_queryId;
S32 m_fieldCount;
/// Field names kept in order
std::vector<std::string> m_fieldNames;
/// Maps the name of the field to the index
/// This is slightly inefficient to store the
/// full copy of the string instead of a reference
/// since it's already being stored in m_fieldNames,
/// but I'm lazy and it's really not an important
/// optimization unless you have a ton of recordsets
/// hanging around.
std::map<std::string, S32> m_fieldNameMap;
/// Record set
record_set_type m_recordSet;
bool m_firstRecord;
record_set_type::iterator m_currentRecord;
std::string m_name;
/// @}
/// @name Console Object Stuff
/// @{
public:
DECLARE_CONOBJECT(RecordSet);
/// @}
}; // class RecordSet
#endif // ZEN_WORLDS_ZGARAGE_RECORD_SET_HPP_INCLUDEDAnd RecordSet.cpp
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
// Zen Worlds
//
// Copyright (C) 2001 - 2009 Tony Richards
//
// This software is provided 'as-is', without any express or implied
// warranty. In no event will the authors be held liable for any damages
// arising from the use of this software.
//
// Permission is granted to anyone to use this software for any purpose,
// including commercial applications, and to alter it and redistribute it
// freely, subject to the following restrictions:
//
// 1. The origin of this software must not be misrepresented; you must not
// claim that you wrote the original software. If you use this software
// in a product, an acknowledgment in the product documentation would be
// appreciated but is not required.
// 2. Altered source versions must be plainly marked as such, and must not be
// misrepresented as being the original software.
// 3. This notice may not be removed or altered from any source distribution.
//
// Tony Richards trichards@indiezen.com
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
#include "RecordSet.hpp"
#include "console/simBase.h"
#include "console/consoleInternal.h"
#include "Database.hpp"
#include <algorithm>
#include <cctype>
IMPLEMENT_CONOBJECT(RecordSet);
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
RecordSet::RecordSet()
: m_queryId(0)
, m_fieldCount(0)
, m_firstRecord(true)
{
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
RecordSet::~RecordSet()
{
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::setName(const char* _pName)
{
m_name = _pName;
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
const char*
RecordSet::getName()
{
return m_name.c_str();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
bool
RecordSet::onAdd()
{
if (!Parent::onAdd())
return false;
const char *name = getName();
if(name && name[0] && getClassRep())
{
Namespace *parent = getClassRep()->getNameSpace();
Con::linkNamespaces(parent->mName, name);
mNameSpace = Con::lookupNamespace(name);
}
return true;
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::onRemove()
{
Parent::onRemove();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::initPersistFields()
{
Parent::initPersistFields();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::setFieldCount(S32 fieldCount)
{
// We don't really need to keep track of this, but we do use it
// as a hint for the vector size.
m_fieldCount = fieldCount;
m_fieldNames.resize(fieldCount);
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
S32
RecordSet::getFieldCount()
{
return m_fieldCount;
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::setFieldName(S32 index, const char* pFieldName)
{
std::string tmpFieldName(pFieldName);
m_fieldNames[index] = tmpFieldName;
m_fieldNameMap[tmpFieldName] = index;
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
const char*
RecordSet::getFieldName(S32 index)
{
return m_fieldNames[index].c_str();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::setFieldValue(S32 index, const char *pValue)
{
m_recordSet.back()[index] = pValue;
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
const char*
RecordSet::getFieldValue(S32 index)
{
if (m_firstRecord)
{
Con::errorf("Call RecordSet::nextRecord() before calling getFieldValue");
return "";
}
return (*m_currentRecord)[index].c_str();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
const char*
RecordSet::getFieldValue(const char* pFieldName)
{
std::string tmpFieldName(pFieldName);
std::transform(tmpFieldName.begin(), tmpFieldName.end(), tmpFieldName.begin(),
(int(*)(int)) std::tolower);
// TODO: Use find() to see if the field name actually
// exists and display an error if it doesn't exist.
// For now, this will return the value of the first field
// if the specified field name doesn't exist.
return getFieldValue(m_fieldNameMap[tmpFieldName]);
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::addRecord()
{
record_type tmpRecord;
m_recordSet.push_back(tmpRecord);
m_recordSet.back().resize(m_fieldCount);
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
bool
RecordSet::nextRecord()
{
if (m_firstRecord)
{
m_firstRecord = false;
m_currentRecord = m_recordSet.begin();
}
else
{
++m_currentRecord;
}
return m_currentRecord != m_recordSet.end();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, getName, const char*, 2, 2, "getName()")
{
return object->getName();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, nextRecord, bool, 2, 2, "nextRecord()")
{
return object->nextRecord();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, getFieldCount, S32, 2, 2, "getFieldCount()")
{
return object->getFieldCount();
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, getFieldName, const char*, 3, 3, "getFieldName(%index)")
{
return object->getFieldName(dAtoi(argv[2]));
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, getFieldValue, const char*, 3, 3, "getFieldValue(%index or %fieldName)")
{
if (dIsalpha(argv[2][0]))
{
return object->getFieldValue(argv[2]);
}
else
{
return object->getFieldValue(dAtoi(argv[2]));
}
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleFunction(query, void, 3, 3, "query(%sqlStatement, %callback)")
{
DatabaseRequest* pRequest = new DatabaseRequest;
pRequest->pRecordSet = static_cast<RecordSet*>(ConsoleObject::create("RecordSet"));
pRequest->pRecordSet->registerObject();
pRequest->pSQLString = dStrdup(argv[1]);
pRequest->pQueryCallback = dStrdup(argv[2]);
gDatabase->postEvent(pRequest);
}
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~You might want to do conditional compiles so that this only gets compiled when you're compiling your dedicated server.
#ifdef DEDICATED .. (mysql code here) #endif
And finally, in scriptAndAssets/server/init.cs, either in initServer() or initDedicated() (depending on if you want this only for dedicated servers or if you want this for all servers)
function initServer()
...
exec("./database.cs")
}And then in that same directory, add database.cs file:
$Database::database = "test";
$Database::host = "node1";
$Database::user = "sgtflame";
$Database::password = "password";
function testDatabase()
{
query("Select * from User", testDatabaseCallback);
}
function testDatabaseCallback(%recordSet)
{
while(%recordSet.nextRecord())
{
// Dump all of the fields
for(%x = 0; %x < %recordSet.getFieldCount(); %x++)
{
echo(%recordSet.getFieldName(%x) SPC "=" SPC %recordSet.getFieldValue(%x));
}
echo(%recordSet.getFieldValue("firstName") SPC %recordSet.getFieldValue("lastName"));
}
// Make sure to clean this up.
%recordSet.delete();
}So there you go. Torque + mysql++, including multi-threading asynchronous database requests and a scriptable record set.
Enjoy!
About the author
I am the founder of IndieZen.org, a website dedicated to the Indie 2.0 Revolution where a number of Indie game development studios and individuals collaborate and share a suite of custom built open source game development tools and middleware.
#3
04/05/2009 (11:44 am)
Nice addition Tony. Definitely a lot cleaner and more flexible than the implementation I came up with.
#4
Edit: I guess this isn't as important with MySQL compared to some other DBs. I will go back to crawling under a rock.
04/05/2009 (12:15 pm)
Is there any transactional processing support? While reading from the DB will work fine, if you are doing quite a bit of writes to it you will need to find a way to do transactional commits otherwise you will have some issues with performance. Otherwise it looks like a nice implementation. Good to see you guys opening up your work on your framework for Torque.Edit: I guess this isn't as important with MySQL compared to some other DBs. I will go back to crawling under a rock.
#5
For example:
But that quickly degrades into needing to insert some form of read, modify, write, where you want to put some script logic inside of the "modify" section.... and I do not suggest you do that this way.
If you need transactional support, I still say that going with a message bus and having a service on an application server is the best way to go.
I will provide several more resources in the near future that will take you in this direction.
04/05/2009 (12:45 pm)
Adding transaction support would be pretty easy. Turn the DatabaseRequest structure into a class that inherits from SimObject, and make it so that it supports multiple queries.For example:
function testDatabase()
{
%query = createTransaction();
%query.addQuery("Update CharacterInventory set itemCount = itemCount - 1 where characterInventoryId = 1");
%query.addQuery("Update CharacterInventory set itemCount = itemCount + 1 where characterInventoryId = 2");
%query.execute("testDatabaseCallback");
}But that quickly degrades into needing to insert some form of read, modify, write, where you want to put some script logic inside of the "modify" section.... and I do not suggest you do that this way.
If you need transactional support, I still say that going with a message bus and having a service on an application server is the best way to go.
I will provide several more resources in the near future that will take you in this direction.
#6
Thanks so much Tony, gonna give this a shot right now.
I kept getting lost trying to figure out how that was done.
04/05/2009 (4:52 pm)
Woot!Thanks so much Tony, gonna give this a shot right now.
Quote:Glad you nailed it!
primarily because TGEA now supports script callbacks from threads other than the main render thread
I kept getting lost trying to figure out how that was done.
#7
I did have to copy libmySQL.dll and mysqlpp.dll into the game directory once I had everything compiled.
Works like a charm!
Now I'm gonna have to put a load on this puppy...
04/05/2009 (7:26 pm)
Took me a couple tries, had to remind myself to re-read your post.I did have to copy libmySQL.dll and mysqlpp.dll into the game directory once I had everything compiled.
Works like a charm!
Now I'm gonna have to put a load on this puppy...
#8
04/05/2009 (8:10 pm)
Just a note, if you distribute those dlls you probably will need to make the source code available for those dlls if ever asked since they are most likely GPL along with MySQL.
#10
Feel free to e-mail me if you need anything else.
04/06/2009 (4:57 am)
You are most welcome... now get Ruin back online ;-)Feel free to e-mail me if you need anything else.
#11
04/06/2009 (6:18 am)
Just wondering if the script callbacks from threads is for TGEA 1.8 only or for 1.7.1 also?
#12
Great work!
04/06/2009 (6:22 am)
Just tried it, that's simply awesome. And a very clean implementation indeed. Thanks for sharing, I've learned a couple of useful lessons from your code. ;)Great work!
#13
This lets you pass variables (like client, player, etc.) along to the callbacks.
In Database.hpp change
04/07/2009 (1:22 am)
I resurrected some of the functionality from the old resource.This lets you pass variables (like client, player, etc.) along to the callbacks.
In Database.hpp change
struct DatabaseRequest
{
const char* pSQLString;
const char* pQueryCallback;
RecordSet* pRecordSet;
};
struct DatabaseReply
{
const char* pQueryCallback;
RecordSet* pRecordSet;
};tostruct DatabaseRequest
{
const char* pSQLString;
const char* pQueryCallback;
RecordSet* pRecordSet;
S32 clientID; //to tuck in the client ID
S32 extra;
S32 extra1;
};
struct DatabaseReply
{
const char* pQueryCallback;
RecordSet* pRecordSet;
S32 clientID; //to tuck in the client ID
S32 extra;
S32 extra1;
};In Database.cpp in void Database::processRequest(DatabaseRequest* _pRequest) changeCon::executef(_pRequest->pQueryCallback, _pRequest->pRecordSet->getIdString());to
Con::executef(_pRequest->pQueryCallback, _pRequest->pRecordSet->getIdString(), Con::getIntArg(_pRequest->clientID), Con::getIntArg(_pRequest->extra), Con::getIntArg(_pRequest->extra1));In RecordSet.cpp replace
ConsoleFunction(query, void, 3, 3, "query(%sqlStatement, %callback)")
{
DatabaseRequest* pRequest = new DatabaseRequest;
pRequest->pRecordSet = static_cast<RecordSet*>(ConsoleObject::create("RecordSet"));
pRequest->pRecordSet->registerObject();
pRequest->pSQLString = dStrdup(argv[1]);
pRequest->pQueryCallback = dStrdup(argv[2]);
gDatabase->postEvent(pRequest);
}withConsoleFunction(query,void,3,6,"query(char sqlStatement, char callback, S32 client, S32 extra, S32 extra1)")
{
DatabaseRequest* pRequest = new DatabaseRequest;
pRequest->pRecordSet = static_cast<RecordSet*>(ConsoleObject::create("RecordSet"));
pRequest->pRecordSet->registerObject();
pRequest->pSQLString = dStrdup(argv[1]);
pRequest->pQueryCallback = dStrdup(argv[2]);
//If a client ID was passed in, make use of it here
if(argc >= 4 && dAtoi(argv[3]))
pRequest->clientID = dAtoi(argv[3]);
else
pRequest->clientID = -1;
//If an extra ID was passed in, make use of it here
if(argc >= 5 && dAtoi(argv[4]))
pRequest->extra = dAtoi(argv[4]);
else
pRequest->extra = -1;
if(argc >= 6 && dAtoi(argv[5]))
pRequest->extra1 = dAtoi(argv[5]);
else
pRequest->extra1 = -1;
gDatabase->postEvent(pRequest);
}Simple Example:function LoadInventory(%client ,%obj)
{
%query = "SELECT * FROM players_inventory WHERE Owner = '"@%client.nameBase@"';";
query(%query,"LoadInventoryCallback",%client,%obj);
}
function LoadInventoryCallback(%recordSet, %client, %obj)
{
while(%recordSet.nextRecord())
{
%dataBlock = %recordSet.getFieldValue("DataBlock");
%Qty = %recordSet.getFieldValue("Qty");
if(%dataBlock && %Qty > 0)
{
%obj.incInventory(%dataBlock,%Qty);
}
}
%recordSet.delete();
}
#14
This line in Database.cpp
Don't be worried, it just means that the query did not return a record set.
04/08/2009 (12:35 am)
Just a heads up for people using this for the first time.This line in Database.cpp
Con::errorf("DatabaseInterface::processEvent error: %s", query.error());will return an error when using commands like UPDATE that put data into the database.Don't be worried, it just means that the query did not return a record set.
#15
04/08/2009 (5:34 am)
I cant see this resource at all.
#16
@Steven Chiu - Sorry, I don't have any experience with any TGEA version between TSE MS4 and TGEA 1.8.1, so I'm not sure exactly when those changes were made.
04/08/2009 (6:45 am)
The recent site changes is causing it not to render... the GG staff is aware of the issue and I have faith that it will be fixed soon.@Steven Chiu - Sorry, I don't have any experience with any TGEA version between TSE MS4 and TGEA 1.8.1, so I'm not sure exactly when those changes were made.
#17
04/09/2009 (10:43 am)
Awesome additions, Ari.
#18
My link command line:
My c/c++ command line:
I can not get this stupid error in the project. O__o
04/29/2009 (10:37 am)
I tried to build the project with tgea 1.8.0 and VS2008 (Team System SP1). But I always make the same mistake in the link.Database.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: class std::basic_ostream<char,struct std::char_traits<char> .... Database.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: void __thiscall std::basic_ostream<char,struct std::char_traits<char> .... Database.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: void __thiscall std::basic_ios<char,struct std::char_traits<char> ....
My link command line:
/OUT:"../../../game/eYe.exe" /INCREMENTAL:NO /NOLOGO /LIBPATH:"../../../../../engine/lib/SDL/win32" /LIBPATH:"../../../../../engine/lib/unicode" /LIBPATH:"../../../../../engine/lib/directx8" /LIBPATH:"../../Link/VC2k8.Release.Win32" /LIBPATH:"C:\Program Files\Microsoft DirectX SDK (November 2008)\Lib\x86" /MANIFEST /MANIFESTFILE:"../../Link/VC2k8.Release.Win32/eYe\eYe.exe.intermediate.manifest" /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /NODEFAULTLIB:"LIBC" /PDB:"../../Link/VC2k8.Release.Win32/eYe.pdb" /SUBSYSTEM:WINDOWS /DYNAMICBASE:NO /MACHINE:X86 /ERRORREPORT:PROMPT x3daudio.lib COMCTL32.LIB COMDLG32.LIB USER32.LIB ADVAPI32.LIB GDI32.LIB WINMM.LIB WSOCK32.LIB vfw32.lib Imm32.lib unicows.lib d3d9.lib dxerr9.lib d3dx9.lib d3d8.lib opengl32.lib ole32.lib shell32.lib oleaut32.lib mysqlpp.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib "..\..\link\vc2k8.release.win32\ljpeg.lib" "..\..\link\vc2k8.release.win32\libogg.lib" "..\..\link\vc2k8.release.win32\tinyxml.lib" "..\..\link\vc2k8.release.win32\libvorbis.lib" "..\..\link\vc2k8.release.win32\lmng.lib" "..\..\link\vc2k8.release.win32\d3dx8dll.lib" "..\..\link\vc2k8.release.win32\lpng.lib" "..\..\link\vc2k8.release.win32\squish.lib" "..\..\link\vc2k8.release.win32\zlib.lib" "..\..\link\vc2k8.release.win32\lungif.lib" "..\..\link\vc2k8.release.win32\opcode.lib"
My c/c++ command line:
/Ox /Ob2 /Oi /I "../../../../../engine/lib/lmng" /I "../../../../../engine/lib/lpng" /I "../../../../../engine/lib/zlib" /I "../../../../../engine/lib/ljpeg" /I "../../../../../engine/lib/lungif" /I "../../../../../engine/lib/tinyxml" /I "../../../../../engine/lib/d3dxdll" /I "../../../../../engine/lib/directx8" /I "../../../../../engine/lib/opcode" /I "../../../../../engine/lib/libvorbis/lib" /I "../../../../../engine/lib/libvorbis/include" /I "../../../../../engine/lib/libogg/include" /I "../../../../../engine/lib/squish" /I "../../../game/shaders" /I "../../../source" /I "../../../../../engine/source/" /I "../../../../../engine/lib/" /I "../../../../../engine/lib/openal/win32" /I "../../../../../engine/lib/mysqlpp/lib" /I "C:\Program Files\Microsoft DirectX SDK (November 2008)\Include" /I "C:\Program Files\MySQL\MySQL Server 5.1\include" /D "DEDICATED" /D "TORQUE_SHADERGEN" /D "TORQUE_PLAYER" /D "TORQUE_OPCODE" /D "TORQUE_ATLAS" /D "TORQUE_OGGVORBIS" /D "LTC_NO_PROTOTYPES" /D "BAN_OPCODE_AUTOLINK" /D "ICE_NO_DLL" /D "NOWINBASEINTERLOCK" /D "UNICODE" /D "INITGUID" /D "_CRT_SECURE_NO_DEPRECATE" /D "_VC80_UPGRADE=0x0710" /GF /Gm /EHsc /MT /GS- /Gy /Zc:wchar_t- /Fo"../../Link/VC2k8.Release.Win32/eYe\" /Fd"../../Link/VC2k8.Release.Win32/eYe.pdb" /FR"../../Link/VC2k8.Release.Win32/eYe\" /W4 /nologo /c /Zi /TP /wd4018 /wd4100 /wd4121 /wd4127 /wd4130 /wd4244 /wd4245 /wd4275 /wd4389 /wd4511 /wd4512 /wd4800 /errorReport:prompt
I can not get this stupid error in the project. O__o
#19
Looks like you missed this step.
04/29/2009 (12:37 pm)
Quote:In the mysqlpp project properites, under C/C++, Code Generation, make sure you change Runtime Library to match Torque (which is normally Multi-threaded and Multi-threaded Debug).
Looks like you missed this step.
#20
Thank you very much
04/29/2009 (2:35 pm)
I said that was something stupid that I blocked. I was mysqlpp Multithreaded DLL and Multithreaded Torque was.Thank you very much

Torque Owner Javier Canon