Game Development Community

dev|Pro Game Development Curriculum

Asynchronous MySQL++ for TGEA 1.8.x

by Tony Richards · 04/05/2009 (8:10 am) · 37 comments

This resource is a sequel to my Event Driven Database resource, except written specifically for TGEA 1.8.x. I've continued using mysql++ in this example, but it should not be terribly difficult to modify for use with other databases, or even with other service oriented architectures.
This resource ended up being a lot simpler than what I had originally anticipated, primarily because TGEA now supports script callbacks from threads other than the main render thread.

I started by copying the ProjectsTemplate folder into ProjectsMySQLDemo.

Then I renamed Template.sln to MySQLDemo.sln, and Template.vcproj to MySQLDemo.vcproj. When you open MySQLDemo.sln, you'll have to remove the Template project and add the MySQLDemo project (and rename it as necessary).

Next, download mysql++. I downloaded the source and placed it in ProjectsMySQLDemo/mysql++, and then I added the mysqlpp project to my solution.

In the mysqlpp project properites, under C/C++, Code Generation, make sure you change Runtime Library to match Torque (which is normally Multi-threaded and Multi-threaded Debug).

Next, make sure you add the MySQL header location to C/C++, General, Additional Include Directories. You'll need to do this to both mysqlpp and MySQLDemo projects. Also, add the location of the mysql++ headers to the MySQLDemo project.

Mine looked like this:
../../../mysql++/lib
C:/Program Files/MySQL/MySQL Server 5.0/include

It's best if you go ahead and make these adjustments to Debug, Release and Optimized configurations while it's fresh on your mind.

Also, right-click on MySQLDemo and click on Project Dependencies... and add mysqlpp project as a dependency (actually, add all of the projects as a dependency to the MySQLDemo project).

Next, in main.cpp include "Database.hpp" and add Database::init();.
...

#include "Database.hpp"

...

S32 TorqueMain(S32 argc, const char **argv)
...

   // Initialize the subsystems.
   StandardMainLoop::init();

   // Handle any command line args.
   if(!StandardMainLoop::handleCommandLine(argc, argv))
   {
      Platform::AlertOK("Error", "Failed to initialize game, shutting down.");

      return 1;
   }

   Database::init();

   // Main loop
   while(StandardMainLoop::doMainLoop());

   // Clean everything up.
   StandardMainLoop::shutdown();
...

Now, add these four files:

Database.hpp:
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
// Zen Worlds
//
// Copyright (C) 2001 - 2009 Tony Richards
//
//  This software is provided 'as-is', without any express or implied
//  warranty.  In no event will the authors be held liable for any damages
//  arising from the use of this software.
//
//  Permission is granted to anyone to use this software for any purpose,
//  including commercial applications, and to alter it and redistribute it
//  freely, subject to the following restrictions:
//
//  1. The origin of this software must not be misrepresented; you must not
//     claim that you wrote the original software. If you use this software
//     in a product, an acknowledgment in the product documentation would be
//     appreciated but is not required.
//  2. Altered source versions must be plainly marked as such, and must not be
//     misrepresented as being the original software.
//  3. This notice may not be removed or altered from any source distribution.
//
//  Tony Richards trichards@indiezen.com
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~

#ifndef ZEN_WORLDS_ZGARAGE_DATABASE_HPP_INCLUDED
#define ZEN_WORLDS_ZGARAGE_DATABASE_HPP_INCLUDED

#include "core/util/tVector.h"

#include "platform/event.h"

#include "platform/threads/thread.h"

#include <mysql++.h>

#include "RecordSet.hpp"

struct DatabaseRequest
{
   const char*    pSQLString;
   const char*    pQueryCallback;
   RecordSet*     pRecordSet;
};

struct DatabaseReply
{
   const char*    pQueryCallback;
   RecordSet*     pRecordSet;
};

class Database
{
    /// @name 'Structors
    /// @{
public:
            Database();
   virtual ~Database();
    /// @}

    /// @name Static methods
    /// @{
public:
    static void init();
    /// @}

    /// @name Implementation
    /// @{
public:

    /// Place an event in Game's event queue.
    virtual void postEvent(DatabaseRequest* _pEvent);

    /// Static thread startup method.
    /// @note Don't call this; the thread lifetime
    ///          is automatically handled.
    static void processThread(void *udata);
    /// @}

    /// @name Additional implementation
    /// @{
protected:
    /// Main process loop
    void processLoop();

    /// Process a single event
    void processEvent(DatabaseRequest* event);

    /// Process a generic query
   void processRequest(DatabaseRequest* _pRequest);
    /// @}

    /// @name Member Variables
    /// @{
private:

    /// Events are stored here by any thread, for processing by the main thread.
    Vector<DatabaseRequest*> m_eventQueue1, m_eventQueue2, *m_pEventQueue;

    volatile bool   m_shuttingDown;
    Mutex*          m_pEventQueueMutex;
    Semaphore*      m_pNotEmptyEvent;
    Thread*         m_processThread;

    mysqlpp::Connection m_con;
    /// @}

};  // class Database

extern Database* gDatabase;

#endif // ZEN_WORLDS_ZGARAGE_DATABASE_HPP_INCLUDED

Database.cpp
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
// Zen Worlds
//
// Copyright (C) 2001 - 2009 Tony Richards
//
//  This software is provided 'as-is', without any express or implied
//  warranty.  In no event will the authors be held liable for any damages
//  arising from the use of this software.
//
//  Permission is granted to anyone to use this software for any purpose,
//  including commercial applications, and to alter it and redistribute it
//  freely, subject to the following restrictions:
//
//  1. The origin of this software must not be misrepresented; you must not
//     claim that you wrote the original software. If you use this software
//     in a product, an acknowledgment in the product documentation would be
//     appreciated but is not required.
//  2. Altered source versions must be plainly marked as such, and must not be
//     misrepresented as being the original software.
//  3. This notice may not be removed or altered from any source distribution.
//
//  Tony Richards trichards@indiezen.com
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~

#include "Database.hpp"

#include "platform/platform.h"
#include "platform/threads/mutex.h"
#include "platform/threads/semaphore.h"

#include "util/messaging/eventManager.h"

#include "console/console.h"

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
Database *gDatabase = NULL;

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
Database::Database()
:  m_pEventQueue(&m_eventQueue1)
,  m_pEventQueueMutex(new Mutex)
,  m_pNotEmptyEvent(new Semaphore)
,  m_shuttingDown(false)
,  m_con(false)
{
   m_processThread = new Thread((ThreadRunFunction)processThread, this, 1);
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
Database::~Database()
{
   // TODO: Shutdown gracefully.  For now everything just quieces correctly
   // anyway, but it'd be nice to do this correctly.
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::init()
{
    gDatabase = new Database();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::postEvent(DatabaseRequest* _pEvent)
{
   // Only one thread can post at a time.
   Mutex::lockMutex(m_pEventQueueMutex);

   m_pEventQueue->push_back(_pEvent);

   // Assert the condition variable
   m_pNotEmptyEvent->release();

   Mutex::unlockMutex(m_pEventQueueMutex);
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::processEvent(DatabaseRequest* _pEvent)
{
    try
    {
        // First, always ping to attempt to reconnect if we've dropped
        if (!m_con.connected())
        {
            Con::printf("Lost database connection... reconnecting");

            if (!m_con.connect(
                Con::getVariable("Database::database"), 
                Con::getVariable("Database::host"), 
                Con::getVariable("Database::user"), 
                Con::getVariable("Database::password")))
            {
                Con::printf("Unable to reconnect with connect()");
            }
        }

        processRequest(_pEvent);
    }
    catch(...)
    {
        Con::printf("Database thread threw an unhandled exception.");
    }
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void Database::processRequest(DatabaseRequest* _pRequest)
{
    // Create the query
    mysqlpp::Query query = m_con.query();
    query << _pRequest->pSQLString;

    // Free up the memory for this query
    delete _pRequest->pSQLString;

    // Execute the query
    mysqlpp::UseQueryResult res = query.use();

    if (res)
    {
        try
        {
            mysqlpp::Row row;

            // Set the field count
            _pRequest->pRecordSet->setFieldCount(res.num_fields());

            // Loop through all of the columns and set them in the recordset
            for(int x = 0; x < res.num_fields(); x++)
            {
                _pRequest->pRecordSet->setFieldName(x, res.field_name(x).c_str());
            }

            // Loop through all of the rows and set the record field values
            while (row = res.fetch_row()) 
            {
                _pRequest->pRecordSet->addRecord();
                for(int x = 0; x < res.num_fields(); x++)
                {
                    _pRequest->pRecordSet->setFieldValue(x, row.at(x).c_str());
                }
            }
        }
        catch(...)
        {
            // Hard error
            Con::errorf("DatabaseInterface::processEvent exception!");
        }
    }
    else
    {
        // Soft error
        Con::errorf("DatabaseInterface::processEvent error: %s", query.error());
    }

    // Since TGEA supports executef() from threads other than the main thread, this
    // is safe.
    Con::executef(_pRequest->pQueryCallback, _pRequest->pRecordSet->getIdString());
    delete _pRequest->pQueryCallback;

    // This will be done in processLoop, so don't do it here.
    //delete _pRequest;
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::processLoop()
{
    if (!m_con.connect(
        Con::getVariable("Database::database"), 
        Con::getVariable("Database::host"), 
        Con::getVariable("Database::user"), 
        Con::getVariable("Database::password")))
    {
        Con::printf("Error connecting to database!");

        // TODO: Loop a few times before giving up?
        return;
    }

    while(!m_shuttingDown)
    {
        // Wait for the event to be signalled
        m_pNotEmptyEvent->acquire(true);

        m_pEventQueueMutex->lock(true);

        // swap event queue pointers
        Vector<DatabaseRequest*> &fullEventQueue = *m_pEventQueue;
        if(m_pEventQueue == &m_eventQueue1)
            m_pEventQueue = &m_eventQueue2;
        else
            m_pEventQueue = &m_eventQueue1;

        m_pEventQueueMutex->unlock();

        // Keep track of the original size
        const int size = fullEventQueue.size();

        // Walk the event queue in fifo order, processing the events, then clear the queue.
        for(int i=0; i < size; i++)
        {
            processEvent(fullEventQueue[i]);
            dFree(fullEventQueue[i]);
        }
        fullEventQueue.clear();
    }

}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
Database::processThread( void *udata )
{
   Database* pThis = (Database *)udata;
   pThis->processLoop();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~

RecordSet.hpp
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
// Zen Worlds
//
// Copyright (C) 2001 - 2009 Tony Richards
//
//  This software is provided 'as-is', without any express or implied
//  warranty.  In no event will the authors be held liable for any damages
//  arising from the use of this software.
//
//  Permission is granted to anyone to use this software for any purpose,
//  including commercial applications, and to alter it and redistribute it
//  freely, subject to the following restrictions:
//
//  1. The origin of this software must not be misrepresented; you must not
//     claim that you wrote the original software. If you use this software
//     in a product, an acknowledgment in the product documentation would be
//     appreciated but is not required.
//  2. Altered source versions must be plainly marked as such, and must not be
//     misrepresented as being the original software.
//  3. This notice may not be removed or altered from any source distribution.
//
//  Tony Richards trichards@indiezen.com
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~

#ifndef ZEN_WORLDS_ZGARAGE_RECORD_SET_HPP_INCLUDED
#define ZEN_WORLDS_ZGARAGE_RECORD_SET_HPP_INCLUDED

#include "console/simBase.h"

#include <vector>
#include <map>
#include <string>
#include <list>

class RecordSet 
:   public SimObject
{
    /// @name Types
    /// @{
private:
    typedef SimObject Parent;
    /// @}

    /// @name 'Structors
    /// @{
public:
             RecordSet();
    virtual ~RecordSet();
    /// @}

    /// @name SimObject implementation
    /// @{
public:
    virtual bool onAdd();
    virtual void onRemove();

    static void initPersistFields();
    /// @}

    /// @name Getter / Setter methods
    /// @{
public:
    void setName(const char* _pName);
    const char* getName();

    void setFieldCount(S32 fieldCount);
    S32 getFieldCount();

    void setFieldName(S32 index, const char* pFieldName);
    const char* getFieldName(S32 index);

    void setFieldValue(S32 index, const char* pValue);
    const char* getFieldValue(S32 index);
    const char* getFieldValue(const char* pFieldName);
    /// @}

    /// @name Record traversal
    /// @{
public:
    void addRecord();
    bool nextRecord();
    /// @}

    /// @name Member variables
    /// @{
private:
    typedef std::vector<std::string>    record_type;
    typedef std::list<record_type>      record_set_type;

    S32 m_queryId;
    S32 m_fieldCount;

    /// Field names kept in order
    std::vector<std::string>            m_fieldNames;

    /// Maps the name of the field to the index
    /// This is slightly inefficient to store the
    /// full copy of the string instead of a reference
    /// since it's already being stored in m_fieldNames,
    /// but I'm lazy and it's really not an important
    /// optimization unless you have a ton of recordsets
    /// hanging around.
    std::map<std::string, S32>          m_fieldNameMap;

    /// Record set
    record_set_type                     m_recordSet;

    bool                                m_firstRecord;
    record_set_type::iterator           m_currentRecord;

    std::string                         m_name;
    /// @}

    /// @name Console Object Stuff
    /// @{
public:
    DECLARE_CONOBJECT(RecordSet);
    /// @}

};  // class RecordSet


#endif // ZEN_WORLDS_ZGARAGE_RECORD_SET_HPP_INCLUDED

And RecordSet.cpp
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
// Zen Worlds
//
// Copyright (C) 2001 - 2009 Tony Richards
//
//  This software is provided 'as-is', without any express or implied
//  warranty.  In no event will the authors be held liable for any damages
//  arising from the use of this software.
//
//  Permission is granted to anyone to use this software for any purpose,
//  including commercial applications, and to alter it and redistribute it
//  freely, subject to the following restrictions:
//
//  1. The origin of this software must not be misrepresented; you must not
//     claim that you wrote the original software. If you use this software
//     in a product, an acknowledgment in the product documentation would be
//     appreciated but is not required.
//  2. Altered source versions must be plainly marked as such, and must not be
//     misrepresented as being the original software.
//  3. This notice may not be removed or altered from any source distribution.
//
//  Tony Richards trichards@indiezen.com
//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
#include "RecordSet.hpp"

#include "console/simBase.h"
#include "console/consoleInternal.h"

#include "Database.hpp"

#include <algorithm>
#include <cctype>

IMPLEMENT_CONOBJECT(RecordSet);

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
RecordSet::RecordSet()
:   m_queryId(0)
,   m_fieldCount(0)
,   m_firstRecord(true)
{
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
RecordSet::~RecordSet()
{
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::setName(const char* _pName)
{
    m_name = _pName;
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
const char*
RecordSet::getName()
{
    return m_name.c_str();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
bool
RecordSet::onAdd()
{
   if (!Parent::onAdd())
      return false;

   const char *name = getName();
   if(name && name[0] && getClassRep())
   {
      Namespace *parent = getClassRep()->getNameSpace();
      Con::linkNamespaces(parent->mName, name);
      mNameSpace = Con::lookupNamespace(name);
   }

   return true;
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::onRemove()
{
   Parent::onRemove();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::initPersistFields()
{
   Parent::initPersistFields();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::setFieldCount(S32 fieldCount)
{
   // We don't really need to keep track of this, but we do use it
   // as a hint for the vector size.
   m_fieldCount = fieldCount;
   m_fieldNames.resize(fieldCount);
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
S32
RecordSet::getFieldCount()
{
   return m_fieldCount;
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::setFieldName(S32 index, const char* pFieldName)
{
   std::string tmpFieldName(pFieldName);
   m_fieldNames[index] = tmpFieldName;
   m_fieldNameMap[tmpFieldName] = index;
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
const char*
RecordSet::getFieldName(S32 index)
{
   return m_fieldNames[index].c_str();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::setFieldValue(S32 index, const char *pValue)
{
   m_recordSet.back()[index] = pValue;
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
const char*
RecordSet::getFieldValue(S32 index)
{
   if (m_firstRecord)
   {
      Con::errorf("Call RecordSet::nextRecord() before calling getFieldValue");
      return "";
   }

   return (*m_currentRecord)[index].c_str();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
const char*
RecordSet::getFieldValue(const char* pFieldName)
{
   std::string tmpFieldName(pFieldName);
   std::transform(tmpFieldName.begin(), tmpFieldName.end(), tmpFieldName.begin(), 
      (int(*)(int)) std::tolower);

   // TODO: Use find() to see if the field name actually 
   // exists and display an error if it doesn't exist.
   // For now, this will return the value of the first field
   // if the specified field name doesn't exist.
   return getFieldValue(m_fieldNameMap[tmpFieldName]);
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
void
RecordSet::addRecord()
{
   record_type tmpRecord;
   m_recordSet.push_back(tmpRecord);
   m_recordSet.back().resize(m_fieldCount);
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
bool
RecordSet::nextRecord()
{
   if (m_firstRecord)
   {
      m_firstRecord = false;
      m_currentRecord = m_recordSet.begin();
   }
   else
   {
      ++m_currentRecord;
   }

   return m_currentRecord != m_recordSet.end();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, getName, const char*, 2, 2, "getName()")
{
   return object->getName();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, nextRecord, bool, 2, 2, "nextRecord()")
{
   return object->nextRecord();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, getFieldCount, S32, 2, 2, "getFieldCount()")
{
   return object->getFieldCount();
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, getFieldName, const char*, 3, 3, "getFieldName(%index)")
{
   return object->getFieldName(dAtoi(argv[2]));
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleMethod(RecordSet, getFieldValue, const char*, 3, 3, "getFieldValue(%index or %fieldName)")
{
   if (dIsalpha(argv[2][0]))
   {
      return object->getFieldValue(argv[2]);
   }
   else
   {
      return object->getFieldValue(dAtoi(argv[2]));
   }
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~
ConsoleFunction(query, void, 3, 3, "query(%sqlStatement, %callback)")
{
   DatabaseRequest* pRequest = new DatabaseRequest;
   pRequest->pRecordSet = static_cast<RecordSet*>(ConsoleObject::create("RecordSet"));
   pRequest->pRecordSet->registerObject();

   pRequest->pSQLString = dStrdup(argv[1]);
   pRequest->pQueryCallback = dStrdup(argv[2]);

   gDatabase->postEvent(pRequest);
}

//-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~-~

You might want to do conditional compiles so that this only gets compiled when you're compiling your dedicated server.
#ifdef DEDICATED
.. (mysql code here)
#endif


And finally, in scriptAndAssets/server/init.cs, either in initServer() or initDedicated() (depending on if you want this only for dedicated servers or if you want this for all servers)
function initServer()
...

    exec("./database.cs")
}

And then in that same directory, add database.cs file:
$Database::database  = "test";
$Database::host      = "node1";
$Database::user      = "sgtflame";
$Database::password  = "password";

function testDatabase()
{
   query("Select * from User", testDatabaseCallback);
}

function testDatabaseCallback(%recordSet)
{
   while(%recordSet.nextRecord())
   {
      // Dump all of the fields
      for(%x = 0; %x < %recordSet.getFieldCount(); %x++)
      {
         echo(%recordSet.getFieldName(%x) SPC "=" SPC %recordSet.getFieldValue(%x));
      }

      echo(%recordSet.getFieldValue("firstName") SPC %recordSet.getFieldValue("lastName"));
   }

   // Make sure to clean this up.
   %recordSet.delete();
}

So there you go. Torque + mysql++, including multi-threading asynchronous database requests and a scriptable record set.

Enjoy!

About the author

I am the founder of IndieZen.org, a website dedicated to the Indie 2.0 Revolution where a number of Indie game development studios and individuals collaborate and share a suite of custom built open source game development tools and middleware.

Page «Previous 1 2
#1
04/05/2009 (11:07 am)
Very cool i will try it, thanks for sharing...
#2
04/05/2009 (11:36 am)
Nice. Thank you.
#3
04/05/2009 (11:44 am)
Nice addition Tony. Definitely a lot cleaner and more flexible than the implementation I came up with.
#4
04/05/2009 (12:15 pm)
Is there any transactional processing support? While reading from the DB will work fine, if you are doing quite a bit of writes to it you will need to find a way to do transactional commits otherwise you will have some issues with performance. Otherwise it looks like a nice implementation. Good to see you guys opening up your work on your framework for Torque.

Edit: I guess this isn't as important with MySQL compared to some other DBs. I will go back to crawling under a rock.
#5
04/05/2009 (12:45 pm)
Adding transaction support would be pretty easy. Turn the DatabaseRequest structure into a class that inherits from SimObject, and make it so that it supports multiple queries.

For example:

function testDatabase()
{
    %query = createTransaction();
    %query.addQuery("Update CharacterInventory set itemCount = itemCount - 1 where characterInventoryId = 1");
    %query.addQuery("Update CharacterInventory set itemCount = itemCount + 1 where characterInventoryId = 2");
    %query.execute("testDatabaseCallback");
}

But that quickly degrades into needing to insert some form of read, modify, write, where you want to put some script logic inside of the "modify" section.... and I do not suggest you do that this way.

If you need transactional support, I still say that going with a message bus and having a service on an application server is the best way to go.

I will provide several more resources in the near future that will take you in this direction.
#6
04/05/2009 (4:52 pm)
Woot!
Thanks so much Tony, gonna give this a shot right now.
Quote:
primarily because TGEA now supports script callbacks from threads other than the main render thread
Glad you nailed it!
I kept getting lost trying to figure out how that was done.
#7
04/05/2009 (7:26 pm)
Took me a couple tries, had to remind myself to re-read your post.
I did have to copy libmySQL.dll and mysqlpp.dll into the game directory once I had everything compiled.
Works like a charm!
Now I'm gonna have to put a load on this puppy...
#8
04/05/2009 (8:10 pm)
Just a note, if you distribute those dlls you probably will need to make the source code available for those dlls if ever asked since they are most likely GPL along with MySQL.

#9
04/05/2009 (8:16 pm)
@Tony
Dude, THANK YOU!
#10
04/06/2009 (4:57 am)
You are most welcome... now get Ruin back online ;-)

Feel free to e-mail me if you need anything else.
#11
04/06/2009 (6:18 am)
Just wondering if the script callbacks from threads is for TGEA 1.8 only or for 1.7.1 also?
#12
04/06/2009 (6:22 am)
Just tried it, that's simply awesome. And a very clean implementation indeed. Thanks for sharing, I've learned a couple of useful lessons from your code. ;)
Great work!
#13
04/07/2009 (1:22 am)
I resurrected some of the functionality from the old resource.
This lets you pass variables (like client, player, etc.) along to the callbacks.

In Database.hpp change
struct DatabaseRequest
{
   const char*    pSQLString;
   const char*    pQueryCallback;
   RecordSet*     pRecordSet;
};

struct DatabaseReply
{
   const char*    pQueryCallback;
   RecordSet*     pRecordSet;
};
to
struct DatabaseRequest
{
   const char*    pSQLString;
   const char*    pQueryCallback;
   RecordSet*     pRecordSet;
   S32 clientID;  //to tuck in the client ID
   S32 extra;
   S32 extra1;
};

struct DatabaseReply
{
   const char*    pQueryCallback;
   RecordSet*     pRecordSet;
   S32 clientID;  //to tuck in the client ID
   S32 extra;
   S32 extra1;
};
In Database.cpp in void Database::processRequest(DatabaseRequest* _pRequest) change
Con::executef(_pRequest->pQueryCallback, _pRequest->pRecordSet->getIdString());
to
Con::executef(_pRequest->pQueryCallback, _pRequest->pRecordSet->getIdString(), Con::getIntArg(_pRequest->clientID), Con::getIntArg(_pRequest->extra), Con::getIntArg(_pRequest->extra1));
In RecordSet.cpp replace
ConsoleFunction(query, void, 3, 3, "query(%sqlStatement, %callback)")
{
   DatabaseRequest* pRequest = new DatabaseRequest;
   pRequest->pRecordSet = static_cast<RecordSet*>(ConsoleObject::create("RecordSet"));
   pRequest->pRecordSet->registerObject();

   pRequest->pSQLString = dStrdup(argv[1]);
   pRequest->pQueryCallback = dStrdup(argv[2]);

   gDatabase->postEvent(pRequest);
}
with
ConsoleFunction(query,void,3,6,"query(char sqlStatement, char callback, S32 client, S32 extra, S32 extra1)")
{
	DatabaseRequest* pRequest = new DatabaseRequest;
	pRequest->pRecordSet = static_cast<RecordSet*>(ConsoleObject::create("RecordSet"));
	pRequest->pRecordSet->registerObject();

	pRequest->pSQLString = dStrdup(argv[1]);
	pRequest->pQueryCallback = dStrdup(argv[2]);

	//If a client ID was passed in, make use of it here
	if(argc >= 4 && dAtoi(argv[3]))
		pRequest->clientID = dAtoi(argv[3]);
	else
		pRequest->clientID = -1;

	//If an extra ID was passed in, make use of it here
	if(argc >= 5 && dAtoi(argv[4]))
		pRequest->extra = dAtoi(argv[4]);
	else
		pRequest->extra = -1;

	if(argc >= 6 && dAtoi(argv[5]))
		pRequest->extra1 = dAtoi(argv[5]);
	else
		pRequest->extra1 = -1;

	gDatabase->postEvent(pRequest);
}
Simple Example:
function LoadInventory(%client ,%obj)
{
   %query = "SELECT * FROM players_inventory WHERE Owner = '"@%client.nameBase@"';";
   query(%query,"LoadInventoryCallback",%client,%obj);
}

function LoadInventoryCallback(%recordSet, %client, %obj)
{
   while(%recordSet.nextRecord())
   {
      %dataBlock = %recordSet.getFieldValue("DataBlock");
      %Qty = %recordSet.getFieldValue("Qty");
      if(%dataBlock && %Qty > 0)
      {
         %obj.incInventory(%dataBlock,%Qty);
      }
   }
   %recordSet.delete();
}
#14
04/08/2009 (12:35 am)
Just a heads up for people using this for the first time.
This line in Database.cpp
Con::errorf("DatabaseInterface::processEvent error: %s", query.error());
will return an error when using commands like UPDATE that put data into the database.
Don't be worried, it just means that the query did not return a record set.
#15
04/08/2009 (5:34 am)
I cant see this resource at all.
#16
04/08/2009 (6:45 am)
The recent site changes is causing it not to render... the GG staff is aware of the issue and I have faith that it will be fixed soon.

@Steven Chiu - Sorry, I don't have any experience with any TGEA version between TSE MS4 and TGEA 1.8.1, so I'm not sure exactly when those changes were made.
#17
04/09/2009 (10:43 am)
Awesome additions, Ari.
#18
04/29/2009 (10:37 am)
I tried to build the project with tgea 1.8.0 and VS2008 (Team System SP1). But I always make the same mistake in the link.

Database.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: class std::basic_ostream<char,struct std::char_traits<char> ....
Database.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: void __thiscall std::basic_ostream<char,struct std::char_traits<char> ....
Database.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: void __thiscall std::basic_ios<char,struct std::char_traits<char> ....


My link command line:
/OUT:"../../../game/eYe.exe" /INCREMENTAL:NO /NOLOGO /LIBPATH:"../../../../../engine/lib/SDL/win32" /LIBPATH:"../../../../../engine/lib/unicode" /LIBPATH:"../../../../../engine/lib/directx8" /LIBPATH:"../../Link/VC2k8.Release.Win32" /LIBPATH:"C:\Program Files\Microsoft DirectX SDK (November 2008)\Lib\x86" /MANIFEST /MANIFESTFILE:"../../Link/VC2k8.Release.Win32/eYe\eYe.exe.intermediate.manifest" /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /NODEFAULTLIB:"LIBC" /PDB:"../../Link/VC2k8.Release.Win32/eYe.pdb" /SUBSYSTEM:WINDOWS /DYNAMICBASE:NO /MACHINE:X86 /ERRORREPORT:PROMPT x3daudio.lib COMCTL32.LIB COMDLG32.LIB USER32.LIB ADVAPI32.LIB GDI32.LIB WINMM.LIB WSOCK32.LIB vfw32.lib Imm32.lib unicows.lib d3d9.lib dxerr9.lib d3dx9.lib d3d8.lib opengl32.lib ole32.lib shell32.lib oleaut32.lib mysqlpp.lib  kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib "..\..\link\vc2k8.release.win32\ljpeg.lib" "..\..\link\vc2k8.release.win32\libogg.lib" "..\..\link\vc2k8.release.win32\tinyxml.lib" "..\..\link\vc2k8.release.win32\libvorbis.lib" "..\..\link\vc2k8.release.win32\lmng.lib" "..\..\link\vc2k8.release.win32\d3dx8dll.lib" "..\..\link\vc2k8.release.win32\lpng.lib" "..\..\link\vc2k8.release.win32\squish.lib" "..\..\link\vc2k8.release.win32\zlib.lib" "..\..\link\vc2k8.release.win32\lungif.lib" "..\..\link\vc2k8.release.win32\opcode.lib"

My c/c++ command line:
/Ox /Ob2 /Oi /I "../../../../../engine/lib/lmng" /I "../../../../../engine/lib/lpng" /I "../../../../../engine/lib/zlib" /I "../../../../../engine/lib/ljpeg" /I "../../../../../engine/lib/lungif" /I "../../../../../engine/lib/tinyxml" /I "../../../../../engine/lib/d3dxdll" /I "../../../../../engine/lib/directx8" /I "../../../../../engine/lib/opcode" /I "../../../../../engine/lib/libvorbis/lib" /I "../../../../../engine/lib/libvorbis/include" /I "../../../../../engine/lib/libogg/include" /I "../../../../../engine/lib/squish" /I "../../../game/shaders" /I "../../../source" /I "../../../../../engine/source/" /I "../../../../../engine/lib/" /I "../../../../../engine/lib/openal/win32" /I "../../../../../engine/lib/mysqlpp/lib" /I "C:\Program Files\Microsoft DirectX SDK (November 2008)\Include" /I "C:\Program Files\MySQL\MySQL Server 5.1\include" /D "DEDICATED" /D "TORQUE_SHADERGEN" /D "TORQUE_PLAYER" /D "TORQUE_OPCODE" /D "TORQUE_ATLAS" /D "TORQUE_OGGVORBIS" /D "LTC_NO_PROTOTYPES" /D "BAN_OPCODE_AUTOLINK" /D "ICE_NO_DLL" /D "NOWINBASEINTERLOCK" /D "UNICODE" /D "INITGUID" /D "_CRT_SECURE_NO_DEPRECATE" /D "_VC80_UPGRADE=0x0710" /GF /Gm /EHsc /MT /GS- /Gy /Zc:wchar_t- /Fo"../../Link/VC2k8.Release.Win32/eYe\" /Fd"../../Link/VC2k8.Release.Win32/eYe.pdb" /FR"../../Link/VC2k8.Release.Win32/eYe\" /W4 /nologo /c /Zi /TP /wd4018 /wd4100 /wd4121 /wd4127 /wd4130 /wd4244 /wd4245 /wd4275 /wd4389 /wd4511 /wd4512 /wd4800 /errorReport:prompt

I can not get this stupid error in the project. O__o
#19
04/29/2009 (12:37 pm)
Quote:In the mysqlpp project properites, under C/C++, Code Generation, make sure you change Runtime Library to match Torque (which is normally Multi-threaded and Multi-threaded Debug).

Looks like you missed this step.
#20
04/29/2009 (2:35 pm)
I said that was something stupid that I blocked. I was mysqlpp Multithreaded DLL and Multithreaded Torque was.

Thank you very much
Page «Previous 1 2