Game Development Community

Resurrecting MD5

by Svengali · in Torque 3D Professional · 06/08/2011 (10:53 am) · 14 replies

Ok, let's see if we can stump the community for a third time in a row, I'm trying to resurrect the old MD5 resources, as they do not work in T3D beta 3 and 1.1 either. I've gotten the code to a compliable point, but there's and issue with the return and then it crashes the engine. Can someone take a quick look and tell me what's wrong?
At the top of md5.cpp in core/util:
#include "console/engineAPI.h"
#include "console/consoleTypes.h"
static const char Base16Values[] = "0123456789ABCDEF";
Then at the bottom I've added:
DefineConsoleFunction( getStringMD5, const char *, (const char *What),, "")
{
   MD5_CTX * context;
   U8 digest[16];
   
   static U8 digestB16[128];
 
   U32 len = dStrlen (What);

   MD5Init (context);
   MD5Update (context, (U8 *) What, len);
   MD5Final (digest, context);
   
   for (U32 Index = 0; Index < 16; Index++)
   {
      U32 Byte1 = Index * 2;
      U32 Byte2 = Byte1 + 1;
      
      U8  Value1 = (digest[Index] & 0xF0) >> 4;
      U8  Value2 = digest[Index] & 0x0F;
      
      digestB16[Byte1] = Base16Values[Value1];
      digestB16[Byte2] = Base16Values[Value2];
   }
  
   digestB16[32] = 0;

   char *ret = Con::getReturnBuffer(32);
   ret = (( char *) digestB16);
   return ret;
}
I've got way too much time I've now spent on this, and I'm sure it's something simple that I'm missing. If I can get this to work, or if anyone can help me to fix it, then I figure I'm going to write up a quick resource on using this old method. All the hard work was done by others before in past resources.

About the author

Creator of SvengaliGames, and developing the once up coming title NightLife: The MMORPG Of Dark Horror. Now named ATROX NOCTIS: Daemons Cradle- www.svengaligames.com


#1
06/08/2011 (10:57 am)
Had some help with this, from #19 is for Torque 3D. But the previous version is TGE/A

www.garagegames.com/community/forums/viewthread/124128
#2
06/08/2011 (12:51 pm)
Did you ever get this to work? The codes is basicly the same as in all the resources, and just bombs. I'm not sure if it's compiler differences or what, but it doesn't appear to be compatible with the most recent versions of T3D. Grant it my C++ is rusty, but I've made plety of mods to the engine in the past, resouces and original, so I'm just confused.
#3
06/08/2011 (12:52 pm)
I've got the #19 one working in T3D 1.1 B3. Not tried 1.1 final yet, as I'm waiting for middleware to catch up before I tackle 1.1 final migrations.
#4
06/08/2011 (12:54 pm)
I'll be giving this a go in the next few days when I update the GRS to support Torque side MD5. So I'll re-post again if you still have problems, and let you know how it goes.
#5
06/08/2011 (1:04 pm)
Yeah, thanks! I'm waiting on the same before porting to 1.1, but it's blowing up beta 3 too. This is like one of the last things in my project to complete, but I have a crapy work around so I can keep moving. After you do get it to work in 1.1=, I'd love to here back from you. There something different in b3 and 1.1 that I'm just not getting yet, lol.
Thanks for your help, I look forward to hearing from you, and if I get it to work I'll post back too.
#6
06/08/2011 (1:04 pm)
Ewww! That's not right at all! Please change these last lines of code from
char *ret = Con::getReturnBuffer(32);  
   ret = (( char *) digestB16);  
   return ret;
to
char *ret = Con::getReturnBuffer(33);
   dStrcpy(ret, digestB16);
   return ret;

That should work now! :)


UPDATE:

Here is working code for Torque 3D 1.1:
#include "console/engineAPI.h"
#include "console/consoleTypes.h"
static const char Base16Values[] = "0123456789abcdef";

DefineConsoleFunction( getStringMD5, const char *, (const char *What),, "")
{
   MD5_CTX __context, *context = &__context;
   U8 digest[16];
   char* digestHex = Con::getReturnBuffer(33);
 
   U32 len = dStrlen (What);

   MD5Init(context);
   MD5Update(context, (U8 *) What, len);
   MD5Final(digest, context);

   for(U32 i=0; i<16; i++)
   {
	   digestHex[i*2   ] = Base16Values[ digest[i] >> 4    ];
	   digestHex[i*2 +1] = Base16Values[ digest[i] &  0x0F ];
   }
   digestHex[32] = 0;

   Con::printf("MD5 result %s for \"%s\".\n", digestHex, What);

   return digestHex;
}

Now it isn't entirely bug free. For whatever reason I am not getting the expected hash result. For example "Hello World!" MD5 hash is ed076287532e86365e841e92bfc50d8c but this script exposed function is saying the hash is c33841df39fc6c85b6982ce6d07789d4 which is incorrect.

I compared Torque's md5 sources with those of Command Line Message Digest Utility and they're identical except for int's being changed to U32 in Torque's copy. I haven't thoroughly looked to see if by changing a signed integer (int) to an unsigned integer (U32) would actually change the digest result or not, but that might be the case here.
#7
06/08/2011 (2:34 pm)
I saw it some where in the forum why it doesn't match up. I believe it has to do with the built in md5 functions not being true md5. If I can find the post again I'll throw a link in here.

Compiled beutifuly, now to test...

EDIT: returns a hash too, I don't believe its correct but lets see... No not matching but great point to figure it out from, thanks guys! I'll keep every one informed of any progress I make, I hope you guys do the same! Thanks again.

Here is a copy of the code that supposedly fixs the hashing:
Click Here
#8
06/08/2011 (3:09 pm)
I fixed it. I got it backwards as the sources from Command Line Message Digest Utility are using unsigned integers (uint32) and Torque is using signed integers (int) in md5 sources. In order to generate tried and true MD5 hashes in Torque you must change the stack variables in MD5Transform() from int to U32 as shown:

// current function in Torque
void MD5Transform( int buf[4], int in[16])
{
    register int a, b, c, d;
...
}

// fixed stack variables to conform to real world MD5
void MD5Transform( int buf[4], int in[16])
{
    register U32 a, b, c, d;
...
}

Before you can use the unsigned 32bit integer type U32 you must include the consoleTypes.h header like this:
#include <memory.h>		 /* for memcpy() */
#include "md5.h"
#include "console/consoleTypes.h"    // << New! Needed this for U32 type

This will break MD5 hash generation and checking compatibility over the network with other Torque projects that don't have the same modification, such as joining a game server without this fix with a game client with this fix.

So we now have a real working string MD5 hash generator. :)

Also, be sure to comment out or delete the line using Con::printf(); in my working torque script exposed function in my previous post above as that was just to make sure I was generating the MD5 with the right function argument.
#9
06/08/2011 (3:12 pm)
Wow, your good man. Thank you very much for your help! After I get off tonight, if it's cool with every one I'm gonna write this up into a resource for everyone else.
#10
06/08/2011 (3:14 pm)
Resourcing this is definitely a good idea, have at it! :D
#11
06/08/2011 (4:09 pm)
@Svengali - are you looking to connect to PHP with MD5 hashing?
#12
06/08/2011 (4:23 pm)
At one point I was, but obviously that didn't work for me. Now we are doing the same thing with a customized torque server. It's the common point before loading into our world, once in this zone, which spawns every one as an observer, you can meet up and chat with your friends before going in, or create new characters, or admin your personal account or guilds. It's a completely weird way of doing things, but it keeps all of our db servers' info isolated from exposure, from what would be bad programming on my part due to the fact I've been writing php for about a week.
#13
06/08/2011 (4:27 pm)
The link I used, not sure about Nathan's code works with PHP. Just make sure you're not open to sql injection if you use a MySQL database. Always good to also lock down your game servers to remote web servers by also only allowing specific IP addresses either by php script or firewall.
#14
01/02/2013 (9:11 pm)
in "source\core\util\md5.cpp" i got this:

"#include <memory.h> /* for memcpy() */"

where this memory.h located?



i also got this on Engine/lib/pcre/config.h

"/* Define to 1 if you have the <memory.h> header file. */
#define HAVE_MEMORY_H 1"

is that related to 1st memory.h ?
may be not.