TorqueScript Replacement to HTTPObject
by David Higgins · 02/06/2007 (6:19 pm) · 38 comments
UPDATE: httpPage now supports basic "POST" using either a URL-Encoded Query String style or a SimSet that contains SimObjects with "key" and "value" properties.
-- ORIGINAL POST (with updated code for POST support) --
This is pretty simple, you just instantiate a copy of it like so:
Now, of course, the 'getResult' has to be called after the page has actually been retrieved -- I've left this up to the developer using the code, I put a skeleton 'onDisconnect' method for TCPObject in the code, just fill in the blanks :)
Here's the code:
UPDATE: To pass parameters to your page, you can just build out a 'Query String' like you normally would for a regular browser call ("http://site.com/page.php?var1=val1&var2=val2"). One thing to note, is that 'httpPage' uses a TCP Socket to communicate with the server, and is not intelligent enough to make 'improper' url's 'proper' -- what does this mean? It means ... make sure "spaces" are "%20" and other things ... you can refer to the HTTP RFC for what is a valid request and what is not, 'httpPage' conforms to 'HTTP/1.0' requests (you can make it 1.1 if you want, by changing 'HTTP/1.0' to 'HTTP/1.1' in the 'onConnected' function, if you for some reason need extra features ... You can also, if you want ... extend it to support more then just 'get' by just, again, reading the HTTP RFC and implementing the POST or PUT, etc actions on your own. I may flesh this out to support POST eventually (maybe soon, ;p).
I put this together, using some of the code from my Forum/Blog GUI Kit I started working on a few months back, for a poster in the TGB Private Forums -- as this is TGE, TGB and TGEA friendly, I figured I'd post a .plan and share it for anyone else who may be struggling with HTTP Connectivity -- I leave the idea of "what to do with the http data" up to you ... :)
-- ORIGINAL POST (with updated code for POST support) --
This is pretty simple, you just instantiate a copy of it like so:
new TCPObject(httpPage) { };
httpPage.get("http://gearworxprod.com/index.php");
echo(httpPage.getResult());Now, of course, the 'getResult' has to be called after the page has actually been retrieved -- I've left this up to the developer using the code, I put a skeleton 'onDisconnect' method for TCPObject in the code, just fill in the blanks :)
Here's the code:
$MAX_HTTP_QUERY_STRING = 255;
[b]
function httpPage::init(%this, %url) {
%host = "";
%page = "";
if(strpos(%url, "http://") == 0)
{
%host = getSubStr(%url, 7, strpos(%url, "/", 8) - 7);
%page = getSubStr(%url, strpos(%url, "/", 8), $MAX_HTTP_QUERY_STRING);
}
else
{
%host = getSubStr(%url, 0, strpos(%url, "/", 8));
%page = getSubStr(%url, strpos(%url, "/"));
}
if(strpos(%host, ":") < 0) %host = %host @ ":" @ "80";
%this.Address = %host;
%this.Page = %page;
}
[/b]
function httpPage::get(%this, %url)
{
%this.Buffer = "";
%this.doBuffer = false;
[b]%this.init(%url);[/b]
warn("Connecting to: " @ %this.Address @ %this.Page);
[b]%this.Method = "GET";[/b]
%this.connect(%this.Address);
}
[b]
function httpPage::post(%this, %url, %data)
{
%this.Data = "";
if(isObject(%data)) {
warn("Data is Object: true");
for(%x = 0; %x < %data.getCount(); %x++) {
%datum = %data.getObject(%x);
if(strlen(%postData) > 0) %postData = %postData @ "&";
%this.Data = %datum.key @ "=" @ %datum.value;
}
} else {
warn("Data is Object: false");
%this.Data = %data;
}
error("Data: " @ %this.Data);
error("%data: " @ %data);
%this.init(%url);
warn("Connecting to: " @ %this.Address @ %this.Page);
%this.Method = "POST";
%this.connect(%this.Address);
}
[/b]
function httpPage::onConnected(%this)
{
warn("Connected ...");
[b]
%query = %this.Method @ " " @ %this.page @ " HTTP/1.0\nHost: " @ %this.Address;
if(%this.Method $= "POST") {
%query = %query @ "\n" @ "Content-Type: application/x-www-form-urlencoded\n";
%query = %query @ "Content-Length: " @ strlen(%this.Data) @ "\n\n";
%query = %query @ %this.Data @ "\n";
} else {
%query = %query @ "\n\n";
}
warn("QUERY: " @ %query);
[/b]
%this.send(%query);
}
function httpPage::onLine(%this, %line)
{
warn("LINE: " @ %line);
if(!%this.doBuffer && %line $= "") { %this.doBuffer = true; return; }
if(%this.doBuffer)
{
error("BUFFER: " @ %line);
if(%this.Buffer !$= "") %this.Buffer = %this.Buffer @ "\n";
%this.Buffer = %this.Buffer @ %line;
}
}
function httpPage::getResult(%this)
{
return %this.Buffer;
}
function httpPage::onDisconnect(%this)
{
warn("Disconnected: " @ %this.Address);
}
function httpPage::onConnectFailed(%this)
{
error("Connection Failed: " @ %this.Address);
}UPDATE: To pass parameters to your page, you can just build out a 'Query String' like you normally would for a regular browser call ("http://site.com/page.php?var1=val1&var2=val2"). One thing to note, is that 'httpPage' uses a TCP Socket to communicate with the server, and is not intelligent enough to make 'improper' url's 'proper' -- what does this mean? It means ... make sure "spaces" are "%20" and other things ... you can refer to the HTTP RFC for what is a valid request and what is not, 'httpPage' conforms to 'HTTP/1.0' requests (you can make it 1.1 if you want, by changing 'HTTP/1.0' to 'HTTP/1.1' in the 'onConnected' function, if you for some reason need extra features ... You can also, if you want ... extend it to support more then just 'get' by just, again, reading the HTTP RFC and implementing the POST or PUT, etc actions on your own. I may flesh this out to support POST eventually (maybe soon, ;p).
I put this together, using some of the code from my Forum/Blog GUI Kit I started working on a few months back, for a poster in the TGB Private Forums -- as this is TGE, TGB and TGEA friendly, I figured I'd post a .plan and share it for anyone else who may be struggling with HTTP Connectivity -- I leave the idea of "what to do with the http data" up to you ... :)
About the author
#2
02/07/2007 (4:27 pm)
Cool! :)
#3
02/07/2007 (6:45 pm)
@Tom, do I even want to ask? :) 
#4
02/10/2007 (11:49 pm)
I've been learning Ajax and was toying around with the idea of communicating to TGB with the web browser. Wacky computer tricks...
#5
02/16/2008 (4:58 am)
David, could you tell me how to send url with query string just like http://www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=12262.
#6
You can build the query string however you wish, using whatever string concatenation methods and variable data you wish ...
02/16/2008 (9:01 am)
It's really just as simple as you want it to be, new TCPObject(httpPage) { };
httpPage.get("http://www.garagegames.com/index.php?sec=mg&mod=resource&page=view&qid=12262");
echo(httpPage.getResult());You can build the query string however you wish, using whatever string concatenation methods and variable data you wish ...
#8
03/02/2008 (7:04 pm)
Great resource! This is SO much easier than trying to use HttpObject!
#9
03/02/2008 (7:21 pm)
thanks :)
#10
03/05/2008 (9:15 pm)
Is there more to using this code than simply plugging it in? I tried testing it using my MainMenuGui with a button that runs a function containing those first 3 lines of code (using a custom URL), with the function declarations added to the end of the gui file. I get a script compilation error (syntax error in line 1 -- I've gotten this before, there's never anything wrong with line 1...), but when i remove the function declarations i don't get the error and it still doesn't work. Is there anything else I need to know about implementing this code? Does "httpPage" need to be delcared as a datablock or className or something somewhere? Do the function declarations need to be in a separate .cs file? I'm still learning here..
#11
but keep in mind ... waiting for the 'disconnect' message is necessary ... you can't just call httpPage.get() and then immediately call 'getResult' and expect to get something ... it's a non-blocking operating ...
if your getting compilation errors, then you've either copied and pasted the code incorrectly, or in an inappropriate place .... without seeing your code, I can't say for sure either is true, however.
-- David
03/05/2008 (9:19 pm)
MMPlus, just toss the code into something like 'httpPage.cs' and 'exec' it from somewhere ('game.cs or main.cs') ... then use the 'first three lines' ... but keep in mind ... waiting for the 'disconnect' message is necessary ... you can't just call httpPage.get() and then immediately call 'getResult' and expect to get something ... it's a non-blocking operating ...
if your getting compilation errors, then you've either copied and pasted the code incorrectly, or in an inappropriate place .... without seeing your code, I can't say for sure either is true, however.
-- David
#12
03/06/2008 (3:05 pm)
Thanks, David. I got it working just as I needed it to. BTW, the compilation error was from a missing semicolon in the code, line 7 (%host = ""). Incidentally, is there any way to modify this to support HTTPS connections? I tried a few things, but to no avail.
#13
As for HTTPS ... that's not possible with 'httpPage', and would require a C++ library to be added to Torque to support the handshake and encryption required for SSL ... you'd then also have to add some sort of support for wrapping the TCPObject requests inside of the SSL commands ...
It is possible, just not feasable ... ;)
/Opinion
-- David
03/06/2008 (5:47 pm)
MMPlus, thanks ... I updated the post to reflect the missing semi-colon ... obviously a bad copy/paste job on my part ... since the code was working when I posted it ;pAs for HTTPS ... that's not possible with 'httpPage', and would require a C++ library to be added to Torque to support the handshake and encryption required for SSL ... you'd then also have to add some sort of support for wrapping the TCPObject requests inside of the SSL commands ...
It is possible, just not feasable ... ;)
/Opinion
-- David
#14
First, this is a great resouce a works wonderfully. My problem is when I try to do a subsequent connect to the tcpobject. I always get a bad request response. Should I create a new object everytime I do a get?
Here is my code:
and here is the applicable part of the log file:
The only thing I'm trying to do is save all chat messages. It is part of my dissertation research and I have to have the chat messages to analyze as part of the research. So if you have a better way to store the chats on a server, I'm open to suggestions!
Thank you for your help!
Brian
03/19/2008 (10:36 am)
David,First, this is a great resouce a works wonderfully. My problem is when I try to do a subsequent connect to the tcpobject. I always get a bad request response. Should I create a new object everytime I do a get?
Here is my code:
%query = $AuthServer @ "/VLE/chat.php?nick=" @ $pref::chat::NickName ;
%chat_query = %query @ "&message=" @ %text;
if($chatCounter !$= "1") // only create the httpChat object once!
{
$ChatConnect = new TCPObject(httpChat) {};
error("TCPObject intialized!!!");
$chatCounter = "1";
}
$ChatConnect.get(%chat_query);
echo(%chat_query);I set it to only create the TCPObject once. I was hoping that once would be enough, and I had problems with creating it multiple times: it also wouldn't consistently write to the database.and here is the applicable part of the log file:
*** Initial Control Object Activating DirectInput... keyboard0 input device acquired. keyboard0 input device unacquired. TCPObject intialized!!! Connecting to: www.gardenschools.com:80 http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=test message keyboard0 input device acquired. Mapping string: messageSent to index: 3 Mapping string: ChatMessage to index: 13 Mapping string: %1: %2 to index: 14 Connected ... LINE: HTTP/1.1 400 Bad Request LINE: Content-Type: text/html LINE: Date: Wed, 19 Mar 2008 17:33:27 GMT LINE: Connection: close LINE: Content-Length: 20 LINE: LINE: <h1>Bad Request</h1> BUFFER: <h1>Bad Request</h1> keyboard0 input device unacquired. Connecting to: www.gardenschools.com:80 http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=second message keyboard0 input device acquired. Connected ... LINE: HTTP/1.1 400 Bad Request LINE: Content-Type: text/html LINE: Date: Wed, 19 Mar 2008 17:33:33 GMT LINE: Connection: close LINE: Content-Length: 20 LINE: LINE: <h1>Bad Request</h1> BUFFER: <h1>Bad Request</h1> keyboard0 input device unacquired. Connecting to: www.gardenschools.com:80 http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=third message keyboard0 input device acquired. Connected ... LINE: HTTP/1.1 400 Bad Request LINE: Content-Type: text/html LINE: Date: Wed, 19 Mar 2008 17:33:39 GMT LINE: Connection: close LINE: Content-Length: 20 LINE: LINE: <h1>Bad Request</h1> BUFFER: <h1>Bad Request</h1>
The only thing I'm trying to do is save all chat messages. It is part of my dissertation research and I have to have the chat messages to analyze as part of the research. So if you have a better way to store the chats on a server, I'm open to suggestions!
Thank you for your help!
Brian
#15
Let's add another warn() statement into the onConnected, under the %query = ....
warn(%query);
I'd like to know what it is this.send(%query)'ing ...
and, let's also toss something into the 'onDisconnect', warn('Disconnected: ' @ %this.Address)
This will help me understand what exactly is going on, unfortunately, I don't have a copy of TGB sitting in front of me ... so I can't just whip up a quick test example ... so, please excuse the 'online remote debugging' process and bare with me ;)
-- David
03/19/2008 (5:47 pm)
Brian, can we try to toss one more piece of information into the mix ... Let's add another warn() statement into the onConnected, under the %query = ....
warn(%query);
I'd like to know what it is this.send(%query)'ing ...
and, let's also toss something into the 'onDisconnect', warn('Disconnected: ' @ %this.Address)
This will help me understand what exactly is going on, unfortunately, I don't have a copy of TGB sitting in front of me ... so I can't just whip up a quick test example ... so, please excuse the 'online remote debugging' process and bare with me ;)
-- David
#16
I appreciate your help!
03/19/2008 (6:16 pm)
Okay David, I have added the lines. Here are the applicable results:Activating DirectInput... keyboard0 input device acquired. keyboard0 input device unacquired. TCPObject intialized!!! Connecting to: www.gardenschools.com:80 http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=message 1 keyboard0 input device acquired. Mapping string: messageSent to index: 3 Mapping string: ChatMessage to index: 13 Mapping string: %1: %2 to index: 14 Connected ... GET /VLE/chat.php?nick=ProfB&message=message 1 HTTP/1.0 Host: www.gardenschools.com:80 LINE: HTTP/1.1 400 Bad Request LINE: Content-Type: text/html LINE: Date: Thu, 20 Mar 2008 01:13:16 GMT LINE: Connection: close LINE: Content-Length: 20 LINE: LINE: <h1>Bad Request</h1> BUFFER: <h1>Bad Request</h1> 25www.gardenschools.com:80 keyboard0 input device unacquired. Connecting to: www.gardenschools.com:80 http://www.gardenschools.com:80/VLE/chat.php?nick=ProfB&message=message 2 keyboard0 input device acquired. Connected ... GET /VLE/chat.php?nick=ProfB&message=message 2 HTTP/1.0 Host: www.gardenschools.com:80 LINE: HTTP/1.1 400 Bad Request LINE: Content-Type: text/html LINE: Date: Thu, 20 Mar 2008 01:13:19 GMT LINE: Connection: close LINE: Content-Length: 20 LINE: LINE: <h1>Bad Request</h1> BUFFER: <h1>Bad Request</h1> 25www.gardenschools.com:80
I appreciate your help!
#18
So, before setting what message is, wrap it with a simple string replace for ' ' (thats a space, again) with '%20' to conform to standards ...
Basically, the server's throwing an exception and calling it a bad request, because it doesn't know what to do with "1" or "2", because it was expecting "HTTP/1.0" or something ... using the space as the delimiter ...
Try that, and let me know ... or, optionally, just try making multiple calls with single words ... your choice :)
-- David
03/19/2008 (7:57 pm)
B, ok -- this is most likely the issue -- you'll want to most likely replace all instances of ' ' (that's a space, heh) with '%20' in your query -- more important, in your "message=message 2".So, before setting what message is, wrap it with a simple string replace for ' ' (thats a space, again) with '%20' to conform to standards ...
Basically, the server's throwing an exception and calling it a bad request, because it doesn't know what to do with "1" or "2", because it was expecting "HTTP/1.0" or something ... using the space as the delimiter ...
Try that, and let me know ... or, optionally, just try making multiple calls with single words ... your choice :)
-- David
#19
Wow, can't believe I missed that!
Thank you!
Brian
UPDATE: I added a
03/19/2008 (8:17 pm)
David,Wow, can't believe I missed that!
Thank you!
Brian
UPDATE: I added a
%chat_query = strreplace(%chat_query, " ", "%20");to the messaging, and now everything writes to the database!
#20
Should be posted soon :)
-- David
03/19/2008 (8:33 pm)
B, your email and comments on this thread ... along with some other feedback I've received recently on this resource ... has brought it back into light for me ... I'm working on implementing an httpPage.post() ... that will accept a URL and a SimSet with "name/value" for the post data ... or, optionally, a string that is in the form of the query-string ... Should be posted soon :)
-- David
Torque Owner Tom Bentz