Game Development Community

Would a file transfer system built into T3D have value to anyone?

by Vince Gee · in Torque 3D Professional · 06/10/2013 (6:35 pm) · 6 replies

Hey guys, I am adding finishing touches up to OneWorld and I have a question. I designed an efficient file transfer system for OneWorld built inside of T3D. Would something like this have value as a standalone product and what would be a fair price point for it?

The current implementation works like:

For Server downloading to Clients.

1) Server Starts
2) Server Creates a File Transfer Host Object
3) Server listens on Port + 1 for clients
4) Client Connects
5) Client request File List
6) Server responds with a list of filepaths that have been queued up for download along w/ the crc for each file
7) Client compares CRC from the server against the CRC of the file on the client machine
8) If Client CRC != Server CRC, it queues the file for download
9) Server Send List Complete Signal
10) Client requests file queued in non CRC matching list
9) Sever streams file contents over TCPIP connection to Client
10) Client writes stream to Disk
11) If non matching CRC list of files > 0 go to step 10

For Clients uploading to server

1) client requests to send a file to server
2) Server makes sure no one else is writing to the file and CRC's don't match
3) If no one else is writing to the requested file, server send ok to client
4) Client streams file to server
5) Server buffers file to disk

It all works, and has been tested pretty thoroughly via OneWorld. So what would this be worth, let's say I even provided documentation :)

Vince

#1
06/11/2013 (5:57 am)
I'm not trying to rain on your parade but I want to make it clear to those that reads your post that Torque already supports the expected game environment synchronization of assets that most have expected from quake/idtech and unreal engines.

Stock Torque Pros:

Torque already has this ability since early TGE days to transfer assets from server to client as Starsiege Tribes and Tribes 2 supported it across the same UDP based protocol and communication port that is used for game session handling. Whenever a client connects to a server that has newer or different art assets that the mission file (of which is always transferred via file xfer) and/or datablocks/profiles require then the client requests and server sends the files.

The only difference being that in the Tribes games the server restricted the downloads to only some textures and sound files, while the terrain, models, and difs were flagged as non-xfer able since majority of them were too big and would just cause long downloads at the time. While Torque no longer blocks such assets from being downloaded the last time I checked it'll transfer any asset file requested by the client that the server feels is essential to the currently loaded game environment.

Stock Torque Cons:

Now as for what Torque does not support is the client sending any files or data unrelated to the game session to the server. So those of you who were wanting to support something like Source Engine custom style sprays then you're currently out of luck as that requires the client to upload custom pray images to the server to distribute to all the other clients.

Stock Torque also has very slow transfer speeds of files to the client as the same packet rate limiting settings that you expect to apply to the game session data actually applies to everything being sent across the UDP socket therefore affects file xfers too. Torque also does not support an alternative download source for missing asset files as seen in later quake/idtech and unreal engines. Specifically the download_url cvar.

Conclusion:

Obviously vanilla Torque currently has some short comings in regards to file transfers, but it really all depends on the individual game's needs to know whether or not this is acceptable. :)
#2
06/11/2013 (6:31 am)
@Nathan,

It's all good, but I think you might be mistaken.

Stock never worked

I've never been able to make the *stock* file transfer system work. If you go into some random folder in the game/art folder and delete it, say a rock. At least via my testing, stock torque won't download the rock to the client. I do remember back in the tge days there was a file sync program but I never remembered it working correctly.

Throttling doesn't apply

I am using WinSock directly not going through NetConnection Class which is what uses the settings for PacketRateToServer etc.


Also, the initial file transfer download occurs BEFORE the client establishes a connection the game server. This prevents a timeout to the server while the client is downloading.



#3
06/11/2013 (7:54 am)
I can think of a few cool things to do with this - that are also great ways to get into trouble.... lol.

I've always wanted to make a multiplayer game where players could upload custom "barks" and avatar images - but then it's either the Wild West or I have to find a way to screen and approve the content. Sadly, most of my coolness involves a lot of extra work....
#4
06/12/2013 (2:20 am)
I do like that as yeah I thought Torque supported clients to download new assets for new levels etc but if it didn't work this seems like a superior upgrade but for the base engine I see it being limited to just some assets being the case so to not allow too much to be downloaded through it and possibly cause security issues.
#5
06/12/2013 (6:48 am)
Well for Oneworld, I needed it to download all content so people could just update the server and then when they connect get all the content. So I guess it has it's purposes.

I just wanted to know if people would be interested, cause it would take me a week or so to rip the code out and write documentation for it. If no one sees a value, then I prefer not to take the time.
#6
06/12/2013 (12:39 pm)
I definitely think there's a use for it but now that we have OneWorld we don't really need it!
If it had existed a year or two ago it would have saved us a lot of time cooking up our own.
I suspect at the price point you've set for OW people will just buy that.
Did I mention I'm super pumped about getting One World integrated into our code? It's going to be a massive improvement. It'll really make the separation between client and server more apparent yet tie them together seamlessly. Yeah, something like that...