Developer Interview: BitGap Games' Xenocell and IndieGoGo

Developers in the Torque community have been watching a number of great projects advancing towards the finish line over the years, and one of the games that we've all had our eye on is Xenocell. BitGap Games has been working hard over the years to create a large, persistent world strategy game, and with their final week to go on IndieGoGo, it's shaping up to be an amazing experience. Just take a look at the video below to see what an exciting experience Xenocell is shaping up to be.


You can also find more information about their IndieGoGo project (and support the development push!) at their campaign site!

We had a chance to do a virtual sit-down with Konrad Kiss to discuss Xenocell's development and their IndieGogo campaign.

---

Please describe Xenocell in your own words.
Xenocell is a multiplayer action-strategy game in a sandboxy environment playing in a huge, persistent online world. It's a game that tells the story about the survivors of a terraforming mission in the Alpha Centauri system. It features a little over 4k Torque 3D type missions where each level can be controlled by a player faction (clan). Players can mine resources and build just about anything in the game.

What was your role on Xenocell?

I'm a co-founder of Bitgap, the studio that's been working on Xenocell for the past 4 years. I'm a programmer on the team and I've been directing development on it from day 1. But as all indie developers, my time is often spent working outside my normal role. I'm also responsible for the UI design, most 2D and various promotional graphics along with plenty of asset conversions.


What makes Xenocell unique?

It's really hard to find a game that you can describe as unique nowadays. It might sound as a cliché, and it certainly sounds more meaningful once you have played the game, but with Xenocell my aim was to create a game that is truly social. Where banding with other players is not necessarily about PvP only. I believe that a multiplayer game is only fun when you really feel the multiplayer aspect - not really meaning many players, but getting the impression of a working, living, breathing society that is not supported by everyone role-playing, but by core design choices instead.

You work with others because you depend on others. While you can solve certain tasks, it makes sense to work and build together, to explore and research with other players as everyone has their role within these small in-game player societies. For a long time, I thought of the idea of Xenocell as an RTS where every player is a single unit with their own choices. In Xenocell, running a clan is not a one man task. You have financial, military and research advisors, officers below and above your rank. It's a system that helps players work together without sacrificing free will.

With that said, the social aspect extends far beyond what we usually see with other games as Xenocell makes use of existing popular real-life technologies and incorporates these into the gameplay. For example, you can deploy an alarm trigger in a narrow canyon somewhere in your territory and have it report breaches to your clan via Twitter.

What was your inspiration to create Xenocell?

I created a web-browser based 3D (VRML) game in around 2000 where the player had to program virtual robots to fight in teams. The game was very popular and soon professional writers came forward with short stories and novels that played in the fictional futuristic world of that game. One of the stories mentioned a colonization ship sent to Alpha Centauri that was for a long time thought to be lost. I thought that it would be really nice to explore this story. In a way I wanted to see a continuation of that universe which I've fallen in love with, but I also wanted to create a game that is massive and - while still retaining a strong strategic aspect - was just as much a personal experience. So interestingly the story was born first, and then the rest of the game.


What was your development process like?

We first decided to probe the technology we chose to build Xenocell upon for months - at that time it was called TGEA which eventually became Torque 3D. The steep learning curve of Torque forced me to completely redraw the development schedule after a while, but I didn't want to change over to another technology, because despite the initial hardships I learned to love the infinite possibilities that came with having full source.

The first year went by very fast with mostly me learning about the engine, the various components in it and getting familiar with the asset pipeline. Art was never our strong side, so we picked up content packs to prototype ideas with until we contracted specific artists towards the past year. Some assets stayed, some were redesigned or exchanged, but having something flashy on the screen was always good for internal morale. :)

We had an initial game design document of course and a schedule - both of which changed continuously during development. We came up with new ideas, found out that some didn't work as we hoped they would or required more resources than we were willing to spend. We also made the error of underestimating the creation of backend services that manage servers, request and cache data, authenticate users, solve global player chat, etc... These alone took more than two years and are still being worked on as new technologies become available.

Currently we are wrapping up moving over to the no-SQL DynamoDB from the previous MySQL-based solution which is a painful experience, but it is well-worth it because in the end this leaves us with a perfectly scalable data provider, incredibly fast data fetches and less money spent on administration and general upkeep.

Choices such as this one were frequent, and while I believe that one of the many benefits of being an indie developer is being able to quickly adapt to available technology and the market, it also made development take over twice as much time as we originally anticipated - spending about 5 times as much money along the way.


How many people worked on the dev team? How did you work together?

The team was constantly changing - me and my partner were the only people working on the game full time. While he was handling the business and financial aspects of the company, I was managing development of the game.

We had a number of contractors from all over the world. I'd like to specifically mention Tamas Rehorovszky (programmer), Solaran Laughlin (writer), Kevin Turchik (lead artist), Danila Laurentiu (artist), Maxim Lyulyukin (artist), Keith Murphy (narrator, voice actor), Aaron Blackmar and Jonathan Chio (composers). There were plenty of other people who helped tremendously with the game and I owe a huge thanks to the GarageGames community for the tremendous help we've received over the past years. The final number of people who have helped Xenocell with their work is around 40!

We worked together over the Internet of course. I'm notorious of my hatred for instant messaging apps, so I always preferred email over chat to keep in touch. Most of the time managing the project wasn't at all a task. It didn't need to be managed as it was just happening as it had to - which I'm very thankful for, and I credit that to the awesome people whom I had the chance to work with.

How long did it take to create?

The original idea of Xenocell dates back to 2003. There were about 3 different prototypes in various languages until the final design docs were written for the game in around 2007. The official green light was received on the 1st of March in 2008. I clearly remember, because my daughter was born that day. I was leaving the hospital for the night when I got the call that we were going to create Bitgap and Xenocell.

Of course it took me some time to get started with the actual work on the game, but when I did, I was doing it full time from home. This was an incredible possibility and the fact that I was able to stay at home and spend my daughter's and son's first years with them will stay with me all through my life. Still, working from home is not all fun. I've had some experience with it having spent about 4 years working from home previously on another company of mine, but this time it was different - the project was huge, my kids were little and the technology seemed way too complicated at the beginning.

Working from takes some getting used to and you either feel like you're at home or you feel like you're at your workplace all the time. Many nights I'd spend 12-16 hours a day learning and working on the game - weekends included. I'd take my entire development rig with me on summer vacations to Croatia, France and Spain... It is very hard to stop once you become obsessed with what you work on. Eventually I think I've learned to keep a healthier balance, but in this aspect it seems like it took 20 years to create.

Otherwise, it took about 4.5 years. :)


How did you accomplish QA and beta testing?

Me and close friends were the first line of QA until the game had the server and the client completely separated and even debugging the game relied on certain services being online all the time. At that time the two became one and the same.

Beta testing was not quite beta testing as there were always radical changes after each test and there were still new features being worked on. I chose to pick a few features from the game that needed testing and hide everything else as well as possible to make testers concentrate on the interesting bits from the test's perspective. This made tests no fun at all, of course, but provided me with better feedback. Unfortunately with the exception of a core team of testers, most people on the beta team lost their will to test later on, and I completely understand why. Still, I had to make a choice to either make use of testing and test results or bow to what most AAA games call beta tests - which essentially are demos of the final game - and push out all features at once for testing. I was pretty sure about that not being a good idea, so I instead chose dry testing of a few simple features at a time hoping that eventually our testers would get to play the whole game in its entirety.

What software/tools did you use to create the game? Why did you use those particular tools?

Outside of working with Visual Studio and Torsion - both of which are essential tools when working with Torque, there were many tools along the way.

For example we created our own terrain generation algorithms but we initially worked with L3DT which I still love, and Grome which was pricey but great tool (though paid upgrades were way too frequent for my taste to keep up with the latest versions all the time). When we saw that we were going to go with procedurally generated terrains instead of having a few created by hand we created a prototype within the game and later another one as a stand-alone batch of scripts and programs. This batch heavily relies on Photoshop CS6 and we use PHP to generate the final Torque 3D ready files for the terrain height and layer maps.

One of the tools I love the most is pureLight which we used to light interiors of procedural player bases. This tool alone can boost the aesthetics of a game so much that it looks beautiful just by itself without having any pattern in the diffuse texture. It's phenomenal. It's not always the most intuitive tool for me, but I got used to working with it, and have even written several other tools that make my job easier when I need to work light tens of models as soon as possible.

For deploying the game we originally used Molebox VS which ended up to be a terrible choice. It has caused us much grief through triggering false positive anti-virus alerts. Eventually we went with an engine-specific solution. To offer incremental binary patches we use Puchisoft's Dispatcher, which has been an awesome took and amazingly cheap compared to other similar tools.

We of course used SVN and Git for versioning, Google Docs for managing documents within the team, Gmail for handling emails for the company and the dev team, GTalk and Skype for the rare occasions when IM was needed and VBulletin drives our forums at xenocellforums.com.

The most interesting tools we use however are specific to the game in my opinion and they are a part of our management services array. These include content generators, server state viewers - both map based and instance based, server health overviews, the master server, our chat server solution which presumably is able to handle over 100k concurrent global users (and does not rely on any other software such as IRC) or the entire deployment automation where it takes a single click to create and distribute a new patch for the game. I could probably go on for days about these tools as I'm really proud of them.


Describe 2-3 of your biggest technical hurdles and how you overcame them. Give as much detail as possible, to the point of getting extremely technical.

Well, let's see. One of the hurdles certainly was hosting the game. When we started the game, we bought 5 production servers for development and beta tests, which seemed overkill, but we assumed that we'd have to host our entire game on these and we were hoping that we were going to be able to handle at least 5k users with those servers.

These servers included our web server, the data provider service and its database and the game servers. The first tests have shown me that this was not going to work. I immediately canceled our hosting plan and decided to put things on hold until we can find a scalable solution as there didn't seem to be a way where we could be profitable by having to host and manage servers with a wide enough bandwidth in the heart of Europe for a primarily US based target audience. Everything about that seemed wrong, so I went looking for a solution. Computation power in the cloud was already gaining some traction and I checked out Amazon Web Services which I almost instantly fell in love with. I made some calculations and it was clear that we would be able to host the game as we wanted to. We could add new servers or cut back on their number anytime - putting the server side scaling of the entire game in the hands of the master server automation. I always felt more comfortable with machines handling critical tasks, so I planned to go this way.

I studied AWS and its services to see what we can make a good use of - what services would decrease our server costs or help us solve problems yet unsolved. The effect of having these services available to us in an accessible price range and being able to reserve our servers at any location around the world was so significant, that it - for the nth time - made me completely redesign the game's core gameplay features. This is when the final hex based map was born, and instead of generating zones on the fly we decided to create a LOT of zones and cache them in the cloud.

Currently we make use of EC2 to host all of our servers - from the actual dedicated Torque instances to our web server, chat and versioning services and a high end server started on demand that generates our patch files. We also use S3 with CloudFront to store almost all of our online files from www caches to player profile images and mission files and to be able to offer these at incredible download speeds across the entire globe.

We use CloudWatch to get alerts about when any of the services and servers experience huge traffic or other extremes. Some of these events are handled automatically, while most will issue us an alert through yet another AWS service - SNS. RDS handles our MySQL based data from which we are phasing out as much data as possible into DynamoDB currently to better be able to scale data access which has proven to be one of the more serious bottlenecks of our entire server-side operation. Amazon SES takes takes care of sending out automated emails while we use Route 53 as our DNS and Amazon Glacier takes care of securely backing up every possible file related to the development of the game on a daily basis.

As a reference, our October bill for all this was $430. Of course once many servers are online, this will increase significantly, but the best side of using cloud based compute resources is that costs are highly predictable.

To mention another hurdle, it was without a doubt Torque's steep learning curve. Eventually Mich and his awesome docs team have begun to clear up much of the mist around Torque 3D, but the really interesting bits had to be explored. The inner workings of the rendering system, the TorqueScript compiler, the DTS loader and the terrain and forest system were all completely undocumented when we had to start working with them.

Our only choices were to either go and choose another engine or sit down and look at code for weeks if not months and understand what was going on. I chose the latter and I am very happy that I did so. Once I felt I started having a firmer grasp on Torque, I managed to work as a contractor with some of the people I most look up to in this or the entire game developer community. To be able to strengthen our budget, I took on contract work and got a chance to work with awesome people like Ben Garney (ex-GG, PushButton Labs, now working on something really cool @ The Engine Co.), Phil Shenk (Diablo 2 lead artist) and Kevin Klemmick (Diablo 2 lead programmer) among other very talented people. If I hadn't overcome my initial disappointment in how hard it was to get started with Torque, I would never be able to brag about these! That would be a real shame! :)


Why did you choose IndieGoGo as your crowd-sourcing platform?

I assume this is more like a question about why we didn't go with Kickstarter. :) Both my partner and I are Hungarian nationals, and Bitgap is a company that was incorporated in Hungary. We currently do not have a US or UK citizen in our permanent ranks. Kickstarter uses Amazon Payments which requires a US (or UK since 31., Oct.) social security number for payouts. My family is part American - living in Las Vegas actually -, but the tax nightmare that would follow with channeling all contributions through a US individual to a Hungarian company was something we wanted to avoid if at all it was possible.

How has your experience been with IndieGoGo so far?

So far I like IndieGoGo a lot. The staff is very helpful, and the fact that they accept international projects is something I wanted to value by working with them through this campaign. I'm not an idealist - I also understand that this alters our chances for success. I hope we can still succeed with them - and if that doesn't happen, I will do my best to give the game another chance on Kickstarter.

Would you look at IndieGoGo or Kickstarter for future projects?

I'm not yet familiar with Kickstarter, but I'd hesitate to look at either of them unless I have absolutely no means to fund a project - and if there's a deadline until which I need to launch a project.

I've been getting feedback from certain magazines that are drowning in Kickstarter and IndieGoGo news. While one of the benefits of starting such a campaign (besides the project being funded) is the news coverage, it is becoming less and less significant as more and more projects go that way - some of which are clearly not in need of external funding. Some projects abuse the system and yet other projects depend on field celebrities who don't need to come up anything specific to get funded. Both of these are somewhat repulsive to me and I find that people often generalize these traits onto other, valid projects.

So in short, only if I really must would I try to crowd-fund a future project. Not only potential failure hurts the brand, other dubious projects are often associated with the platform in general. I would probably look for individual investors before I'd try my luck with crowd-funding.

For Xenocell our budget is good until around mid-December. If we don't succeed by then, then Xenocell will most likely be put on ice. This was my only reason to try crowd-funding as it seemed to be the only viable, short term solution to make the game happen.

Give us a good synopsis of what you want players to experience when they start up Xenocell.

I want them to instantly feel the immense size of the world and their unrestricted freedom in it to create, share and have fun - either solo or with others. My initial plans are not shooting for the moon, but eventually I plan to add to the game, focusing on every in-game character class and creating content uniquely for them. This, in essence would form yet another layer of bonding between players of the same character class. In general this is also my policy on expanding the game further. I want to see features succeed that add to both the social aspect of the game and the gameplay.

When the player enters Xenocell, it does not happen in a starting zone. It happens at a pseudo-random place in the world - which is of course a carefully picked position to make sure that the player meets other players but does not land in an overcrowded zone. The player has temporary protection until the basics of the game are learned. A second layer of learning is leveling up in experience from level 1 to 20 in a few short days probably. Experience levels are not included to add a role-playing element, but more to be able to protect and balance the game in favor of new players. Although the real game begins on level 20 where you no longer gather experience, the character will still be able to further evolve through custom items and skills and this is when the real strategic aspect kicks in.


Tell us why players will keep coming back to Xenocell again and again.

I am hoping that they are going to come back for the friends they find playing the game. Of course I really hope Xenocell will be thought of as a great game, but the game will be nothing without a great community, which will be crucial in making Xenocell a fun and financially successful game.

---

For more information on Xenocell, check out their website at http://www.xenocell.com. For more information on their IndieGoGo project or to support Xenocell, check out their campaign.