2D Blogs: OS X Rewrite
by Michael Perry · 11/27/2012 (11:42 am) · 55 comments
Torque 2D Blog: OS X, Cocoa, and Platform Work

Kickoff
Greetings everyone. It's another 2D blog day! When time allows, someone on the 2D team will post a blog or resource related to Torque 2D and/or iTorque 2D. The majority of the posts will come from myself, but don't be shocked if someone else takes over for me from time to time.I don't have much time to write a super detailed blog. Melv and I are in the middle of a very intense development cycle. We have taken on a task that is long overdue and requires at least two developers working simultaneously. Compared to previous blogs, this is a less dazzling feature, but it must be done.
Before proceeding, here's the usual disclosure:

FULL DISLOSURE
Do not take the content from these blogs as law. I'm never going to say "this will be in x.x version" or "this is ready to be used right now". There's going to be a lot of R&D discussion. I might even post a discussion I had with Melv that ended up being scrapped. It might be useful for you to understand how we communicate internally, or amusing to see the mad scientists we really are. I will not post timelines. I will not post release dates. I will not commit the team to something we cannot deliver on.

Goodbye Carbon
For the past three years, Torque 2D and iTorque 2D have been getting nailed by Apple updates. More and more code the engine relies on was being deprecated. This primarily affected the desktop editor and runtime, but the iOS side took an occasional hit as well. I'm talking about the use of Carbon. For those who have not been developing Apple products, Carbon is the old framework for building desktop applications. Apple changed gears and has been pushing its new framework called Cocoa.For a while, internal Torque devs like myself would have to swoop in and apply a bandaid patch that kept the engine at least compiling. When Mountain Lion dropped, all of Carbon hit the deprecation list. Suddenly, the engine stopped compiling. Once more bandaids were applied, some features would flat out not work. For example, trying to go fullscreen would result in a Fatal ISV.
In summary, the OS X version of Torque 2D and iTorque 2D is completely borked as long as it uses Carbon. We've put off the inevitable for too long. It was decided earlier in the year that we would block out a large amount of time to "make things right." We started work in the 3 Step Studio trunk two weeks ago. First and foremost, Carbon has been completely removed as a framework. How did that work out?

No Carbon, No Runtime
All stdlib functions still work. Apple would not dare drop support for these functions. I'm mainly talking about basic file I/O and memory functions. However, this is just a small percentage of the OS X layer for our 2D engine. Everything else was littered with Carbon calls. We were left with one option. Scorched earth!As I mentioned previously, Melv and I are attacking this task from both sides. We are both getting our hands dirty with Cocoa (Objective-C), but we are also specializing a bit. Melv has been cleaning up the messiest parts of Torque "Platform Abstraction" code, as well as implementing long overdue systems like unit tests. I have taken the lead on the Cocoa implementation and helping get systems back online.
We started by adding a new platform layer called "osx" and generated a completely fresh Xcode project. This new layer resides right next to win32Platform, iosPlatform, and macCarbonPlatform. Before we could even write new code, we created stubs for every class and function Torque expects. For the first time ever, we had Torque compiling on OS X without the use of Carbon.

Umm, What Next?
It was at this point we kind of just stared at the code for a bit. Torque's startup sequence and general platform layer was a god awful mess. There was no consistency between platforms and a lot of how things worked just did not make sense. While Melv was cleaning up that area, I got us moving by creating a basic Cocoa application that worked with the rest of the Torque source code. It was painful, but a major milestone.
It Lives!
Next, we created a prioritized list of systems we had to bring online one at a time. As an example, we knew we needed memory, file I/O, threading, and network code working before anything else. After much pain and labor, we had a very clean startup process on OS X. We have gotten as far as executing a main.cs script, loading modules, and running the engine in a loop. That doesn't sound flashy, but any Torque developer who has looked at our past Mac layer should be shedding happy tears right now.
Forging Ahead
We are getting into the thick of it now. Before the end of next week, we should have all video, input, and basic events running. From there, it's just marking items off a list. If at all possible, each system will have its own set of unit tests. These unit tests will work on all platforms, so we will know immediately if a change on Windows breaks OS X or iOS. Along with some discipline, this kind of effort should keep all of our platforms synchronized and running into the distant future.Again, UNIT TESTS. I'm sure our new QA lead will love to see this. I know it makes Melv and I very happy. Not only that, Melv's work can transfer over to Torque 3D as well, so we'll be talking to Dave Wyand about it soon.

Other Platforms
Developing a true Cocoa layer for Torque 2D has reaped many other benefits. Torque's general platform code has been cleaned up, with the work propagating to Windows and iOS. Additionally, our in-house knowledge of Apple frameworks will have tripled before we are finished with this task. Finally, I took a snapshot of an "empty platform layer" for future reference. Melv and I have a much greater understanding of what it takes to go to other platforms, which will be quite useful when we target new devices.
Conclusion
This is my least thought out blog yet. I kind of just started writing until I felt like I did not have anymore time. I gotta get back to the code and get our OpenGL setup. Please feel free to ask questions in the comments and I will do my best to go into further details, like "Yes we are using NSWindow and NSView. Yes we are using blah blah". Until next time!About the author
Programmer.
#42
12/01/2012 (8:48 am)
Quote:after the cake is gone it's right back to finishing this thing, mkay?After the cake AND honey whiskey. Then on a drunken sugar rush, I shall code again.
#43
But seriously, how long is this engine going to take to develop (hasn't it been like 4 years already?). Do you guys even have a schedule or are you just taking the Valve approach... release when it's ready.
I think it's almost a joke how long this engine is taking, especially by a big company that specializes in 2d engines. Only thing that keeps me believing in it is Melv and Michael are on the project.
Would love it if Melv can post more too.
Heres my request for a post... I would like to get your take on the strengths and weaknesses of other engines out there and compare how Torque2D will be different and better than them when it gets released.
12/06/2012 (1:33 pm)
@Michael - Love the blogs... please keep it up.But seriously, how long is this engine going to take to develop (hasn't it been like 4 years already?). Do you guys even have a schedule or are you just taking the Valve approach... release when it's ready.
I think it's almost a joke how long this engine is taking, especially by a big company that specializes in 2d engines. Only thing that keeps me believing in it is Melv and Michael are on the project.
Would love it if Melv can post more too.
Heres my request for a post... I would like to get your take on the strengths and weaknesses of other engines out there and compare how Torque2D will be different and better than them when it gets released.
#44
EDIT: Alright. I know Eric is working on a blog that should probably come before I do a full rundown of strengths and weaknesses. I'm going to give him breathing room to do that first.
12/06/2012 (2:02 pm)
Quote:But seriously, how long is this engine going to take to develop (hasn't it been like 4 years already?)Kind of. The 2D technology has gotten sporadic pushes in the past four years. iT2D and T2D have always been developed separately, not really having a team larger than 2 or 3 at any given time. The past year was the first time since mid 2009 that the engine got a serious development push. T3D has always gotten the bulk of the company resources, but 2012 has been different. We've managed to do more with the 2D technology, including unification, in the past 12 months than we have accomplished in three years.
Quote:Do you guys even have a scheduleYes. There is a schedule and it's pretty aggressive.
Quote:I think it's almost a joke how long this engine is taking, especially by a big company that specializes in 2d engines.We're still not that big. There was essentially one dedicated programmer on iT2D last year, with a couple of others who would jump in to help with a few bug fixes. This year we managed to throw about 8 programmers at the 2D technology, 2 of which dedicated just to core engine changes. Basically, the tech is finally getting the resources I've been wanting it to get since I first started focusing on iT2D.
Quote:Only thing that keeps me believing in it is Melv and Michael are on the project.You have both of our appreciation =). It's community members like you that keep us honest and drives us to be better.
Quote:I would like to get your take on the strengths and weaknesses of other engines out there and compare how Torque2D will be different and better than them when it gets released.Oof. This is going to take a bit more time than I have for this post. I'll split out a second post for this.
EDIT: Alright. I know Eric is working on a blog that should probably come before I do a full rundown of strengths and weaknesses. I'm going to give him breathing room to do that first.
#45
12/06/2012 (2:45 pm)
@Michael - Thanks for the awesome answers.
#46
12/06/2012 (2:50 pm)
@Michael - Thanks for the awesome answers.
#47
12/06/2012 (2:54 pm)
@Johnny - Always a pleasure to talk with you. I look forward to being able to post a full picture of the 2D tech later.
#48
12/06/2012 (6:32 pm)
Sounds great. Can't wait to see what you guys come up with!
#49
(click to see larger version)
12/07/2012 (8:14 am)
Of course, it's not always easy. We managed to have OpenGL rendering going in a couple of days, but it took almost four to get fonts working. Sometimes we are fighting Torque, other times just the Cocoa API. Then there are tasks that have you fighting both at the same time and you want to drive your head through the desk. Still, when you get results like this, you dance a jig of joy:(click to see larger version)
#50
What are you doing about fonts?
12/07/2012 (6:30 pm)
Fighting Cocoa is just a slap fight. Fighting old Torque code is more like war.What are you doing about fonts?
#51
12/08/2012 (4:25 am)
We are likely going to take a couple of passes at the font system. The first pass is utilizes the current Torque way of doing things, but we removed all the Carbon code. Future passes will either improve Torque's system or rebuild it, but that after the platform work.
#52
While you guys are redoing the whole system, any chance the new platform will be able to play MP3s directly in the simulator? Currently I must switch to XCode's emulator to hear the music.
12/11/2012 (7:24 pm)
The progress sounds great. Keep it up!While you guys are redoing the whole system, any chance the new platform will be able to play MP3s directly in the simulator? Currently I must switch to XCode's emulator to hear the music.
#53
12/12/2012 (12:44 am)
MP3s would need some licensing. Maybe if they implemented use of the OS frameworks for media playback on the desktop. If they did, developers could be tempted to release with MP3s, though, and that still requires that they have licensing in order. Whatever that means. The Xcode simulator (not emulator) uses the desktop version of the iOS media frameworks, which are pretty close in how they operate. You could use that as a starting point to implement it if you want to try it yourself :)
#54
12/12/2012 (4:29 am)
@Watermark - Basically what Ronny said. No plans to implement MP3 into our desktop runtimes. If anything, we would replace OGG with another streaming format that could work on all platforms, but that's not in the immediate future.
#55
Now that Eric has released the big news about the next Torque 2D release, I will be working on a few more in depth technical blogs. They will be on the front page, unlike these blogs, with no R&D disclaimer.
12/12/2012 (2:15 pm)
Torque 2D MIT AnnouncementNow that Eric has released the big news about the next Torque 2D release, I will be working on a few more in depth technical blogs. They will be on the front page, unlike these blogs, with no R&D disclaimer.

Torque 3D Owner Ronny Bangsund
Torque Cheerleaders
You can take half the day off, but after the cake is gone it's right back to finishing this thing, mkay?