Loading Time. Cmon Now!

#21 Geoff, I went through the engine and removed the TGB folder, anything related to dts/3d support, any gui controls I didn't need.

I've also gone through and weeded out any gui profiles I wasn't using, and reduced my font usage as much as possible to keep the number of unique fonts down to a minimum.

Unfortunately Apple has rejected Ninja Rally because it takes too long to load and thus exits back to the desktop.

Loading time on the iPod Touch is still around 12 seconds. It's over 20 on the iPhone. The day it was running at around 16 seconds was a fluke, I can't get it back down to there. I'm even rebooting the devices, closing down pages in Safari, etc.

This is a MAJOR issue now.

I am waiting until after the title screen to load all datablocks, common scripts, game scripts, and sounds. I don't initialize Guis until they are used, am not initializing network scripts until the multiplayer menu is activated. In other words, I'm close to bare minimum. I took 120 files out of the engine. I removed that code that Mat suggested also.

My only 2 known steps left are to try to inject some code into the engine loading process to get some activity on the startup screen. Springboard kicking out the app because it's taking too long to load is the same as a crash to the users.

On the plus side, the game is really fun, the multiplayer experience is awesome, and it feels like a smash hit.

#22 Should you need an 1st generation ipod touch to test, drop me a line. I got one myself for testing (actually it was my first iDevice as the regular iphone was not launched here in Switzerland). It uses the OS 3.0 beta

#23 Thanks Marc, someone donated one at work to test and it runs as well as the 2nd gen one. It's the dang iPhones that are taking forever.

#24 I've integrated Melv May's memory reduction code he came up with for TGB. I'll be testing it today on a 2nd gen iTouch to see how it affects loading time.

Tomorrow I will be able to get my hands on a 1st gen iTouch to test it on slower hardware. There is also a cached datablock resource I'm going to dig up to see if it will work with iTGB.

Between the memory reduction, cached datablocks, and the disabling of unnecessary systems, I think we can significantly lower the loading time. If it all works out it, this will be in the 1.2 release. If there are still bugs, we will release the memory reduction as a hotfix.

#25 The main problem with the iphone is the far lower ram thats available. At least thats my conclussion from various checks on both, be it after startup or alter.

I compared the 1st gen itouch to iphone 3G

iphone 3G commonly has around 20 - 25 MB of RAM free according freememory, even right after start.
itouch 1st gen is commonly around 40mb

so the iphone likely is "paging" a lot during loading too.

#26 @Marc

The iPhone has a few extra programs resident in memory compared to the iTouch. Things like the phones call handling, ring tones, phonebook etc. etc.

Some are loaded as needed but the call handling stuff is always resident. It's designed to interupt any other program if a call is detected.

As for memory paging. As far as I could tell this isn't happening at all and any apps that attempt to use more memory than available will simply be terminated.

#27 Yes but call handling does definitely not require 20MB of RAM or said differently: the whole OS reserves nearly 110MB of RAM of the 128MB the iphone has, which definitely is broken and in no way acceptable for a phone OS. call handling and connection protocol handling requires 10-20mb at worst, so where are the other 90mb?

Its like the unrealistic memory usage of OSX (which has been as worse as vista and more worse since the early XP days) ie totally overdriven. The way the memory is handled in OSX just makes it less obvious and for common users less troublesome than the way windows works with memory.
The problem on the idevice is just that there is not enough RAM to avoid memory dellocation enforcements to get even basic crap running.
I hoped that this is adressed with 2.1, then with 2.2
Can't say if it is better on 3.0 as I won't sacrifice my work phone for the beta just to know if the os is as broken again.

#28 Marc:

I don't know what basis your comparing OSX to vista, the memory management and usage under OSX is WAY better than Vista and XP. I've been running OSX for the past 8 months on a machine with less powerful hardware than I use to run XP and Vista on and it performs twice as good and uses memory much more effectively.

The iphone OS is very well written and from what I remember from reading the specs only 1/2 of the memory is available for application usage, the rest is reserved for the OS regardless of whether it's used fully or not. Writing an OS that supports everything that the iPhone does with a kernel, file cache etc utilizing less than 64mb is a real challenge.

#29 OSX manages memory significantly different than Windows.
Windows keeps its system related memory in RAM including all services, even when other apps are running.
OSX switches out the whole memory to give the application the full RAM to work with.
That is what makes the difference there and seem OSX "less stressing".

As for iPhone OS: Nobody said that it is a trivial thing to write such an OS, I'm very well aware of that.
But the fact is that Apple is claiming that they succeeded in it, which as a matter of facts isn't the case.
If only 30 - 35mb the RAM are free while nothing is going on beside the Free Memory app, where actually 50-60mb should, which is even the case right after a full restart, a point in time where NOTHING other than the call and message handling services should be running, then there are problems.
One of the things I have found out is that Mail is violating Apples own official stance of "only a single app running at a time", as it is permanentely active. Sadly its not possible to say how much RAM it uses as Apple does not feel like offering those process data.
At 30mb of RAM left, the memory problem triggers in much earlier than it should. I think it even might be fluctuating, as even games like Field Runners crash.
I have no single game on my 3G that does not crash once per 20 - 30mins, even right after install and a fresh restart.

#30 I'm not sure where you got the information that OSX switches out the whole memory to give the application the full RAM to work with but that's not correct. OSX uses demand paging with second chance failure, additionally it uses statistical analysis to determine applications working set so that only the minimal number of memory pages are allocated to the application during startup time. Also the integration of the file system with the VM enables extremely fast file caching and access, which is the primary reason Mac's and Linux perform so much better than Windows which has a terrible implementation of VM and file system integration, which is why Windows hard drives always seem to be thrashing even when you have 1.5 gig of 2 gigabytes free.

If your really interested in how Mac OSX handles memory read MAC OS X Internals by SINGH, it has a lot of material on the Mac VM and is what I used for the basis for a lot of my research paper on Mac OS X.

As far as saying apple's iphone OS is a failure because it doesn't have as much application memory free as you'd like for your 3rd party application is rather ridiculous. Apples official stance of only a single app is in relation to 3rd party developers, this gives them relative control over the user experience to prevent poorly written apps from affecting the users entire experience when using their phone.

Saying you know that the iPhone should only have those basic services active is a quite arrogant from my viewpoint. The iPhone is not just a platform for you to write applications and as such Apple has prioritized certain applications above others. There is also a whole slew of brilliant people who work on this and they have quite a few good reasons I'm sure for having that amount of memory left.

I'm not sure why your experience with the 3G is so poor, but mine isn't even close to it. I've run fieldrunners and many other applications for several hours without crashes. In fact I rarely have an application crash on my phone and a simple reboot fixes the crashes for days, if not weeks on end.

#31 From XCode documentation (at least I think, potentially a different place specific on 32bit memory restrictions) with explanation on the maximal usable RAM on 32bit systems on mac, which layed out that an app is able to adress the full amount of installed RAM, not only the amount of RAM not used by the memory.
In addition it was put in comparision to how windows works where the app is only able to the adress space of 32bit minus what the OS and DMA require.
Its thus possible that this is not fully correct, but at least the memory mapping heavily implies that the stuff is moved out as I don't see how memory could be adressed thats reserved for the OS without crashing or paging. Neither really usefull and desireable.

I don't say that the iPhone OS is a failure, but I say that Apple behaves like Sony and claims things that don't hold true.
I also don't see how it is arrogant that I as a user don't want mail to run permanentely if it is used at max once per day if at all. My MBP with Mail runs fulltime next to me for communication, I only need the iphone if I am out for at least half a day which happens very rarely.
To me, permanentely running Mail means more cpu time and RAM powering required, thus lower battery time.
After all I payed premium price for a phone and expect that I have the option to decide what runs and what not from the optional stuff and Mail is optional, its no phone service related thing (unlike MMS which is a core aspect here in europe yet it still took an eternity by phone lifetime spans to integrate it).
The mail to me is no different than bluetooth. Both just cost energy and potentially performance without any gain for me outside of very isolated small timeframes, so I want the option to decide if it does so or not.

There is no single game I have on the iphone that does not crash, so it has nothing to do with beeing a brilliant person to develop for it but with problems in the OS itself, problems that are pretty deep rooted if nobody has been able to work against them to prevent them from happening on a near 100% scale.

The only hope I have is that 3.0 finally has a sandbox, not only a security box like what is present in 2.x, so those "application has run" memory leaks are gone.
Be it leaks from running an app or installing an app.
In either case, the allocated memory is meant to be gone the moment the application is finished. This is, at least out of my sight, the major problem with 2.x and one that apple must adress as they started to officially advertise the iDevices as gaming platform. Gaming platforms don't crash all the time because the available RAM varies by 40MB depending on how many apps you started before.


How you come to the conclusion that it is no platform for me is over my head.
I've worked under tighter resource requirements in the past.
The main difference is that all those platforms were able to follow their specification and have those resources available that were meant to be available, which at least with the iphone os up to 2.2.1 isn't the case on the 3G.
That is my problem, not the rest.

#32 The amount of addressable memory is not related to how memory is allocated to processes. Process use Page tables (hardware backed) to translate their virtual pointers into physical memory addresses.

On windows the lower portion of your virtual pointer is reserved for operating system code, and the upper values are your addressable memory.

On Mac OS it works a bit differently.

However the point is actual physical memory is allocated to the process in 4k (usually 4k) chunks known as pages. This is pretty much hardware defined because it requires hardware support to enable this functionality at a reasonable speed (known as a TLB or Translation Lookup Block).

This is why you have 4-5 applications running and using 5 gigs of ram total but only have 2 gigs on your system. If the OS manages the 4k pages effectively then these 5 applications will run very fast without much paging. If their managed poorly then they run very slow because they have a lot of paging. The effectiveness of paging is primarily determined by how well the OS manages the applications working set, or the memory that it accesses 90% of the time, which is usually much smaller than the actual ram it's using.

The reason the iPhone restricts your memory usage is likely because it doesn't support Paging because Paging can be expensive depending on the hardware backing it up, software paging performs terribly.

Also for things like Push email, auto-polling, etc you need to have the email application running, at least part of it. The same is true for text messaging etc. T

Also since your probably a windows guy, like I was for the past 30 years until I purchased a mac mini for iphone development, you need to understand that Apple does things differently. They sell you a User experience, not just hardware with the ability to run software. You pay for the experience and as such you give up some control over tweaking every little detail to your specific liking.

The memory leaks from applications that have run previously is likely an OS limitation or bug because typically all memory allocated to an application is cleaned up by the Operating system on shutdown. This is likely a resource leak of some sort due to some constraints the kernel developers had to work within when porting the Mach kernel to the iphone. Depending on hardware support cleaning up everything a process uses when it shuts down can be difficult.

#33 I had my share of lecturers on the topic of how OS (especially *nix) works, so the theory isn't the problem, independent of the OS.

My problem is: What happens if an application really uses all 4GB it is able to adress on a 4GB system and uses them?
How does OSX handle that siutation? Paging is no option in this case, take a 3d renderer for example, as the performance degrades by magnitudes.
So why would anyone make all ram adressable to applications even if the OS needs a fixed amount of RAM permanentely?

As for the push mail: OS 3.0 shows how to handle that correctly. An on screen message where I can choose to open the app or not. This API out of my view has to be enforced for all for a consistent feeling and battery & resource usage. I don't see why the Mail should be an exception to that.
Text messaging etc for me are part of the common communication stuff (together with MMS), so fully agree that this stuff has to be perma present. But that stuff requires a handfull MB of RAM, not 20-30MB, thats what is fishy to me. Sadly its not possible to find out what is wasting the RAM as Apple does not offer any access to the relevant process informations.


Its out of question that the memory leak problem is an OS bug or better a bug in the sandbox handling (because if the sandbox worked right, leaks would be impossible. Anything allocated in the app is at last deallocated when the app ends and the sandbox is cleaned from memory. Anything else ie the way it works now, is a serious security problem).

This bug is a major bug which Apple neither has adressed with any 2.x release nor announced any official timeline for fixing at least as far as I've seen.
This is what I tried to get at right from the start.
Because I assume that apples own application suffer from the same leak. Especially mail has desastrous consequences if you have 8 mail boxes in like me. basically I open mail twice either restart then or can skip the idea of playing most games

#34 An update.

Keeping in mind that I have already taken the mindset of deferring loading. Deferring loading of guis, I dramatically pruned the guiProfiles I was using, yanked over 100 files out of the engine, compressed textures, and got maybe a 1-2 second gain out of that. The network loading and initialization is a good chunk of that.

The main problem to be solved (aside from all this interesting OS discussion) is that "the app takes too long to get to the first point of interaction" ie a title screen. This is because the startup sequence is identical to stock TGB with a couple *additions*. Certainly not optimized at all for an iPhone or iPod, and a poor user experience.

So this weekend I went through and re-architected the entire startup sequence by rearranging all kinds of stuff scientifically.

The major points:
I focused on what was needed to intialize the canvas and display a GUI and sound.
This meant isolating the 2 profiles I use in the gui, execing the gui, and showing it. The very next thing I do is initialize openal, exec the audiodescription and profile I need to play the title music and play the title music.

I defer everything else, all other guis and guiprofiles, network loading, script execing, etc until after that point.

This gives me a barebones first gui time of 9 seconds on the iPhone and 8.5 seconds on the iPod. It still takes another 10 seconds to finish loading, but at least the user experience is dramatically improved, much more inline with other games I've played. My next step is to pvr the startup screen's texture so that it decompresses faster. This will be only a slight gain.

So, use traditional optimization methods to squeeze out performance. Don't accept any of the default loading sequences, they don't belong on the iPhone. I hope this 5 day adventure of time I will never get back helps someone else out in the future.

#35 pvr are not decompressed unlike jpg / png
they move 1:1 into memory like that and can be used by the hardware in their compressed state (like DXT)
thats their real benefit because they remain that small in memory too, not only in within the app bundle

#36 Marc:

Paging is always used for pretty much all modern operating systems.
Windows and MAC memory implementation is mostly the same from that standpoint. They both use paging and virtual memory pointers with hardware support for the mapping of virtual pointers to physical memory locations.

The difference is that the OS components are always mapped to a fixed address range for windows while it's dynamically mapped as needed for MAC OS. I could be wrong here, not 100% on this piece though it's easy to look up.

The speed advantage on MAC Os comes from integration of the File system access into the Virtual Memory manager. This lets file system access be cached using virtual memory pointers that have hardware backing. That's part of the reason why linux/mac file systems are so much faster. Windows run faster in VMware on my mac mini (anything remotely FS based that is) than it does natively due to MAC's much faster IO.
The other reason is that MAC does a much better job of keeping the applications working set in memory than windows does, so it pages ALOT less than windows does.

The simple answer is the advantage is that if you put 6 gigs of ram in your system on windows it'll only make 3 gigs available to applications, and the other 1gig is reserved for OS (2 gigs wasted). On macintosh (32bit if you can even still find them, most are 64bit now) the a full 4 gigs of ram would be available to the application and the only portion of the operating system code reserved in that range would be for access to operating system code (2 gigs still wasted).



I think we've gotten off topic enough so I'm not going to hijack this thread anymore.

#37 @Marc and Bret

Getting back to iPhone and ignoring it's big brother OSX. I think you will find that the iPhone does not accept dynamic linking and everything is linked statically. Being a portable device. It does not allow for paging of things in and out of memory. Either your app manages it's memory correctly or it tries to overrun it's memory and is terminated end of story. The iPhone OS does not jump in at that point and start paging bits of data in and out.

This is why, if you check for memory before attempting to allocate it and you get a no memory available error. Your app simply throws an out of memory to the customer and terminates itself (or at least it should).

You seem to be comparing the iPhone and iTouch OS with full on OS X. Full on OS X usually has a few gigabytes of ram available and can use the hard drive as virtual memory. An iPhone ort iTouch has ONLY a single block of 8, 16 or in the case of the touch 32gb of storage. Each app is allowed a maximum block of that memory and no more. You don't get to use extra if your app somehow grows to fill available memory. The iPhone OS simply registers that you've overstepped your bounds and kills you in the nastiest most destructive way possible.

#38 getting back to the thread topic, I did a few tests with our game and came up with an interesting result. We have custom datablock loading, so datablocks are not loaded until they are needed by a level.

on the iPhone, we are getting the 8badf00d exception with application timeout because the app is taking over 25 seconds to load (takes 15 on the ipod touch)

if i remove all images (we probably have over 200 images) that are loaded by datablocks and in theory, should not be loaded until a level is loaded - our loading time on the iphone drops from 25+ seconds to 12 seconds. The thing is, these images aren't loaded until the level is loaded....which is after our main GUI screen. I would think removing these images would have no impact on loading time since the images aren't loaded until required by the level...any ideas why this is?

#39 Are you sure they aren't loaded?
Have you enabled the preload flag?

#40

Quote:
I would think removing these images would have no impact on loading time since the images aren't loaded until required by the level...any ideas why this is?

I'm curious about this too. If an iPhone app is 5MB, and the art is part of its "resouces," then does it load that full 5MB up at startup?