Atlas2 Problems

#21 Vincent is spot on. This is a preview release, thus under heavy dev, thus my nose is still in the code. :)

I'm in process of solving the various texture mismatch issues, color scrambling, and the texture performance issues but it's not quite there yet. This is a classic example of reality not necessarily surviving contact with the enemy. :)

Incidentally - are you all updated to the latest D3DX libraries? You may find that this will resolve some of the DDS issues you are experiencing.

#22

Quote:are you all updated to the latest D3DX libraries?

Latest DirectX SDK (June 2006)
Latest HEAD Downloaded
Rebuilded TSE
Re-executing Code for generation of Atlas Files

And the result is... The same.

Edit : So It might be somewhere else.
Edit 2 : Do you want my raw, jpeg and atlas files ?

#23 @Vincent - I didn't ask that because I expected it would fix your issues, but rather because I was wondering if it was related to some other people getting DDS-related asserts in MS3.5. Remember what I said about scattered discussion tending to confuse me? ;)

You're welcome to send them to me (or at least a link to where I can download them - my inbox doesn't like super big attachments!).

Insofar as something that might address the color-swapping issue - try changing line 279 of atlasImportLargeImage.cpp to read:

atc->mFormat = AtlasTexChunk::FormatPNG;

That's the only outstanding diff in my codebase for the texture path (ie, in atlasImportLargeImage, and AtlasTextureChunk, and texture TOC classes). My GFX directory code is the same. If this fixes it indicates something's off in how JPEGs are dealt with.

The other possibility is that, as I said earlier, something is calling GBitmap::convertToBGRx() too much or too little on your systems for some reason that won't reproduce on my end. It might make sense to try putting a breakpoint on once you're in-game and loading texture data via Atlas - it should be flipping once right before the data is uploaded, to allow for DX9's odd texture ordering. It's an in-place flip, so if you do it once too much or too little you'll get what you're seeing, e.g. messed up colors.

Running on 64 bit systems might also relate to this (although it's unlikely, relatively speaking).

All you guys have to do to solve this is run the import tool, and see what data gets passed to the image codec - say, for the first pixel. Note the first pixel read back out at load (for that chunk). Then follow that GBitmap till it gets loaded into a texture (happens in AtlasTexChunk::process), and note what the first pixel is there (and particularly if convertBGRx is called!)

Now compare these values. What you should see is that it's the same (ie, correct RGBx order) all the way through till convertBGRx is called, right before it's uploaded, at which point the values will be flipped in order. (ie, BGR instead of RGB.)

This is what is happening on my end - and presumably why I'm able to import JPEGs just fine.

Just updated CVS and checked... and it works great for me (with the abovementioned change, testing using the GuiAtlasTextureViewer). Source is a 4096px JPEG, ran commands:

atlasGenerateTextureTOCFromLargeJPEG("terrain_water_demo/terrTexture_shadowed.jpg", 3, "terrain_water_demo/texture.atlas");
quit();

So if we're going to solve this I definitely need info from your end! :) Otherwise we can just trade posts about how it's working/not working for a really long time, which might not be as much fun. ;)

#24

Quote:Just updated CVS and checked

I Don't see any changes :o

You can download my test files here : www.dragonheadstudio.com/gg/terrains.rar (68 799 235 bytes).

I Also Updated my previous pics to show How it mirror (refresh this thread).

For Line 279, I was Allready good. But At line 277!? (Perhaps a CVS issue?)

I tried also atlasGenerateTextureTOCFromLargeJPEG(jpgFile , 3,...); It gives the same result.

I'm going to make checks in AtlasTexChunk::process to verify if ConvertBGRx is called. Edit : ConvertBGRx??? Where?? I haven't this word in TSE!

For Today, It's 14 July, and in france we don't work (It's like 4 July in U.S) And my wife want to take me away from the computer... So .. :) It may take times before my next post.

Final note : could someone tell me if bandwidth when downloading this file is good for U.S?

#25 @Vincent:

Good point about the "Nose in code" (never heard that one before :D).
So I take it these issues only happen with Atlas2, and not the previous incarnation - correct?

#26

Quote:So I take it these issues only happen with Atlas2, and not the previous incarnation - correct?

Yes. But "Nose in code" isn't specific to Atlas ;)

#27 I'm trying to seek... But I don't see any conversion to BGR in Atlas2. So Is there a problem with CVS? Or did I misss something else?

#28 These particular issues are relating to Atlas2 code. I'm also having issues with the old Atlas1 code since the new code release.

#29 @Vincent: By updated, I mean "did a cvs update" - ie, checked to make sure I have latest code from CVS.

Ah - that'll teach me to post at 3am! It's convertToBGRx, not convertBGRx. My search results:

Searching for: convertToBGRx
engine\gfx\gBitmap.cpp(103): void GBitmap::convertToBGRx()
engine\gfx\gNewFont.cpp(1186): mTextureSheets[i].getBitmap()->convertToBGRx(); // Diddle it back to RGB. This is awful.
engine\gfx\D3D\gfxD3DTextureManager.cpp(235): pDL->convertToBGRx(); // This does checking for format inside it, fear not
Found 3 occurrence(s) in 3 file(s)

Anyway - I just did another clean checkout, clean build on the virgin codebase (double checked after running through the test), downloaded your files, imported your JPEG, and got this:



So whatever is breaking for you - I can't reproduce with HEAD, so we're going to have to debug on your end to fix it! :)

Your texture.atlas did load with swapped colors for me.

@Mark - did you check your D3DX version like I mentioned? I just pointed an AtlasInstance (Atlas1, not Atlas2) at the smallTexture dataset we ship with, and once I had the old atlas shader in there everything loaded and rendered fine! I did experience a crash when importing that same TQT to a .atlas file, which bug I have added to my list.

#30

Quote:... once I had the old atlas shader in there...

Ok let's assume I'm dumb for a moment (not much of a stretch) and explain to me exactly what you mean by this statement. Do I need to change the file path of my AtlasMaterial definition in Materials.cs?

#31 It means you need to change the path to point to shaders/atlasSurface(V or P.hlsl. If you have a 3.0 codebase copying the atlasSurface shaders from there should work, too.

#32

Quote:Your texture.atlas did load with swapped colors for me.

I noticed when I import Large JPEG that ConvertToBGRx is called, and colors are swapped. So I think The texture.atlas is setted correctly.

So now is there a render problem ?

#33 :o Ben I tried importOldTQT and it said me something about a Ben (I think it's another one) who is lazy... LOL

#34 Quick and VERY DIRTY patch for colors :
in gfx/D3D/gfxD3DTextureManager.h
Add before inline void D3DAssert (line 37), add :

bool vincent_ok = true;

@ Line 237, make it look like :

if (vincent_ok) {pDL->convertToBGRx();} // This does checking for format inside it, fear not

in atlasImportLargeImage.cpp @ line 286, make the code look like :

// Get the stub. (Note we swap i and j)
         vincent_ok = false;
         AtlasResourceTexStub *tStub = arttoc->getStub(treeDepth-1, Point2I(j,i));

         // Store, then purge, the chunk.
         arttoc->instateNewChunk(tStub, atc);
         tStub->purge();
         vincent_ok = true;

don't foget near line 12 to add :

extern bool vincent_ok;

This dirty patch don't solve mirror issue. I'm not sure it's the right place to do this, but colors are swapped twice : 1 time in atlasGenerateTextureTOCFromLargeJPEG (colors become BGR) and 1 time in atlasGenerateUniqueTerrain (colors become RGB), and think directx wait BGR values ;)

Edit : Can anyOne test this patch and say me if it's OK (At least for colors) ?

@Ben :

Quote:(Note we swap i and j)

I don't understand why?

#35 Where and when is convertToBGRx called in the large texture importer for you? The data that is saved should be RGB...

#36 It's called from this line (approx atlasImportLargeImage.cpp @ line 286) :
arttoc->instateNewChunk(tStub, atc);

The same case is in atlasGenerateUniqueTerrain when copying chunks.
So... Each time you fill texture chunks with bitmaps.

#37 Hmm.... What happens if you put a return; at the start of AtlasTexChunk::process()? (Which is where I assume convertToBGRX is being called, due to the texture load.)

#38 Ah hah.

Do the import call from main.cs BEFORE anything else. See if that works properly.

I think I know what the problem is... finally. ;)

#39 YAY! That did it Ben. Let me guess: you didn't expect us to be running the functions from the console after the engine has initialized? It looks like all of us were doing it that way. I imagine Vincent was using his HTC tool, personally I'm using Neo Bindell's GUI.

However, I would like to re-iterate Vicent's question: why are you flipping the i & j indeces? It looks like the texture is rotated 90 degrees. I know L3DT was doing that before but I don't think its a problem now. Where can we turn off that "automatic rotation"?

#40 Mark - yup, you got it. And of course, the fact that there's an implicit flip in texture loads with no reversal later... no good! See, I told you it worked fine for me... :P

The simple hack is to just add a global boolean that you set when running generators, and turn off for normal paging. I've a more comprehensive fix in mind - distinguishing when one is interested in the raw bitmaps or in textures and doing things slightly differently depending...

As for the coordinate stuff - if flipping it back fixes it, please let me know! I'm in the middle of reconciling my UVs vs. actual texture data, so that everything matches up perfectly on initial import without flips(at least for L3DT's notion of correct - not all tools agree), and also making sure my data lines up nicely... so an upcoming drop should have a more sane arrangement here. It's still pretty lame that DX9 and GL have different UV mappings, though. :-/