Performance and Crashes with a lot of atlas files

[Emperor of The Great Unknown]

A common problem all large mods suffer from is constant crashing in the middle of battles. I am suspecting that is tied to a lot/large atlas files defined by the mod. I also suspect NTW might be hitting a hard limit on either memory or space for graphics. So I am going ask you guys is there a way to lessen this effect.

First off which is more important the number of atlases used in a single battle or the total number of atlases in the mod?
My idea of maybe fixing this is reducing the size of the original vanilla atlases, especially fairly useless ones like austrian_rebels. However if these aren't loaded until they are needed then it wouldn't do any good.

[Steph]

Try avoiding battles with too many atlases.
If you have for example a mod that adds new units for Bavaria, Wurttemberg, Saxony...
Try to have an army with only Bavarians, another with only Saxons, etc. instead of an army with 1/3 Bavarians, 1/3 Saxons...

Beside this, the only solution is not to make new units / uniforms...

CA could improve things IMO if they use individual textures files for each uniform parts, loading only what they need, and loading a texture only once if it is share between several soldiers.
For example, the "belt" texture is almost always the same (in white/black, for two belts or one belt). 4 variations which are virtually duplicated in every atlas.

Unfortunately, it is not something we can mod.

[Emperor of The Great Unknown]

This is true, however vanilla can have over 8 different factions on the field without crash. While big mods will crash with just two factions (two atlases). I have replaced a lot of the unused textures with 5x5 blank texture files. Basically removing them from memory. It seems to have fixed crashes with my mod, but since these events are transient i can't know for sure.

[Steph]

1) Some big mods can have more texture than that. For example, in Masters of Europe France actually has 5 atlases. There could also be extra textures for artillery or horses (faction specific saddlecloth)
2) Some factions in Vanilla have 'half atlas', that could be replaced in some mods with full atlas for the same faction
3) Some mod use textures without midmaps. CA texture usually have it

All this could have an impact.

[Emperor of The Great Unknown]

hmm I wonder if 3. is important. Mipmaps take up more space and therefore maybe more memory. (although it may increase performance depending on how Napoleon uses textures with mipmaps.)

Although perhaps Napoleon has a way of only loading in the mipmaps into memory at certain LODs, and maybe with too many textures being loaded full it causes crashes.

[Slyk]

hmm I wonder if 3. is important. Mipmaps take up more space and therefore maybe more memory. (although it may increase performance depending on how Napoleon uses textures with mipmaps.)

Although perhaps Napoleon has a way of only loading in the mipmaps into memory at certain LODs, and maybe with too many textures being loaded full it causes crashes.

Hey guys! After a few years working on the ACW mod, I have a few ideas. First off, Emperor, I do think you have ONE of the basic issues, which is then spun into far more by connections across the files. What I believe we need to manage and how...

- Atlas files.. Yes, too many are bad, as much as too many diffuse .dds files. I think that maximum effort should be made to use every pixel possible when making new uniforms/gear/weapon textures. When Primergy turned over ACW to me a few years ago, the first thing I did was set out realigning the uniform textures and re-configuring the Atlas file. I ended up with two diffuse .dds but ONE Atlas. I think texture load is a big deal with CA games, at least the early ones. I haven't modded beyond NTW.

- Variant Models: I think there is a potential to have too many variant models, even within ACW, I have added quite a few new ones for special 'Brigade' or "Regiment' types. Adding new leaders as well, if you want to have them available in the MP army/faction list and have a portrait/icon show up. I think that adding more variants, very evident in European mods for NTW's original time frame, is dangerous. You have to do some deep resource tracking to watch memory load, I suppose.

- Sound: I think ACW suffered from sound loop glitches early on and to a large extent, I have corrected some of that. When paired with other resource issues as I mentioned already, I think that sounds must have a lower priority for resources and in specific circumstances, can lock up and crash a battle. Again... overall loading of textures, variants and then special 'hi-fi' sounds can be deadly.

- Maps: So far, if you have played my ACW v1.0 release from October 2014, you will see that overloaded maps are NOT the issue. Every building, rock, stump and fence model is a building... 30,000+ on some of the Gettysburg maps. I have some very heave wooded maps coming up in version 2.0, "Wilderness" and "Chancellorsville", specifically, and they are hell on load time, but run rather smooth, so I don't see trees hurting over all resources as much as textures and their related files.

I have been convinced since 2011 that this game has memory leaks. We ran some tests back then that more or less proved it did suffer from memory leaks and most certainly from poor memory management. All I can say for sure is that by combining diffuse and texture files into more compact .dds/fewer .dds files and using fewer Atlas files seems to have helped a lot. My mod data base files are huge. 10,000 unique units and growing ;p to support the historical scenarios. You know how those tables bloat, so that's not an issue, in my opinion. I'm not going to say that I don't wonder about the localization file though..... I do, sometimes. That might be a memory issue somehow.

I am happy to keep a dialog about this. I do believe that local machine RAM helps, more is better... possibly allowing the game to chew up more room with 'crap' ??? Speculation there.

Anyway, I'll add what I can or run tests if you guys have some ideas on how to figure out better memory management beyond what I've said so far.

Thanks,
Slyk

[Steph]

I think if the engine would load individual textures instead of atlases, with a smart management to load a texture only once if it uses by several units, would save a load of memory. How many atlases have duplicates of the heads, belts, plumes or other basic shared textures?
And make modding more flexible.

However, I don't think it is possible. The packs do contian all the individual textures in separate folder (and atlases, again duplicating stuff...), but I couldn't find a way to configure the variants to use the textures directly, without using the atlas. It works for the mesh, but not for the texture
I believe maybe A has an internal tool that may create the dds automatically from the atlas file, by copying the individual textures in it, maybe that's why they have both

[Slyk]

I agree, Steph. Other games I worked with, COD1/UO and Red Orchestra used model based skins that I suppose you could argue were good or bad, but you only loaded the textures you used per map. I think NTW loads everything in each pack file. I also believe that having multiple texture and atlas files is bad as it is almost an exponential increase in memory use which magnifies the poor memory management. I love diversity in the game and mods that the file system offers, but there are reasons they didn't keep adding more and more content... opinion.

I know that I have added hundreds of new icons, increasing the load there. It is an interesting debate and I'm sure CA will never tell. But, back to my maps/scenarios...with so many models placed and up to 200 total units on the field at once, numbering in some cases 20,000 troops... the game performs. BUT, I would bet that if I had 10 atlas files and matching texture files, the game would choke and crash constantly.

[Emperor of The Great Unknown]

Interesting, this may seem drastic, but maybe useful to mods like ACW. But would it may be worth deleting all the unused vanilla atlases, variant_models, and icons? If variant models are the problem it may save a lot of memory there. I don't think icons are a problem, since surely the game is smart enough to only load the 20 - 80 cards you need, and the size of icons are puny compared to texture files. Deleting vanilla files (or backing up and replacing them) would be tricky with installation.

On note about altas numbers, I wonder if we can mess around with stability using vanilla resources.

Test 1 would be to test if the total number of atlases in your packs, affects stability. Easy way to do this with vanilla resources is to duplicate all vanilla atlases and textures and rename them, in effect doubling the size of the atlases. Run a normal 1v1 and increase the total number of atlases until it crashes or becomes unstable.

Test 2 would be to test if only the atlases loaded to battle matter, and if the number of atlases in a battle is important. Would be to create 8 factions with units that use 20 unique vanilla atlases. Start small and gradually increase the number of units with different atlases and find out when NTW breaks.

If both tests run fine then, crashes from texture/atlas files are an artifact of our editing and saving procedures. Similar tests could be run with variant_models I suppose.

[Steph]

I hope at least CA did not load ALL the atlases from all the packs at once, but only the atlases actually needed for a battle. So increase the number of atlas should not an effect, unless they are used.
So I think it is better to have one atlas per faction, with infantry, cavalry, artillery in it, instead of doing for example one atlas for hussars, one for dragoons, one for grenadiers, etc, with all the factions in them.
In the first case, we can hope that in a 1 vs 1 battle, the engine will load two atlas, one for each faction. While in second situation, it may load all the atlases to get the full coverage of units.

I think an easy test may be to start the game without any mod in the user script, and check memory usage.
Then load the game again, with an atlas heavy mod such as Masters of Europe, and check again memory usage.
Repeat for a land battle with only one unit on each side (so only one atlas needed).
Repeat for a land battle involving several atlas.

This way we can try to figure when the atlases are actually loaded.

I'll try to test that this evening when I'm back home, I'll let you know the results.

[Emperor of The Great Unknown]

Any updates on your results Steph?

[Steph]

Too busy at work. Yesterday I left home at 5 am, and came back at 7:30 pm, just had time to have dinner and go to movie theater to see a movie my daughter really wanted to see. Yesterday was the worst, but the whole week is more or less like that.

I hope I'll have time during the week end.

[Steph]

Test:

- start the game without any mod in the user script: NTW uses 420 Mb of memory
- start the game again, with the full MoE mod added to the user script: NTW uses 610 Mb of memory.

--> the engine probably loads quite a few tables at start, but probably not the textue

- start a land battle with MoE loaded, but with only one general of the same faction on each side: NTW uses 985 Mb. It increases a little to 1035 Mb when I zoom very close
- exit the battle, go back to the main screen, NTW uses 680 Mb/ Textures are probably unloaded when exiting a battle.
- start again a battle, France vs Russia, with many different units (and so atlases) : NTW uses 1160 Mb, and it almost doesn't increase when zooming very close.


Conclusion:
- NTW doesn't load all the atlases when starting the game
- NTW loads only the atlases which are needed for the battle, and unload them when exiting the battle
- Memory management of the atlases doesn't seem to be too bad, since with only one atlas it was 985 Mb, and with several 1160 Mb