Rome 2 puzzle - potentially reasons for the performance issues

[High Fist]

Rep for ProtoBo!

[ProtoBo]

This is quite interesting because from the result of the benchmark it seems is not rendering properly, but also it seems he is rendering anything we need to guess as the FPS are not constant aswell. I really would like to know what is happening when the window is minimized. I will try out if this behaviour is reproducible with other DirectX applications.

My guess would be that it loads in all the wireframes and textures it needs (loading up the VRAM perhaps) even when you don't look it at, causing the FPS variation based on the varying number of things it needs to create in each shot. Only when you actually display it on screen i.e. process those textures and wireframes through your graphics card so as to populate the pixels with the correct colours, does the real work get done, which we can safely assume is the most system-intensive part.

[Morfans]

Horray!

Amazing thread, seriously !

But what's even more amazing is this alQamar is the same guy who started a thread ("silencio") on why the vast majority of members have quit TWC lately ...

[alQamar]

I do understand your hint Morfans. I am sorry if this thread is not meeting the quality we are expected to face on TWC but that said in the OP.

[alQamar]

My guess would be that it loads in all the wireframes and textures it needs (loading up the VRAM perhaps) even when you don't look it at, causing the FPS variation based on the varying number of things it needs to create in each shot. Only when you actually display it on screen i.e. process those textures and wireframes through your graphics card so as to populate the pixels with the correct colours, does the real work get done, which we can safely assume is the most system-intensive part.

Hey again,

I tested the behaviour on other DX applications and it is quite similar, so minimizing boosts the fps. Seems you ideas about it ProtoBo are quite right. Heaven 4.0 benchmark for example when minimized will stop the rendering , so will not run throught like Rome 2 does, still the behaviour of higher framerates is visible when you re-open the window again.

[Tr0tskY]

the cpu still has to handle all the non-rendering stuff such as the AI scripting, battle calculations. The game is still loaded into the memory and being processed, there are just no "graphics".

[TheLastRonin]

I get 1000 FPS...

All I have to do is turn my computer off...

How does knowing the Rome Total War 2 runs better when it's not processing graphics help anyone? Obvious observation is obvious?

[alQamar]

Challenge is over, sad some few people seemed not get the fun point in it.

[DogSoldierSPQR]

Thanks for this thread, Al. No doubt CA will come across this and they'd report it to their performance team who may come up with something. Least we can do is throw suggestions and tips CA's way and they can use things that are useful.

[alQamar]

thanks for appreciating it that way. i will do some more research about cpu and gpu load.
Rome2 commonly known lacks of a cpu bottleneck. but if I see this know I think we can start a new dig for reasons.

at least it tells me some. The engine CAN be perfomant.

[DogSoldierSPQR]

Yeah, I have faith that we can run this game with very minimal lag. For some reason I feel CA may be onto something already. We haven't really been getting performance patches so here's to thinking CA are up to something.

I like your threads, Al. You're putting in some work for this game and we appreciate it

[DarrenTotalWar]

Loads of games have this? If you run something in a window and then make the window tiny, you will have massive frame boosts.

Think of it as lowering your resolution as you drag the window smaller, because that is what you are doing.

Last year before I got my PC I played my laptop on a 4K monitor, I had set all my games to run at 720p because the PC couldnt handle it running on that scale. Another trick is to just window it and make it smaller. This even works with minecraft guys.

[alQamar]

no Darren it is not about the resolution (which is obvious) but to find out why the game can run faster when minimised. I am willing to test the loads as said.

minimized = alt tab out the game and have it minimized on the task bar while running. Perhaps it was not clear enough. Having the game running in windowed mode and resize it is what you (DarrenTW) was speaking about.

[RedFox]

From a game developers perspective the reason is quite simple, actually:

runSimulation(...); // process game simulation

if (windowFocused) { // only trigger entity rendering if window is focused
    scene_root.render(...); // render visible scene
}

So yeah, it's just not rendering anything. This is interesting though - I would expect a much higher speed-up when rendering is disabled, like 400-500fps. Perhaps this just shows us that R2TW simulation engine itself is the major bottleneck, not the rendering engine? Simple speculations. I doubt they'll ever be able to make it run as good as the final patched Shogun 2 engine did...

[alQamar]

From a game developers perspective the reason is quite simple, actually:

runSimulation(...); // process game simulation

if (windowFocused) { // only trigger entity rendering if window is focused
    scene_root.render(...); // render visible scene
}

So yeah, it's just not rendering anything. This is interesting though - I would expect a much higher speed-up when rendering is disabled, like 400-500fps. Perhaps this just shows us that R2TW simulation engine itself is the major bottleneck, not the rendering engine? Simple speculations. I doubt they'll ever be able to make it run as good as the final patched Shogun 2 engine did...

Sounds reasonable. I will start my tests soon and have a look what actually happens when the game is minimized and how the CPU / GPU load aswell as the VRAM load is. Some people said Rome 2 needs 2 GB VRAM. If so my computer would need to degrade textures, but from the log files he is not doing it.

Also I will try to find out how much the GPU and CPU load relates to another. I know some other guys on .com did that before but perhaps I can learn something new doing it my own.
Stay tuned.

[alQamar]

So here are some details when Rome 2 is running the benchmark on maxed out settings on my system, 1080p res.

Rome 2 1.81 final
Windows 8.1 final, EVGA 670GTX FTW (Stock), Sandy Bridge i7-2500K @4,6 GHZ, 2x4 GB DDR3 1600 MHz CL9-9-9-27, 1,35 Volt, Dual Channel mode.


CPU = processor
GPU = processor of the graphics card
RAM = system RAM (DDR3)
VRAM = dedicated video RAM on the graphics card (GDDR5)
Swapfile = virtual system RAM by utilizing the harddisk / SSD - filebased


unlimited VRAM off
--> RAM usage: 828 MB
--> Windows swap file usage: 2.760 MB (just by Rome 2)
--> VIDEO RAM USAGE: 1926 MB
--> max GPU load /average GPU load: 99% / 98,882 %
--> Max FPS / Average FPS: 46.8 fps / 32.045 fps

CPU Utilization
Core 1: 90.63 % / 49.03 %
Core 2: 58.48 % / 5.23 %
Core 3: 56.25 % / 28.38 %
Core 4: 32.33 % / 7.2 %
Core 5: 34.38 % / 8.8 %
Core 6: 40.02 % / 17.36 %
Core 7: 50.00 % / 7.85 %
Core 8: 46.87 % / 26.85 %

all core load max / avg 51,12 % / 18,84 %

unlimited VRAM on
--> RAM usage: 1182 MB
--> Windows swap file usage: 3088 MB (just by Rome 2)
--> VIDEO RAM USAGE: 1918 MB
--> max GPU load /average GPU load: 99% / 98,991 %
--> Max FPS / Average FPS: 46.7 fps / 31.132 fps

-->max CPU load / average CPU load (i7 HT enabled)

CPU Utilization
Core 1: 93.75 % / 47.35 %
Core 2: 28.12 % / 5.79 %
Core 3: 57.81 % / 28.33 %
Core 4: 26.56 % / 7.15 %
Core 5: 37.50 % / 7.8 %
Core 6: 35.94 % / 16.06 %
Core 7: 23.44 % / 6.6 %
Core 8: 45.31 % / 25.36 %

all core load max / avg 43,55 % / 18,08 %


The standard scenario gives me quite interesting but also controverse results:


Rome swaps a lot:
Rome 2 uses the Windows swapfile wastely, this could explain FPS drops people encounter when their harddisks are slow (busy) and they do not have SSD drives to run their Windows OS (system drive contains the default storage for swapfiles, this can be customized.)


Strange unlimited VRAM behaviour:
If you enable unlimited VRAM the amount of needed RAM and swapfile used increases, while the video RAM utilization slightly decreases.

Also when VRAM unlimited is checked the overall CPU load decreases significantly in peaks, but only a few average.
At least on my system there is no quality improvement, regardless unlimited VRAM is on or off.

VRAM usage:
As someone said before Rome 2 uses indeed nearly 2 GB of VRAM on maxed out settings, and entirely loads the GPU core capatibilities on my 670GTX and the cards VRAM.

I wonder how much the fact the game is running so much on swap and RAM instead of VRAM does harm the performance. From my amateurs skill in that case I can just guess
that the textures seem to be too big to load up in the GPU and this may also result in blurry textures and slow performance as the CPU is doing the job carrying the stuff out on the HDD (SDD), while the system RAM will not be utilized much.

Bottlenecks:
From the data gathered when I am running on low FPS (7-8) this clearly belongs to bottleneck of the GPU as it is fully loaded while the CPU is literally doing nothing. The issue for that drop is the enabled alpha vegetation.

CPU utilization:
the CPU utilization is very nice. Nearly all cores have something to do. However there is not much pressure on them at all. If we consider to turn off HT we will loose a bit performance (tested once), I would have my quadcore utilized up to 85% peak and 36% average during the benchmark scene, which are very good results.
However the CPU utilization is very inconsistent, sometimes up to 2 cores are completely unaddressed. This might make sense from a programmers point of view (threading).

GPU utilization:
The GPU utilization at least on my card, using these maxed out settings is excellent. All the time the GPU core is busy. However this do not tell us if the processes are optimized. Just because you are busy with work, it doesn't mean necessarily you are doing your task efficiently. But to estaminate or judge this, is clearly beyond my horizon and knowledge.


Need assistance:
To have a better idea about this all I really could need someone to assist me with a 3 GB or 4 GB VRAM (no dual GPU card) graphics card repeating the same tests, loggin and report.

It is vital to understand if Rome 2 is then filling the VRAM or still using the swapfile, which would be a bad decision to do (slow access). Anyone out there?
another attempt of explaination could be the following:
CA decompresses all eventually needed files from their pack files to the swap file, then loading it to the GPU once needed.

Lookout:
- This was the first bunch of benches. I will do some more tomorrow when the game is minimized as said, so to lift the veil why it makes Rome 2 so much faster.
- Also I will try to reassign the swap file on system RAM (RAMdrive), to see if this could improve the games performance. Placing the swapfile on system RAM should hopefully affect the performance positively. I guess my system RAM should be a faster than my SSD.
- I will also try to test performance completely deactivating my swapfile, this way Rome 2 should be forced to utilize my system RAM directly.

[♔KillZoneGB♔]

" Also I will try to reassign the swap file on system RAM (RAMdrive), to see if this could improve the games performance. Placing the swapfile on system RAM should hopefully affect the performance positively. I guess my system RAM should be a faster than my SSD.
- I will also try to test performance completely deactivating my swapfile, this way Rome 2 should be forced to utilize my system RAM directly"

I look forward to your findings.

R2 has been know to downgrade my settings, just by going over by 2igig mark by 24meg and that's running on 720p.
Even tho unlimted vram option was ticked.

with Vram, you should test out the worse kind of scenario, 3 Ai`s with 20 stacks and your self, all comprising of balanced army, 2V2 city siege.

Spoiler Alert, click show to read: 

Have a look at a buildings banner, (settings at max)

Notice the 80`s style of shadowing (one texture over the banner)
Easy to detect under certain angles as the alpha channels seem to shine.

[RedFox]

unlimited VRAM off
--> RAM usage: 828 MB
--> Windows swap file usage: 2.760 MB
--> VIDEO RAM USAGE: 1926 MB
--> max GPU load /average GPU load: 99% / 98,882 %
--> Max FPS / Average FPS: 46.8 fps / 32.045 fps

CPU Utilization
Core 1: 90.63 % / 49.03 %
Core 2: 58.48 % / 5.23 %
Core 3: 56.25 % / 28.38 %
Core 4: 32.33 % / 7.2 %
Core 5: 34.38 % / 8.8 %
Core 6: 40.02 % / 17.36 %
Core 7: 50.00 % / 7.85 %
Core 8: 46.87 % / 26.85 %

all core load max / avg 51,12 % / 18,84 %

Interesting data, however there are some problems with measurement. First of all, you should make a snapshot of your system before you run R2TW. Otherwise all the measured data is meaningless, since there is no baseline. Can you spend some time and rerun your tests with a baseline?

Secondly, Swap File is a purely operating system managed resource. If a process is running low on Physical Memory, Windows will write least used Virtual Memory pages (4KB chunks of RAM) to Swap File. This frees up Physical Memory.
When the program has to access a part of Virtual Memory that has been Swapped by Windows, a "Page Fault" interrupt is triggered and the Page is automatically loaded into Physical Memory. This process is extremely slow. I should note that all memory accessed by applications is Virtual Memory - only OS and kernel level drivers can access Physical Memory.

If you want to get data on R2TW "Page Faults", check Windows Resource Monitor -> Memory -> [Hard Faults/sec]. You can't actually disable the entire Swap File, since its required for Windows to function. Even if you have 16GB of ram.

Creating Swap File on a RAMdrive might be a very good test though. Still, I'm very interested to see how many Page faults are triggered by R2TW per second. Perhaps that's why user experience is so radically different for so many people? Some have SSD's => faster page faults, some however have HDD's, meaning extremely slow page faults.

--
If CA is using Memory Mapped File IO, then that might be the reason for the large Swap File, too. According to MSDN, Memory Mapped File IO is implemented partly via Swap File. For TL;DR; people: files are mapped to virtual memory addresses and page blocks are read on-demand into physical memory. Additionally, these pages are swapped in and out of RAM as needed. This means a CA programmer wouldn't really have that much control over the process.

[alQamar]

Hi RedFox, I will test that. Thanks for your worthy additions.

Interesting data, however there are some problems with measurement. First of all, you should make a snapshot of your system before you run R2TW. Otherwise all the measured data is meaningless, since there is no baseline. Can you spend some time and rerun your tests with a baseline?

I've made a baseline before the test. The swapfile size I've listed is infact the difference between the base load of my swap file and the file size while he is running the benchmark, so it reflects the pure gain of the swap file. It will shrink by the same amount as soon I terminate Rome 2.exe

[A Barbarian]

A large part of the performance gain is probably that all the DirectX draw calls to the GPU are eliminated. This is actually one of the main reasons for the AMD Mantle API:

AMD claims that Mantle can generate up to 9 times more draw calls per second than comparable APIs by reducing CPU overhead.[3]

http://en.wikipedia.org/wiki/Mantle_(API)


I guess that your benchmark result is minimized about 3 times as high (104fps) compared to normal (35fps ?). Incidentally, Oxide (developer of the 'Star Swarm' RTS game) claims a 3 fold performance increase in their engine when using Mantle

With everything else being equal, enabling Mantle increased the demo's frame rate by nearly 300 percent, from an unplayable 13 fps to a buttery 44 fps

Mantle speeds up the communication between the CPU and GPU, allowing multiple CPU cores to delegate tasks to the GPU without causing a jam. The result is a GPU that does more, like calculating lighting effects for every object, and a CPU that is also freed up to do more, such as simulating independent AI-controlled gun turrets on bigger ships

http://www.engadget.com/2014/01/14/o...y-mantle-demo/


Since Rome 2 is well known for it's CPU limitations (esp visible to the end of the benchmark when the units clash), removing all the DirectX call overhead might explain it.