"Insufficient memory" & "CTD" - workaround
It's only a matter of time before this becomes common knowledge, I hinted at this issue for a couple days already, so I wanted to handle this subject responsibly because it is something I am well versed in.
Do you get "Insufficient Video Memory" crashes?
How about repeated crashes to desktop precluded by everything slowing down and feeling sluggish?
These are obvious signs of running out of virtual address space.
You may even get a crash after several battles or a another repeated crash due to this specific cause.
I'll keep this article basic and in laymens terms with a little explanation, you can spend a week researching it to begin getting a deep understanding of it, so I will keep things simple since I have done the heavy lifting for you.
BACKGROUND (explains the issue)
What is Virtual Addressing?
Virtual Address Space is a set of numerical addresses that windows maps and keeps track of for memory usage. It has nothing to do with how much RAM you have or how much virtual memory you have. There is only a fixed amount of addresses available for a 32 bit application like empire and that is 2GB, regardless if you are on a 32 bit or 64 bit OS. A 332 bit OS itself has 2GB available in "user" address space, with 2GB normally called "kernel" address space. In addition to this, and irrespective of this, a 32 bit application is limited to 2GB for it's address space determined by the binary header (this is true for 32 bit and 64 bit OS's alike). A person cannot track virtual address space with tools like task manager.
Why are you affected when others are not?
Each machine maps different amounts of virtual addresses on start up. It's affected by many things. Large impact to virtual addresses being reserved, and therefore becoming unavailable, is due to things like Vista, large memory video cards, multiple devices whose drivers reserve addresses on boot up, etc. These reserved addresses can be viewed in ones device manager.
Why is Empire affected by this issue?
Quite simply it uses a lot of memory and we are still in predominantly a 32 bit world. Addresses are limited and Vista and high memory video cards reserve and make addresses unavailable. CA chose not to utilize 64 bit safe coding practices OR natively support 64 bit so people with 64 bit OS's are also affected by this issue.
This is predominantly a 32 bit problem, as game assets grow then memory footprints grow. Also people with large video ram (>768) usually have no idea that they are cutting their available address space that video applications can use. It's why 1GB video RAM is not as common as one would think or is never fully utilized used by devs, because the operating system cannot handle really it due to limited addresses.
This three part series on a "Messy Transition" at Anandtech was born from a case study of a memory address workaround I made for Supreme Commander, similar to the one I am making here, and it is a great series to explain the issues and brought alot to the surface for all to see.
Part 1 -http://www.anandtech.com/gadgets/showdoc.aspx?i=3034
Part 2 -http://www.anandtech.com/cpuchipsets...oc.aspx?i=3044
Part 3 -http://www.anandtech.com/systems/showdoc.aspx?i=3060
What can the developer do?
Two things primarily. Lower memory usage by the game, that will likely be the first goal but that may not be enough for people who's machines reserve alot of addresses on startup for the video card. Secondly, they can make the game 64 bit safe allowing people with 64 bit to immediately bypass this issue. While 32 bit users can then extend the OS's user address space, not optimal, but likely the best solution we will have unless they drastically reduce memory usage and management in game. Making the code 64 bit safe is unlikely because of the work involved and would have been much easier to do properly from the beginning. CA have a full plate and have many bugs to address that need attention so developer resources and money is limited, that is why the latter option is doubtful. For the foreseeable future this workaround should help if no real progress is made otherwise, remember hitting the 32 bit limit is not hard and many games do.
EDIT: According to a post on this page below, the developers are indeed preparing a 64 bit "safe", not necessarily native 64 bit, Large Address Aware executable that is currently in testing. So this is encouraging and is a pleasant surprise from what I expected, definitely good news.
EDIT: This update to the executable has arrived on the 4-29-09 patch.
THE WORKAROUND (the details)
I am providing a workaround similar to the one I provided for Supreme Commander back in the day here for reference.
What does it do?
It has to do two things that both rely on each other to work. For 32 bit operating systems we have to tell the operating system it can use more than 2GB user address space, 2.5GB actually, while leaving 1.5Gb for the kernel. We also have to set the Empire.exe executable to be allowed to address beyond the 2GB ceiling. 64 bit OS users don't have to do anything.
What does it help with?
This is not the solution for all crashes. It deals primarily with the crash to desktop that is caused by lack of virtual address space.Due to the unique nature of how addresses are mapped differently on each machine this crash may happen at different places and times of the game, even loading a map, menu or waiting on an AI turn. If you try this fix, and you follow directions properly, and it doesn't help you then your crash is due to something else.
How do I do it?
Follow directions explicitly. I have seen people consistently skip steps or not read the directions properly so it doesn't work for them whne they don't do what it says. it's not as complicated as it sounds, I'm just being thorough.
APPLY THE WORKAROUND (juicy caramel center)
Requirements
Only real requirements is that you have atleast 2GB RAM. You don't need beyond that even though we are allowing addressing up to 2.5 GB. Also following instructions and being somewhat familiar with working with files in windows is required. Make sure you are using Vista SP1. If not then you need this hotfix that fixes a major blunder with Vista address space - http://support.microsoft.com/kb/940105
That's all, follow the 3 steps.
1) Prepare the operating system to be able to use beyone 2GB address space with a userva of 2.5GB (only 32 bit OS's need to do this): I am purposely using only 2500 for userva for specific reasons and any more will likely not be needed, as a result the kernel will have plenty of space to play in.
All 64 bit OS's
Nothing to do on the OS.Proceed to step 2 on setting the executable. 64 bit OS's running 32 bit binaries are also limited to 2GB application space.
XP 32
* Find your Boot.ini located usually on the C:.
If you cannot see it, you probably have system files hidden, then follow these instructions at microsoft (also uncheck "hide protected operating system files")
* Right click on the boot.ini and choose properties. Uncheck "read only"
* Open the boot.ini by double clicking it.
* Under the section called [operating systems] find the line that says "multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows XP Professional...". Yours may look a bit different. We are going to make a duplicate, so that if you make a mistake you will not harm yourself from booting into your computer. Drag you're mouse across the entire line, right click and choose copy. Then right below the line you just copied right click your mouse and choose paste. You should now have two lines one after another that are exact duplicates.
* Change the bottom line that you just copied with the following addition on the end. Add a space at the end of the line and then this without the quotes "/3GB /USERVA=2500"
* In that lower boot entry you just made, find the name of the operating system in quotes and change it to something like "Windows XP 3GB", so you can differentiate at boot up.
* Close and save the boot.ini.
* Right click on the boot.ini, hit properties, check read only.
Vista 32, Win 7 32bit
* In Vista go to Start menu. Go to Accessories. Locate the command prompt shortcut and hover mouse over it. Right Click on the shortcut then select Run as Administrator.
* In the command prompt type this exactly: bcdedit /set increaseuserva 2500. Then hit enter.
* Make sure you get a message back confirming the change was made. To verify the entry is there you can type just bcdedit, hit enter, and you should see the entry now listed.
* Then close the command prompt. You just told Vista to increase user virtual address (userva) space to 2500MB.
* Changes take effect on reboot but don't reboot yet.
SUMMARY
We extended the user virtual address space in XP and Vista to 2500MB, so now applications can utilize up to that much if they are set to be Large Address Aware. While some 32 bit games/applications are set, Empire is not.
2) Now reboot the PC because Windows needs to set the userva at 2500 which only happens after startup.
If you skip any step it will not work. When you have rebooted you should be good to go. Run the game as normal with the original game shortcut.
TO UNDO THE CHANGES
For the XP 32 bit just delete the line from your boot.ini.
For Vista 32 Open command prompt as administrator and type: bcdedit /deletevalue increaseuserva. That deletes the entry.
Reboot and you are back to normal.
THIS IS NOT A FIX
No it is not. We should still continue to put pressure on the developers to fix this issue. A bad side effect of user made workarounds, while they actually help people be able to play the game which is important, the problem is that devs stop hearing about the issue by a large group so they think the impact isn't big. So continue to update the issues and CTD's you have to the official boards. One such topic exists here on the official boards: https://web.archive.org/web/20161220...om/topic/49357
POSSIBLE ISSUES
There are two side effects to watch out for and one addendum.
1) A very small percentage of machines may have issue booting due to a badly written driver after making the change to the OS in addressing above 2GB because drivers utilize higher addresses. To fix this boot problem, with XP you have the other boot option that appears at boot to choose. With Vista go into safe mode and follow the undo instructions above on deleting the userva entry. This workaround will not work if you have a bad driver affecting your boot up.
I'll edit the topic as necessary. Post your experiences, I will try and help if can with specific issues on this workaround.
OTHER REFERENCES (incase the wall of text wasn't enough)
Windows memory limits in 32 and 64 bit:
http://msdn.microsoft.com/en-us/library/aa366778.aspx
A bit about memory addressing and 3GB switch:
http://www.microsoft.com/whdc/system...AE/PAEmem.mspx
Thanks to TheSpaceHamster/TheCookieMonster for testing this and starting a thread in the official tech support that got my attention.
Reply With Quote
Originally Posted by jmbuehler
