Text Manipulation - Tips, Tricks and Tools

[MasterOfNone]

Text Manipulation - Tips, Tricks and Tools

(Non-Platform-/Non-Game-Specific Resource)

Having coded RTW mods for a number of years I have picked up some tips and tricks for text manipulation along the way which has made life a lot easier. This post is to share some of the tools and techniques I've used. If you have other tips then please share them in this thread too. This post is not intended to be an exhaustive tutorial.

What I will address here are some resolutions to some of the deeper pains of text editing. All of the tools are free except for Excel (but most wil have that on their PC anyway).

1. How To Global Replace in Multiple Files

Tool: Crimson Editor
Link: http://www.crimsoneditor.com

Crimson Editor has become my number one modding tool since I started using it in late 2005/early 2006. I spent the first year using Notepad and wish I had found this earlier. It does so much. Currently a new version is being developed called EmeraldEditor which will no doubt do even more! There's an entry for CE on Wikipedia should the link above ever expire.

CE is a tabbed editor, meaning it can have more than one file open at a time. When you click the Search | Replace option from the drop down menus a dialogue box will open and you can change the default radio-button option to "All open files". CE will then replace your search term with your new word or phrase in ALL the files open in CE. This can save a ton of time when, for example, changing internal names of regions. You just open up every file that may refer to the internal name of a region and Bob's your uncle As with all text manipulation you need to be savvy about it. You might, for example, choose to activate the "Match case" option when wanting to differentiate between internal and external names, or include curly brackets in the search term for some files so as to target only the internals for files that use curly brackets.

Notepad++, I am told, also has this functionality.

2. How to Join Lines

Tool: Crimson Editor
Link: http://www.crimsoneditor.com

I felt very stupid when I found this. I had been doing it manually for years! Sometimes you will want to include text in your mod and the lines will be broken rather than wrapped. CE's Edit | Advanced | Join Lines feature will get rid of these swiftly. Just place the cursor at the beginning of the block of text and then hit ALT J (you can hold it down to go faster) and it will repidly remove all line breaks.


3. How to Concatenate

Tool: Excel
Link: n/a

This can be useful for creating internal curly-bracketed tags once you have your "lookup" list. This is very easily done by Concatenation. Let's say you have George in cell D31. In A31 (or wherever you want, though I'd recommend this 2-column gap as it will copy and paste the right number of tab spaces into the names.txt file) you could type: =CONCATENATE("{",D31,"}") and this will cause {George} to display. Then just highlight the entire D column and copy the formula down and your internals are done!

4. How To Compare Changes

Tool: BeyondCompare3
Link: http://www.scootersoftware.com

This is very useful for Build Managers - those who maintain the current build of a mod in develpment. It has the ability to compare folders and files so you can compare not only different files but also entire versions of builds. It's quite intuitive to use. It colour highlights differences between files and folders. Howevev, it is not just a file/folder-comparision utility, it also allows you to merge or copy over parts of text from one file to another to "sync" them etc. Extremely useful for debugging

Another free compare tool is WinMerge: http://winmerge.org/ but I have not used it so cannot comment.

5. How To Perform Multi-Line Editing

Tool: ElitePadLite
Link: http://www.editpadlite.com/editpadlite.html

This had me beat for a long time until I found this tool. Have you ever wondered how to search and replace terms than are on multiple lines but which are not unique? For example, some files list culture blocks like this:

variant
		{
			cultures
			{
				barbarian
				nomad
				hun
			}

How would you go about adding a new culture to the end such as "celtic":

variant
		{
			cultures
			{
				barbarian
				nomad
				hun
				celtic
			}

You'd have to do it manually with nearly all free text editors. That's where ElitePadLite comes in! This is not entirely intuitive though so here's how to do it:

1. Open the Search Panel (Search | Show Search Panel) - it appears at the bottom, under the main window
2. Paste in the upper window of the Search Panel the lines of the text block to be searched for
3. Type what you want to add/change in the main window and then copy that to the lower window of the Search Panel
4. Press the Replace All button of the Search Panel

This is a powerful tool. For example, you can now add a building to all settlements in the descr_strat.txt file with one click of a button if you so chose!


6. How To Batch Rename File Names

Tool: LupusRename
Link: http://rename.lupasfreeware.org/download.php

This utility will allow you to mass-batch edit filenames in any selected folder. It has quite a lot of options for detailed renaming. This can, for example, rename all files beginning with "#roman_" to begin with "#barbarian_". Essential when setting up new cultures among other things!

7. Regular Expressions (Complex Replacing)

Tool: Crimson Editor
Link: http://www.crimsoneditor.com

Tool: Notepad++
Link: http://notepad-plus.sourceforge.net/uk/site.htm

If you want to replace a specific re-occuring element within variable other elements then use aqd's "regular expression" method in post #11 of the version of this tutorial in the general modding resources' forum. This is extremely useful. You can find out more about regular expressions at http://www.regular-expressions.info/

[florin80]

This is a modder`s paradise! Very good job!
One - possible stupid - question. Is BeyondCompare3 (point 4 on the list) also able to compare non text files too? I mean I`d be interested in something that can make you a list with differences of size and date between two sets of non text files(say textures and models). That will also highlight if a file is present in one of the sets/folders and absent in the other. Something able to do that would make life very easy for the interface(portraits) and other graphics folders when you want to be sure you haven`t forgotten to add something or make sure you have the latest version of the files.

[Kylan271]

Appreciate this resource update,rep for you-even though not shown...just saying thanks.

[MasterOfNone]

Thx.

Yes, BC3 does show if a graphic or audio file is different from another, though it does not show the actual difference like a text file obviously. One thing to watch out for is that sometimes it is over-sensitive and reports identical files as different - I think it does this because of things like someone sending in a file from a PC in a different time zone (so the file is seen as having a different time stamp). However, there's a function on BC3, something like "binary comparison" which should refresh the display and get rid of these blips.

[toluas]

Great. Thanks.
+rep