Release of the CUF (font & text rendering) mod tool

[SigniferOne]

+rep!

[Augustus Lucifer]

The EB FTP server seems to be requesting a password.

[Tellos Athenaios]

A telltale sign that it is Firefox you are using which -sigh- is doing something so brain dead that it is inappropriate to say on these forums. However you can safely leave the password blank because this is anonymous FTP access.

[nedensiz1]

Hi, I have a problem about M2TW Fonts:

Three monts ago I and some of my friends decided to translate M2TW into Turkish. We created a translation group in our web site and started to work on it. We translated some of M2TW files and we have made some progress. But we encountered some problems while translating. We can't use some letters like "Ğ", "ğ", "İ", "Ş", "ş" (which placed in our Turkish alphabet) in some of the text files.

For example: I tried to change Iconium city name in "imperial_campaign_regions_and_settlement_names.txt" file:


Result: I can't see city name on the top


Is there any way to use these letters, add them into M2TW fonts, change some fonts, maybe change font style or something like that?

I tried to edit some fonts on my computer with TypeTool program but it didn't work.

Codes:


We will be happy If you can help us, thanks.

[Tellos Athenaios]

Posting to say that I replied to your copy post on the ORG. (I hadn't seen this one here yet otherwise I'd've replied to it because it seems you are more active on these boards).

[nedensiz1]

Sorry for the same post. I will continue on tw.org.

[Tellos Athenaios]

I'm back with another update. Pretty big one this time around:

CUF files

• Preliminary support for kerning. This is something that people modding M2TW will probably not give much thought, but this should make the tool compatible with ETW CUF fonts too. Thanks to just at the TWC for pointing out to me how this feature is supposed to work in those files.
• Added the code to resolve the long standing todo of having some translation layer for CUFProperty objects to provide a better mechanism for checking the values you want to set.
• Added CUFSourceEditor which is a class that provides support for a declarative approach to editing individual glyph attributes like advance width and y-offset relative to some sort of baseline. This allows people to fix corner badly offset glyphs by injecting better values (this is mainly an issue with italic fonts where rounding errors do more damage to spacing between glyphs). All standard font classes (deriving from AbstractCUFSource) support this feature: you can use edit() and getEdits() respectively to work with it.

Font previews

• Better caching for glyphs. By removing the temporary solution with a proper cache anything that has a lot of the same characters being used (e.g. "Hello world") should see a significant speed up when a preview is rendered.
• Improved support for editing the CUF properties in the table of the preview window.

CUF Shell:

• Fix to allow prompting for user input from fork'ed scripts. In particular this includes a rewrite of the way the shells in the GUI obtain your commands and write output to the GUI.

Help files (need more work to be a proper reference documentation):

• The tool now contains support for the javax.help (JavaHelp) package which provides a Help browser. In addition the original readme.html file was cut into various pieces which are accessible through this browser (and bundled in dist/cuf-help.jar)
• Added necessary code to deal with the case where no JavaHelp is present (at compile time and/or at run time) so you can still build & use the CUF tool without it.
• Changed build scripts so that the Javadoc ends up as part of the help bundle too, which means you can use the search function of the JavaHelp browser to navigate it.

Misc:

• Rather than using an UnsupportedOperationException() stub, some of the code now throws an NotEditableException() when you attempt to do editing operations on read only data (something which would not work anyway).

[nedensiz1]

Nice, thanks.

[Tellos Athenaios]

Pretty big update:

Features

1. Support for JLine. This provides the program with some readline functionality when it runs directly from the command line.
2. Added a “Fontwizard” GUI which provides a for a simpeler way to produce fonts than by writing your own scripts. You can find it under the Build menu. It is a bit rough round the edges, but
   sufficient for quickly building a font or mocking up something. Use the Export buttons to pop up a dialog for previewing/saving your work.
3. Added more extensive logging capabilities to the program/software through IOHelp.Log.

Restructured code layout
Lots of the auxiliary code has been restructured due to the additions of Fontwizard, JLine support and better logging.

A particular item of interest is that the code which handles program settings has been moved from org.europabarbarorum.cuf to org.europabarbarorum.cuf.support, and
that it is now confined to the Setting class (the Prefs class merged with the Setting interface into the Setting class), and the specific program
settings moved to the modules that actually use them.

The upshot of these changes is that if you have script code which manipulates program settings directly you will need to update it.

Another item of interest are the various restrict() methods in FontToolkit: these require now an addition boolean parameter. Old code which used these operations
should use “true” here (this yields equivalent behavior to previous versions of these restrict() methods). Use “false” to invert the result of the old style
restrict() methods.

Bugs

1. CUFWriter catches a couple more errors you might make and fails with an error message of its own.
   So although the net result will be largely the same, at least you will have a useful error message to work wih.
2. The control character font now sets the number of glyphs it contains, along with a slight change in CUFWriter's behaviour this
   fixes two edge-cases: a) that CUFWriter previously could fail seemingly random (but see above) if no glyphs were declared on the font,
   b) that someone might actually want to use the control character font more directly.
3. Add missing check & default values in CUFRenderPanel when looking up certain CUFProperty settings.
4. Fix Classloader to not load classes that the JVM has already loaded. Prevents doing double work as well as duplicating shared resources.

[Tellos Athenaios]

Important update. This update contains a fix for a bug in the code which generated “keyed” strings.bin files. This fix involves generating proper lookup tables at the end of the file: without those Medieval II/Kingdoms suffer significant performance penalties. Mea culpa, the code that generated the strings.bin files was based off alpaca's bin converter which does not (nor does it need to) deal with this. I would like to thank my fellow EB members who tested and suffered the performance hit for their patience and help in debugging the problem, in particular Atilius and bobbin. Thanks, guys!

Apart from having a fix which brings performance back to something more reasonable, the silver lining here is that this fix also addresses a bug in the Medieval II/Kingdoms files. Apparently the data/text/export_units.txt.strings.bin files extracted from the game packs are not built correctly either (implying that the game does not convert export_units properly). The lookup table in this file does not refer to the source code, rather it contains a dump of 795 entries for the 265 RTW units from RTW Vanilla (795 = 265 *3, and there are 3 keys per unit in export_units data.)

This update also contains a simplification of the strings compiler, so the StringsSource types are no longer required. As a result if you were using StringsSources directly, you will need to update to use StringsFeeders. Furthermore setOrder(boolean) in StringsToolkit is gone and replaced with format(String). You can use “Ordered” where previously you would use “true”.

Furthermore this update incorporates some of the improved understanding of the CUF file format, with regards to certain CUF properties. These are “maximum width” and “maximum height” of glyphs. This means that the suitable values will be automatically inferred by the CUF compiler by default. This behaviour can be controlled/disabled per property through CUFWriter.SetVSize, CUFWriter.SetHSize and CUFWriter.SetGlyphDataSize options: see Javadoc for details.

This behaviour is new for VSize and HSize properties, both of which refer to cut-off values (if a glyph is wider or taller than either value it appears cut off in the game). The code rendering the previews in the tool do not take these values into account yet. For GlyphDataSize property it means you now have the final control over what is written to the CUF file.

[Tellos Athenaios]

There's a new important update available. This one fixes a few serious issues, and one less so.
[list]
New features:

• Add support for getting/setting macro names on CUFSources (MacroSource, AbstractCUFSource, Converter).
• Add support for hiding multiple fonts under a single macro (WrappedSource.MacroSource).
• Add support for exporting macros from the FontWizard (ExportDialog).
• Add support for editing CUF properties from the FontWizard.
• Add support for specifying range of characters in FontWizard.
• Add support for shifting character of fonts in FontWizard.

Bugs:

• Fix reference passing instead of copying arrays in AbstractCUFSource.
• Fix broken format message for missing macros
• Fix editing properties on the CUFSource (font) during previews.
• Fix occasional disabled next/cancel buttons in FontWizard

Misc:

• Merge various FontWizard pages to achieve same effect with less code.
• Move common FontWizard UI to its own package.
• Move create new font pages to own package.
• Move algorithms out of FontToolkit.
• Improve error handling in FontWizard pages when building fonts.
• Improve error handling when formatting localised messages.
• Tidy up a number of error messages.
• Move ComponentState from the fontwizard to the gui.support package.
• Use different labels on FontWizard next/cancel buttons depending on context.

[Tellos Athenaios]

There is another update of the tool. This update changes how sub shells interact with each other. Shells now live in their own thread (as managed by the JVM) rather than as task in a thread pool (as managed on top of the JVM). This update also fixes the long standing bug of output messages appearing out-of-order when running multiple shells in parallel.

The result is a tool which is able to tax the CPU more aggressively when a lot of shells are forked, and thus finish parallel tasks faster. Subjectively, the command prompt feels faster from the GUI, as well.

There are a few other minor changes:

• Added SAXHelp which contains the tools for XML crunching outside of generating strings.bin files, should you need it. (I did.)
• Added Context (right click) menu's on text fields and areas with copy/paste/cut actions (as appropriate). Of course you can also use Ctrl+C,Ctrl+V,Ctrl+X.
• Added alternative PlainKeys mode to FormatOptions for StringsWriter: this version retains the old behaviour where lookup tables would be omitted.

[Tellos Athenaios]

Update of the tool. The update itself is fairly small, but the main changes are worth updating for: a bug squashed and the tool is easier to use.

Main changes

• Update Macro, StringMapping and RawHandler to give better error messages.
• Fixed a bug in FontToolkit.fixChartableCompatibilityFor where it could end up inadvertently dropping characters from the chartable. The method was renamed into fixChartable.
• Use SortedMaps for chartables in the tool (most of the CUFSources already did this, but some used HashMaps). This ensure that the order of characters in a chartable is maintained for instance when compacting a font.

Misc changes

• Move algorithms out of FontToolkit, and together with WrappedSource to the font.impl package. Now there is an AlgorithmSource for these operations. (This change allows the code of the algorithms to be simplified and also makes the code elsewhere a bit neater.)
• Remove some redundancy in the error messages for bad file types.
• Fix cuf-impl.xml Ant script which suffered from bit rot. The update-licence task works again on Ant 1.8.1. This has otherwise no effect on the tool.

[Gigantus]

I love how you keep track of stuff - if I just could wrap my head around how to get it working properly.
Any chance one day to have a user friendly interface?

[Tellos Athenaios]

I've identified & fixed a new problem with the tool: the ScriptRunner component would kill its host shell too soon meaning that the exit code of the program could be 0 (indicating no errors) even if errors did occur. This is sufficiently serious for me to release a new version of the tool.

This update also contains a few other changes to the tool. The main ones involve the Shell and they are: fork always expects a boolean argument to accompany the CallBack argument (it indicates whether it the CallBack must run in the same thread as the shell on which you call fork), and there is a blockUntilExit method now if you need to block a Shell until another one exits.

@Gigantus:
If you run the tool in GUI mode you will find a simple and perhaps somewhat spartan GUI to build fonts. To run the tool in GUI mode usually you can simply run the JAR file but alternatively you can invoke the cuf.wsf script from a batch prompt with the -gui or --with-chrome arguments. (Note to people who do not use Windows: you probably want to run cuf.sh instead with same arguments).

Here's a screenshot of that:

[Alone Elf]

I am unable to download the file. The error is:

Your submission could not be processed because a security token was invalid.

[Tellos Athenaios]

It appears the forum suffers from bit rot. The URL format used by TWC download service/tracker apparently changed, hence the old links were no longer valid.

Anyway I've fixed the links, so you should be good to go.

[L'Ost]

*bump*
I don't really understand how to use it ? Your help file directly talk to me as if I'm a software ingeneer that know the cuf format but... I just don't know that format

Could you cover the basics ? Explaining this to me as if I were your grandmother would help me a lot

[Tellos Athenaios]

The help files are unfortunately incomplete, and even more unfortunately do not cover anything related to the GUI itself. So now the help file is only really useful if you are able to cook up code, but need some example bits to get you started and appreciate the easy access to the included source code documentation.

So. The basics. Assuming you want the most basic thing of creating new fonts for your mod without any extra complexity and you want to use the GUI to do it, I'll quote something I PM'ed another modder:

The basics are covered by the “CUF” menu item from the “Build” menu. Simply (double) clicking the JAR file in the dist folder should normally launch the program in GUI mode where you can find the Build menu on the menubar:


The CUF item mentioned above will pop up a new Window which contains a fair few options. One of them is a drop down box with font type. For now, you want to create (convert) a “Source Font”. Specify a name in the name field (this will be used for e.g. file name suggestions). Use something simple, yet descriptive, e.g. “header_font”.
Click the Next button. (You may need to resize the window, it's in the bottom right).


From the next page you can refine your earlier font type selection, you want to set the drop down to “System Font” (TTF/installed font). When done, click next.


From the final page you can import the actual TTF file you want to convert using the Load button on the right and specify other settings such as size (required) and style options (optional). When done, click next.

(Note that strikethrough and underline probably don't do what you want.)

You can make adjustments to settings for your font from the tabs on its overview page:


And use the Export button to either preview or save the result:




This article is available in PDF form with all the pictures at TWCenter's download section.

[L'Ost]

Cool, thanks for your quick reply, and for the "as to your grandmother part", because I don't know very well these things !

But if I allready have the .cuf files ? I have the .cuf files from Shogun 2 (unpacked them from the game). I don't want to create a new cuf font, I want to read these one, resize it and resave it in a cuf format.