Page 1 of 2 12 LastLast
Results 1 to 20 of 21

Thread: PlainEdit

  1. #1

    Default PlainEdit


    This program is intended to facilitate the modification of large contents of text with nothing but a few key strokes. With it, tasks that could take hours or days of painstaking manual work can be accomplished in mere moments. It was born out of my disappointment with regular expressions and their inability to modify matches in the manner I wanted. Though I made it with Total War games in mind, it can be used for any text editing purposes in which attention to detail and repetitive work are required. The program is written in ISO C99 and the GUI in C++ with Qt 5.15. It was cross-compiled for Windows with Mingw in Linux.




    Here’s some examples of what it can do:

    * Split lines in the document in tokens based on a delimiter character, and access each resulting field based on its position, for advanced modifications.
    * Treat records in a line as numbers, not text, and perform arithmetic operations on them, optionally establishing conditions for when this happens. In example, is the value greater or equal to a specific number?
    * Use different records with numerical values in a line as the base to modify another numerical record.
    * Establish a filter for what you want to modify in the file. Areas that don't match this filter will not be modified.
    * Create conditions to determine whether or not to carry out your changes.
    * Change any record in a file according test parameters you establish. e.g in a units definition file, you could change attributes like type, class, category, formations, stats, or any other, and have those changes carried out for all other units in the file that share the same identifying characteristic you specified
    * Export files into a table-like, comma separated value format, useful for viewing in spreadsheet software. With the added possibility of customizing what records are exported, the order, etc.
    * Import files previously exported back into standard format.



    Preliminaries:




    Details:


    The editing section has two main modes of operation, regular and line mode. Regular mode behaves like a normal find and replace function, wherein each pattern matched is replaced with the specified expression. In line mode the program will search for a specific pattern, and will treat the rest of the line as a list, using the pattern as a header and the rest of the line as fields of the list. Modifications can be made to these fields according to various criteria you specify. Fields are identified by their separating character, be it a comma, a space, or any character you specify.

    Different flags can further alter how the program behaves. For instance, the insert flag makes the program insert an expression after the pattern rather than replace it. The discard flag, deletes everything else in the line after inserting an expression, and with reversal, an expression is inserted before the pattern instead of the other way around.

    The true strength of the program, where it really shines, is in its ability to define conditions for when to perform modifications. It is possible to define an unlimited number of such conditions, stating for instance: I want to change this record if this other record has this value, and this other record exists, and this and that or this or that, etc. If all the conditions are met, the program proceeds to perform the requested changes, according to your specifications.

    Unlike other programs with which you might be familiar, you can establish a specific search area within the text to scan for patterns. i.e the pattern need not be matched and replaced throughout the whole document, but in the areas you establish. To establish a processing area, we make use of section markers. By design, section markers are not bound by formatting. i.e markers are matched word for word, regardless of spaces, tabs, newlines or non printable characters in between.
    For instance, imagine we have a book and want to modify an expression in a specific chapter only; we can establish an area encompassing this chapter by setting the starting section marker to "Chapter 5" assuming 5 is the chapter we want to modify, and the section marker end to "Chapter 6". The text will then be scanned between the words "Chapter 5" and "Chapter 6". If no markers are specified, the whole book will be scanned, instead of just Chapter 5.

    The program's built-in help and the included documentation offer ample explanation of each feature and what they do. Instead of going over that here, I will present real cases in which I've used the program. This will hopefully give you some idea of what the tool is about and what it can do for you.




    Real Use Examples:

    In Stainless Steel, cavalry units are way too powerful for my taste. Not only during charges which are devastating even to powerful spear units, but in regular melee as well. To address this, I decided to nerf the base attack damage of all cavalry units by 30%, if their attack damage was greater than 6 points. It was precisely this task that prompted me to create the initial version of this program in the first place. Now, my knowledge of the EDU is very limited, so my changes might not have been the most accurate, or balanced, but they worked stupendously for my taste.
    The task itself I achieved easily with these program options:
    Code:
    Start Marker: "category cavalry" End Marker: "ownership" Ignore: ";" Search Pattern: "stat_pri" Exact: On Line Mode: On Delimiter: "," Field: "1" Numeric: On Replacement: 0.7 Arithmetic: "*" Qualifier: ">7" Condition:"[stat_pri]spear"
    That reduced the base damage of all cavalry with an attack damage greater than 6 (technically >= 7) by 30%. I used a condition to modify the text only if the stat_pri line contained the word spear; the reason for that is that I didn't want to change cavalry archers. I could have used a longer marker like "category cavalry class heavy" but then I'd have to do another pass to cover light cavalry.
    With that done, I decided to "overhaul" the cavalry charge bonus by basing it on the value of the primary weapon. I thought an amount of twice the primary attack value, would be a good number:
    Code:
    Start Marker: "category cavalry" End Marker: "ownership" Ignore: ";" Search Pattern: "stat_pri" Exact: On Line Mode: On Delimiter: "," Field: 2 Numeric: On Replacement: "L1 0.7" Arithmetic: "*" Condition:"[stat_pri]spear"
    After that, I wasn't happy with some new charge values, as they were a bit low, so I increased the charge damage by 2 points if they were below 10 points:
    Code:
    Start Marker: "category cavalry" End Marker: "ownership" Ignore: ";" Search Pattern: "stat_pri" Exact: On Line Mode: On Delimiter: "," Field: 2 Numeric: On Replacement: 2 Arithmetic: "+" Field: 10 Condition:"[stat_pri]spear
    With that done, I nerfed the damage done with their secondary weapon by 40% if it was greater than 8. Some of these knights have insanely high damage in regular melee with their sword or mace, which is why they can obliterate units so quickly:
    Code:
    Start Marker: "category cavalry" End Marker: "ownership" Ignore: ";" Search Pattern: "stat_sec" Exact: On Line Mode: On Delimiter: "," Field: 1 Numeric: On Replacement: 0.6 Arithmetic: "*" Condition:"[stat_pri]spear
    The results: overpowered cavalry units were a bit weaker, but in most cases stronger during a charge and much more manageable in regular melee combat.





    In the same vein with the cavalry changes, I wanted to rebalance spear units and their strength vs cavalry. I thought it best to start from a blank slate and so used the program to delete all the attributes in the stat_pri_attr line. It didn't matter whether my EDU file contained a single spear unit or one thousand. All it took was to write one line:
    Code:
     Start Marker: "class spearmen" End Marker: "era" Search Pattern: "stat_pri_attr" Replacement: "" Discard: On
    Then I did a second pass inserting basic common attributes, spear and spear_bonus_2.
    Code:
     Start Marker: "class spearmen" End Marker: "era" Search Pattern: "stat_pri_attr" Replacement: "    spear, spear_bonus_2" Insert: On
    Technically, this could be done in a single pass(and that's how I originally did it), but I wanted to compartmentalize it here for illustrative purposes.

    This served as a solid foundation on which to base further customization. From here, we can establish further attributes based on whether the unit is a pike unit, the the quality of the unit, the experience, etc. To do that I made use of conditions. Conditions help to establish a set of qualifications that must be met before the program executes our orders. In this case, I wanted 2 things in addition to have the spearmen class: that the unit used a pike and that it was well trained. With those two conditions met, I deemed the unit worthy of having the pike attribute, and gave it a spear bonus of 4.
    This is how I accomplished the task:
    Code:
     Start Marker: "class spearmen" End Marker: "\n\n" Special: On Search Pattern: "stat_pri_attr" Exact: On Line Mode: On Ignore: ";" Delimiter: "," Field: 2 Replacement: "long_pike, spear_bonus_4" Insert: On Condition: "[stat_pri]pike" Condition: "[stat_mental]trained"
    This removed the old bonus, which was the second attribute in stat_pri_attr, leaving now: spear, long_pike and spear_bonus_4. All the units that met the criteria now had the correct attributes. I continued to customize things further by creating a new set of conditions each tailored to fit the change I had in mind.




    Recently, I finished what in most certainty will be my absolute last playIgnore:hrough of the Witcher 3. After a good 120 hours I bid farewell to my friend Geralt of Rivia leaving him to enjoy the rest he well deserves. I've finished the game maybe 3-4 times since its release. but always return for one reason or another. This time, it was a mod that prompted me to return, the W3EE. In retrospect, I regret having played because I had to spend so much time fixing and changing the mod, that it took from my enjoyment of the game. This program helped me more than a couple of times to do tedious edits to xml item definition files. I present one of those below:

    Code:
    <!-- ARMOR LIGHT -->
        <ability name="Light armor 01_crafted 1 _Stats">
            <weight                            type="base" min="4.0"                />
            <quality                        type="add"    min="1"        max="1"        />
            <armor                            type="base"    min="42"    max="42"    />
            <slashing_resistance_perc        type="base"    min="0.10"                />
            <piercing_resistance_perc        type="base"    min="0.09"                />
            <bludgeoning_resistance_perc    type="base"    min="0.09"                />
            <elemental_resistance_perc        type="base"    min="0.10"                />
            <armor_regen_penalty            type="mult"    min="-0.01"                />
            <armor_stamina_efficiency        type="mult"    min="-0.01"                />
            <armor_speed                    type="mult"    min="-0.030"            />
        </ability>
    This is one of several entries that fill the armor definition files. Basically these dictate the properties of armor in the game. I wanted to make light armor have a lesser effect on the speed of the player and, since modifying these by hand would be immensely tedious, I used the to accomplish my task:
    Code:
    Start Marker: "<!-- ARMOR LIGHT -->" End Marker: "<!-- ARMOR  MEDIUM -->" Search Pattern: "armor_speed" Replacement: 0.01 Arithmetic: 3 Line Mode: On Delimiter: " Field: 4 Numeric: On Arithmetic: "+"
    In the above command, I make use of section markers to establish a "working area" that encompasses only light armor, the rest of the armor entries are not modified. I also increase the arithmetic precision to 3 decimal places to match the precision of the numbers in the file and use a quote " character as the CSV delimiter since spaces, and the equal sign were not close enough to the number. As for the operation itself, it's a simple subtraction, I subtract 0.01 from whatever value armor_speed has, so all the light armors in the game ended up 0.01 lighter.
    For the particular example shown above the result is:
    Code:
    <armor_speed                    type="mult"    min="-0.020"            />



    While we are talking about The Witcher 3, here's another task in which I had to use the program:
    Code:
    [Exploration]
    IK_F=(Action=TakePaintGreen)
    IK_F=(Action=Look)
    IK_F=(Action=Stash)
    IK_F=(Action=PlaceBeans)
    IK_F=(Action=PlaceLure)
    IK_F=(Action=GiveAlms)
    IK_F=(Action=PlaceSword)
    IK_F=(Action=PlaceArmor)
    IK_F=(Action=HangPainting)
    This is a snippet of the input.settings file. Basically, this file controls what each key press does in the game. Normally, this can be configured in game, but certain modDiscard: Ondded keys and other select functionality can only be changed manually by changing this file. Besides the length of the file, there's another hurdle to overcome: each key behaves differently depending on the context used so if for instance you want the the space key to open the stash while in exploration mode, but to do something else while swimming, you have to configure those separately. Luckily, with the program I can limit the area of my changes to a specific context by using section markers. Additionally I don't need to look up what key is bound to the action I want to change in what context, I can just say I want to change this action, in this and this context to use this specific key.
    So, going back to the example above and say we wish to change the key used to open the stash in the exploration context to the space key, we can do so like:
    Code:
    Start Marker: "[Exploration]" End Marker: "\n\n" Special: On Search Pattern: "(Action=Stash)" Replacement: "IK_SPACE=" Discard: On Special: On
    The program searches for the action I specify, then inserts the key before the action, removes everything else on the line and finally writes the result to the same file. The result for the example above:
    Code:
    IK_SPACE=(Action=Stash)
    I would have spent a lot of time on this, had I done it manually.








    Download Link


    changelog available in download thread.
    Last edited by XXZit; November 23, 2020 at 07:03 PM. Reason: clarifications. Program update.

  2. #2
    Frunk's Avatar Form Follows Function
    Artifex took an arrow to the knee

    Join Date
    Jun 2009
    Location
    Gold Coast
    Posts
    6,504

    Default Re: String Editor

    Always nice to see new tools! +rep Interested to check this out. EDU has always been a scary file to me.

  3. #3

    Default Re: String Editor

    Quote Originally Posted by Frunk View Post
    Always nice to see new tools! +rep Interested to check this out. EDU has always been a scary file to me.
    Thanks. I'm a bit surprised by the lack of interest. If there's something out there similar or that at least allows you to change EDU attributes in a unit without having to manually modify similar units, I missed the memo; in that case I wish someone had told me sooner to avoid the trouble. In any case I just released an update with fixes and new essential functionality in addition to the source code of the program.
    PlainEdit Multipurpose editor designed to automate repetitive text modification tasks.

  4. #4
    Dismounted Feudal Knight's Avatar my horse for a unicode
    Content Director Citizen

    Join Date
    Aug 2017
    Location
    there!
    Posts
    3,142
    Blog Entries
    1

    Default Re: String Editor

    Low traffic across the board means 2 days is not a good sample for interest. Given the roundabout way of its operation, does reduce the amount of people who will actually try it regardless of its simplicity in command line terms. A visual interface would probably be a strong boost to its usage, as there is indeed a possible market for something nice that performs 'mass functions' and is a nifty addition to the toolbox. Few modders around these days are especially technical to the point of likely needing/discovering/going for this, though I'm sure some will trickle in and appreciate it.

  5. #5
    Frunk's Avatar Form Follows Function
    Artifex took an arrow to the knee

    Join Date
    Jun 2009
    Location
    Gold Coast
    Posts
    6,504

    Default Re: String Editor

    Aye, feel free to post an announcement here, which may provide some attention. It's for resources as much as it is for mods.

  6. #6

    Default Re: String Editor

    Itís possible to use with RTW? Do you think that is compatible?

    ROME TOTAL REALISM ANABASIS FORUM: HERE

  7. #7

    Default Re: String Editor

    Quote Originally Posted by Solon de Atenas View Post
    Itís possible to use with RTW? Do you think that is compatible?
    It can be used for any task that involves text manipulation, within the confines of features I created. Its flexibility and breadth is the reason why it is sort of complex to use and has all those options. So yes, it can be used for RTW or any Total War game or any other game, or any suitable task.

    Recently I used it to remap keybindings for the Witcher 3, and I've used it to adjust out of sync subtitles, just to name two examples.

    Please let me know if the documentation is understandable. I plan on improving it, but I'm afraid to touch it until I have more time because of those bloody  characters that keep appearing out of nowhere.

    It also occurred to me a very simple way of improving usability, by having the program read instructions from the user directly form STDIN; that way people can just double click the executable without having to mess with the Windows cmd first.
    PlainEdit Multipurpose editor designed to automate repetitive text modification tasks.

  8. #8

    Default Re: String Editor

    Oh my god, such I good new!!

    The other day I changed manually the attack value of javelins of hundreds units. I´m used to make this things in "old style", but obviously I was needing something like that.

    Thank you so much!! +rep

    ROME TOTAL REALISM ANABASIS FORUM: HERE

  9. #9

    Default Re: String Editor

    Just a quick note to include additional information, an use example and what is in my TODO list. I don't want to edit the main post until I have time to figure out the mystery of the  characters; I remove them and they appear elsewhere...

    I forgot to mention an important limitation of the mod. It doesn't support character encodings other than UTF8. UTF8 is the most used character encoding in the software world and pretty much elsewhere. And not without reason, it is backwards compatible with ASCII and is the most efficient of them, occupying only 8 bytes per character. UTF16 is common as well in some places with an abundance of non-latin characters, mainly Asia. If you wish to modify a non UTF8 file you can temporarily change it to UTF8 and then back to whatever encoding it had. changing encoding is very simple, especially if you use a good text editor such as Notepad++.

    Howevert, it can be puzzling if the program reports it can't find a match in the file you are reading, when you clearly see the word. So I will add encoding detection and report if the file is using incompatible encoding.

    Other than that these are my current short-term plans:

    -Read from stdin on double click. User enters commands and types Exit when he's finished using the program (Or Ctrld+d)
    -Create an interactive mode, where the program asks the users step by step how and what they want to modify.
    -Pase special characters so the user will be able to pass \t \n " easily without having to escape the characters.
    -Detect and maintain if possible the spacing after a replacement in regular mode.
    -In CSV mode, when non numerical expressions are detected, report the length of the field only(from matching field delimiter until next delimiter or end of line without the newline)
    Currently it's reporting 16 characters and it can be confusing.
    -Fix an issue where a negative number starting with a - will be interpreted as a non numerical field, in CSV mode.



    Use example:

    Just now I was editing item definition files on the Witcher 3 and I wanted to lower the speed penalty of light armor. The files use html format and look as follows:
    Code:
    <!-- Armor Light -->
        <ability= ...>
            <tag.../>
            ...
            <armor_speed    type="mult"  min="0.03"/>
            ...
        </ability>
    To achieve my goal I specified <!-- Armor Light --> as section marker start and <!-- Armor (Next Armor following light goes here) --> as section marker end. Then I set my pattern to armor_speed and using CSV mode I intended to modify the second field in the line,increasing it by 0.01.
    There was an issue here, the delimiting character between these two fields was not a space but a tab and being in windows, I didn't know how to pass the special \t character in Windows. So instead I used " as my delimiter (luckly " did work) The resulting program line was:
    Code:
    streditor.exe -f def_items_armor.xml -m "<!-- Armor Light -->" -n "<!-- Armor  Medium -->" -p armor_speed -e 0.01 -v -d " -c 4 -u -R "+" -o def_items_armor.xml
    In the next version of the program, specifying special characters will be easier.
    Last edited by XXZit; May 31, 2020 at 03:18 PM. Reason: Format.
    PlainEdit Multipurpose editor designed to automate repetitive text modification tasks.

  10. #10

    Default Re: String Editor

    Thank you for making this tool available. I think more people would be interested in this, but they would first need to know it actually exists if you get my point.

  11. #11
    Araval's Avatar Protector Domesticus
    Join Date
    Nov 2010
    Location
    Tartu, Estonia
    Posts
    4,754

    Default Re: String Editor

    Seems like a powerful tool but the command lines scare me.

  12. #12

    Default Re: String Editor

    Hello everyone. I just released the biggest update so far, which, among important fixes and new functionality, brings a new interactive mode that will hold your hand through the process of creating a set of options to make the program do what you want. In addition, you won't have to use the terminal at all to start the program, a simple double click will do. I think other than a GUI this is the next best thing.

    Please let me know how it works for you or if any tweak is needed.
    Enjoy
    Last edited by XXZit; June 21, 2020 at 04:26 PM.
    PlainEdit Multipurpose editor designed to automate repetitive text modification tasks.

  13. #13

    Default Re: PlainEdit

    Bump.
    Just released a new update which implements a GUI, file export-import functionality and lots of small changes here and there.
    People who saw this in the past but were put off by it being cmd only, may want to give the new version a try.
    PlainEdit Multipurpose editor designed to automate repetitive text modification tasks.

  14. #14
    Gigantus's Avatar I am not special - I am a limited edition.
    Patrician took an arrow to the knee spy of the council

    Join Date
    Aug 2006
    Location
    Goa - India
    Posts
    53,125
    Blog Entries
    35

    Default Re: PlainEdit

    An interesting tool, will put it through it's paces to see how user friendly it is.

    In the meantime I suggest to upload the set up here in our download (M2TW\tools) section. Saves you editing your opening post all the time as all you have to do is editing there. Attachment have been lost in the past - that will also ensure availability.










  15. #15

    Default Re: PlainEdit

    Thanks. Let me know what you think. I was thinking perhaps the tooltips could use some work, and I could integrate the help file into a tab in the program.

    I have an update ready, which fixes a memory bug in the file export code that could cause the program to crash, plus a couple of additions to editing mode, like negative conditions. Sadly I can't compile the Windows build with my current machine(my good laptop was stolen), which is a shame because I learned how to to get a normal sized cross-compiled executable.
    If I can get a friend to compile it for me, I'll upload the update and post it in the section you mention. Perhaps that will help with visibility as well and give the program some exposure, although tbh, I don't care if people use it at this point. I stopped working on this for months because of the lack of interest and only came back to do the GUI because, for a while, I've wanted to learn C++ and thought a GUI would be a good starting project. I was not disappointed.

    I was looking for a more "general" tool section since the program isn't just for Medieval 2, but I don't think these forums have something like that. From the top level, everything seems to be compartmentalized into each specific game. :shrug:
    PlainEdit Multipurpose editor designed to automate repetitive text modification tasks.

  16. #16

    Default Re: PlainEdit

    Hey XXZit, the interface looks nice, I'll try it in the comming days. Thanks for releasing it!
    Belovèse's Toolbox: export text files to spreadsheet, detailed unit stats
    Stainless Steel Historical Improvement Project (SSHIP) team member.
    Mini-mods: diplomacy and relation/reputation - detailled unit stats

  17. #17
    Gigantus's Avatar I am not special - I am a limited edition.
    Patrician took an arrow to the knee spy of the council

    Join Date
    Aug 2006
    Location
    Goa - India
    Posts
    53,125
    Blog Entries
    35

    Default Re: PlainEdit

    So far I have only glanced over the provided information but right now I find myself in a position where I wish to add a single SPACE at the end of each line in a 55K line TXT document.
    This tool appears to be ideal to replace\modify existing data, but will it be able to perform the desired operation?










  18. #18

    Default Re: PlainEdit

    Quote Originally Posted by Belovèse View Post
    Hey XXZit, the interface looks nice, I'll try it in the comming days. Thanks for releasing it!
    Thanks. I might improve a few things in the future, add a tab with the documentation, a few extra flags, who knows. For now the most pressing thing is adding a search functionality in the document viewer.
    Quote Originally Posted by Gigantus View Post
    So far I have only glanced over the provided information but right now I find myself in a position where I wish to add a single SPACE at the end of each line in a 55K line TXT document.
    This tool appears to be ideal to replace\modify existing data, but will it be able to perform the desired operation?
    Yes, it can do it easily, but to be honest so could any half decent text editor that supports regular expressions.
    For the task itself, you'll want to use these options:
    Code:
    Search Pattern: "\n"   Replacement Expression: " \n"   Special: On    Exact: Off
    Last edited by XXZit; November 22, 2020 at 09:32 PM. Reason: Messed up code tags
    PlainEdit Multipurpose editor designed to automate repetitive text modification tasks.

  19. #19
    Gigantus's Avatar I am not special - I am a limited edition.
    Patrician took an arrow to the knee spy of the council

    Join Date
    Aug 2006
    Location
    Goa - India
    Posts
    53,125
    Blog Entries
    35

    Default Re: PlainEdit

    Well, I am not really conversant with the 'regular expression' option. I am aware of it existing in Notepad++ and have used it with assistance on the one occasion I really needed it: making sure text files have proper CRLF at the end of a line (a fairly rare formatting issue). I certainly wrote it down:
    Quote Originally Posted by Withwnar
    EOL fixing
    Find: ([^\r])(\n)
    Replace: \1\r\n
    Might as well be Cantonese for all I know
    Much obliged for the quick response, I'll give it a shot in Notepad++ as well.

    Edit: in hindsight this should have been intuitive (if I had a better basic knowledge of it) - after all in the text display files (data\text directory) we use \n to force a new line. This entry:
    text\nnewline
    will display as:
    text
    newline
    Last edited by Gigantus; November 22, 2020 at 10:09 PM.










  20. #20

    Default Re: PlainEdit

    Quote Originally Posted by Gigantus View Post
    Well, I am not really conversant with the 'regular expression' option.
    If you dont already know it, this site is great to try ans learn regexpr https://regexr.com/

    I'm also learning regexpr these days and found it a great help.
    Last edited by Belovèse; November 23, 2020 at 02:09 AM.
    Belovèse's Toolbox: export text files to spreadsheet, detailed unit stats
    Stainless Steel Historical Improvement Project (SSHIP) team member.
    Mini-mods: diplomacy and relation/reputation - detailled unit stats

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •