Belovèse tool for extracting data from EDU

[Belovèse]

Well, as a matter of fact, we did. I didn't know you'd be making changes in the EDU. There're some accents changed by JLMP; I've also run your program, had a look at some units, and adjusted many plances (eg. recruitment times of urban militia, or some heat stats, and also added type "artillery" to the relevant units). Your program is very helpful in finding odd things.

Happy to help! I have a quick question: in the "soldier" entry, for some unit there is 4 numbers instead of 3 (like in georgian light spearmen), is it normal?

However, I think it's easy for you to make your changes: just Ctrl-F " ; " and replace with your text, and you'll get 484 cases (so 10 units missing, to be found otherwise).
But I would be inclined to use in the EDU something more simple, why to make long < dictionary Javelinmen ; Levy Javelinmen SSHIP_class feudal levy >, and not simply < dictionary Javelinmen ; Levy Javelinmen ; feudal levy > ?

I can use almost any separator you like, but unfortunately the semicolon has a hard-coded use in the programing tool I use and I can't find a way around it. I agree it should be easy and basic to use semicolon, but remember it's built with live musical performance as objective, so what I'm doing is far fetched! But we could use (almost!) any another symbol or word, tell me what would be the best for you and I'll try it.

And just drop a line before if I shouldn't meddle with EDU ;-)

Will do! For now I'm not sure how much time I'll have the next days, so I think it's best you work on it as you please, and when I finish the new version of my program I'll implement the keyword in "dictionary" myself and re-test it to save you some time.

[Jurand of Cracow]

4 not 3 - I've got no idea.

frankly speaking, why not a simple colon? It doesn't have double meaning, it looks clear.

dictionary Woodsmen ; Woodsmen, feudal levy
dictionary Mercenary_Frankish_Knights ; Mercenary Frankish Knights, mercenary unit
dictionary georgian_light_spearmen ; Metsikhovne Spearmen, urban militia

[Belovèse]

frankly speaking, why not a simple colon? It doesn't have double meaning, it looks clear.

dictionary Woodsmen ; Woodsmen, feudal levy
dictionary Mercenary_Frankish_Knights ; Mercenary Frankish Knights, mercenary unit
dictionary georgian_light_spearmen ; Metsikhovne Spearmen, urban militia

I tried it this morning, the colon works! Howerver in the exemple you used a comma, wich also as a hardcoded use. But this works:


dictionary Woodsmen ; Woodsmen : feudal levy
dictionary Mercenary_Frankish_Knights ; Mercenary Frankish Knights : mercenary unit
dictionary georgian_light_spearmen ; Metsikhovne Spearmen : urban militia

So I'll go with that in the program.

[Jurand of Cracow]

I tried it this morning, the colon works! Howerver in the exemple you used a comma, wich also as a hardcoded use. But this works:


dictionary Woodsmen ; Woodsmen : feudal levy
dictionary Mercenary_Frankish_Knights ; Mercenary Frankish Knights : mercenary unit
dictionary georgian_light_spearmen ; Metsikhovne Spearmen : urban militia

So I'll go with that in the program.

Very good! I'm looking forward to your program for after getting it I'll develop my formula for assessing the quality of the units and then their prices.

[paleologos]

It's a work in progress, but if it would be useful for you I would put more time on it.

It would be immensely useful to me, please do make it!
I hope it works with all mods.

[Belovèse]

It would be immensely useful to me, please do make it!
I hope it works with all mods.

Thanks for the rep!

You are of course welcome to use the tool, I think I'll open a tread in the Workshop in the future. I think I could adapt the program to other mods quite easily. That said I'll first tailor it for SSHIP: it's the only mod I play currently and it's my way to give something back to the devs.

But I think my program will still be usable for other mods even before I adapt it, with maybe a few missing entries.

Here's the roadmap I have in mind:
- Release v1.0: a few days
- Adding missing entries (era, officer, formation): a few days
- Adding support for EDA and EDCT: a few weeks
- After that I'll open a workshop tread and will add support for other mods, on demand and if I can!

And I'll also clean this tread and make a nice first post as soon as I can edit my own posts.

Cheers!

Sent from my FP2 using Tapatalk

[Belovèse]

Hello everyone, here is the new version. It's a work in progress, I have a few things to finish but as I won't have much time the next days I found it best to releases it "as is".

What's new:
- a lot of the stats have their own colums (like the soldier entry, etc...). I tried to aim for the SS6.4 file Jurand sent me.
- I isolated the is_peasant attribute under the "law" column, to show if a unit is at 100% or 50% for maintaining order. I could later isolate all the attributes.

Very good! I'm looking forward to your program for after getting it I'll develop my formula for assessing the quality of the units and then their prices.

All the stat_pri data have their own column (attack, charge, missile type, ammo, etc... ), I think it'll be useful for you. There is a small bug affecting attack rate and skeleton stats if the line as a comment of a certain length at the end, I'm looking for a workaround. If you like it I'll do the same for stat_sec and stat_ter.

Jurand, if you send me the last version of the EDU I could add the comma separator on dictionary entries today.

[Jurand of Cracow]

Here is the latest version of the EDU.
Separation of all attributes will help as well indeed.
Thanks, Belovese!

[Belovèse]

Here is the EDU with the colon separator.
I added a few informations in the beginning of the file, to explain the colon separator in dictionary. I didn't ask you first because I was trying to send you the EDU as fast as possible, I hope it's okay, I'm not yet sure of the customs here about documenting the edits done to files.

I tried it with the program, it works. I also have tested a fix for the skeleton and attack rate, so I'll implement it in the next days and will finish to make separate columns for everything.

I think it would be useful if I do the same for EDCT and EDA? I believe (but have not tested yet) I could export my new program logic to at least those files as well, in the comming weeks.

[Jurand of Cracow]

Here is the EDU with the colon separator.
I added a few informations in the beginning of the file, to explain the colon separator in dictionary. I didn't ask you first because I was trying to send you the EDU as fast as possible, I hope it's okay, I'm not yet sure of the customs here about documenting the edits done to files.

I tried it with the program, it works. I also have tested a fix for the skeleton and attack rate, so I'll implement it in the next days and will finish to make separate columns for everything.

I think it would be useful if I do the same for EDCT and EDA? I believe (but have not tested yet) I could export my new program logic to at least those files as well, in the comming weeks.

Adding information is absolutely ok! Thanks for it!

However, the file you've uploaded is not EDU, but the .cvs :-) Very useful, but I need also edu.txt to make changes here and there ;-)

Btw, I hope it's still possible to have values of the stat_pri_armour or stat_cost in the separate columns (in the csv they're not, but I think it depends on how you run your program.

EDCT and EDA - if you can do it, it'll be quite useful. It think though that the structure is a bit more complicated (more levels etc.). But overcoming obstacles is THE fun of modding or coding programs, isn't?

[Belovèse]

Adding information is absolutely ok! Thanks for it!

Ok, noted!

However, the file you've uploaded is not EDU, but the .cvs :-) Very useful, but I need also edu.txt to make changes here and there ;-)

Ops, my bad, I was in a rush and didn't double-check! Here is the right file.

Btw, I hope it's still possible to have values of the stat_pri_armour or stat_cost in the separate columns (in the csv they're not, but I think it depends on how you run your program.

Yep, it's some of the entries I still have to rework in this new version. I'll prioritize stat_sec, stat_pri_armour, stat_sec_armour, stat_cost and stat_mental for the next release, then will make all the rest.

EDCT and EDA - if you can do it, it'll be quite useful. It think though that the structure is a bit more complicated (more levels etc.). But overcoming obstacles is THE fun of modding or coding programs, isn't?

Yes it's really fun and challenging! For EDA and EDCT, my program cares only about keywords and lines number, so that's why I think it's good to at least have a try.

[Jurand of Cracow]

Yep, it's some of the entries I still have to rework in this new version. I'll prioritize stat_sec, stat_pri_armour, stat_sec_armour, stat_cost and stat_mental for the next release, then will make all the rest.

Yeah, these are most important for my formula.
Among the attributes I pay attention are also:
- hardy/very_hardy (this has quite an impact on morale),
- free_upkeep_unit (you can save money with that militia),
- extreme_range (important for quality of archers),
- cantabrian_circle (for the HA),
- command (adds morale to the neighbours),
- frighten_foot, frighten_mounted (removes morale)
And info if the unit can form shielwall or schieltrom.
I also count the stat_ground.

[Belovèse]

OK, then I'll release next version when it'll support all these. I think sunday at the most, maybe sooner.

[Belovèse]

Hello everyone, here is the new version.

What's new:

Spoiler Alert, click show to read: 

- added a "STOP" button
- workaround for the changing number of arguments in stat_pri
- corrected "law": it was upside down (is_peasant was giving 100% law instead of 50%)
- added almost all single columns for the supported entries
- minor graphical changes

Please give your opinion about the column names and how it works, in the meantime I'll work on minor quality of life improvements: select/unselect a whole entry at once, keep in memory the selection event if the program is closed, and anything you suggest and can manage. I'll also add the missing columns an entries, and work on any bug reports.

Question: would you like to have a separate column for ALL attributes?

Yeah, these are most important for my formula.
Among the attributes I pay attention are also:
- hardy/very_hardy (this has quite an impact on morale),
- free_upkeep_unit (you can save money with that militia),
- extreme_range (important for quality of archers),
- cantabrian_circle (for the HA),
- command (adds morale to the neighbours),
- frighten_foot, frighten_mounted (removes morale)
And info if the unit can form shielwall or schieltrom.
I also count the stat_ground.

I think I've done what you needed, but please tell me if you need any improvements/changes.

[Belovèse]

I spotted a few broken options: voice, "soldier health" and hardness. It's fixed and updated on the previous link.

[Jurand of Cracow]

Thanks a lot, Belovese,, I'm using your final version.

At the same time we need to think which parameters are actually important. Eg. in this entry it is reported that "command" might be a non-working attribute.

Belovese, maybe you can have a look at this submod for the EBII, maybe have a look at how the stats are presented in-game in the Divide et Conquera, and maybe you'd be able to do similar things for the SSHIP?
The issue is: the descriptions for the units do not contain enough data, and after the recent changes the descriptions of the "type" and "quality" of units are from the SS6.4 and completely differ from what you call SSHIP_class.

[Belovèse]

JoC, I had a look at the EDII tool link. I don't know Python, that is used for the script, and I fear what you're looking for is a bit complex in the only tool I know, Max/MSP. I'll give it a try but it'll take weeks or months, and I think someone like XXZit would be your best option. That said I was looking for a project to get me started in java or python, this could be it, but that also would take some tome before I could give use something usable...

About the non-working (or suspected) attributes, I'll put some message in the program. I was looking at this tread that list those suspected triggers for my syntax highlighting, if I find the same for unit attributes I'll use it at least for documentation.

[Jurand of Cracow]

Belovese,

your "EDU to CSV" program is very useful for me, I'll publish the results once I come up with all formulas and balancing.

If you'd put your time into Python and then produce a tool for the units' description file, as detailed before, this would also be very valuable - and would be applied directly into the files. The descriptions of the units are very much outdated so new info would be extremely valuable. How long does it take, doesn't matter much. The result is important :-)

JoC

[Belovèse]

Thanks! Well, for now I made a few tries at the EDCT and EDA (and it's going well so far), but is export_unit more a priority for you?

[Jurand of Cracow]

Yes, indeed, the export_unit is more important for me as the descriptions of the units is very outdated now.

The almost final version of the file (03 July) has been made with help of your program (errors inside, for sure).