EU4 Country Selection Tool

[Richard Karlsson]

I realize that its supposed to automatically update, and nothing shows up

Are you using Internet Explorer?

I just realized that I have the same problem you describe when I use IE. I usually use Firefox myself and haven't tested it using other browsers (which, of course, I should have).

 

[cuendillar]

For those who can't get it to work: What browser do you use, it works for me when I open it in Firefox but not with IE11.

@Richard, the Third Rome patch changed the map somewhat in the Russia-Lithuania area by adding a bunch of provinces. Notably, Beloozero and Rostov are new 1444 countries while Yaroslavl and Pskov have gotten an extra province each. Lithuania, Muscovy and Novgorod probably has a few more each as well. Update?

It's an awesome tool, really. Especially for finding interesting minors to play.

 

[Shadoon4]

Yeah IE is the problem thanks m8

 

[Richard Karlsson]

Does anyone know if it's possible to somehow connect to an updated (official?) database with the most recent information about regions, countries, national ideas, religions, etc.?

It's a huge amount of work to collect and then update the database every time a patch is released, so I would prefer not to do it if it's already done by someone else.

EDIT: Or, if it is possible to extract the information from the game files. Any suggestion would be much appreciated!

 

[defy]

ryukyu>china>yuan>russia>prussia>germany>hre stuff)

Is such a thing even possible?

 

[Zohtun]

Is such a thing even possible?

Yes.

Bloody costly in admin mana but yes.

 

[BSM]

Does anyone know if it's possible to somehow connect to an updated (official?) database with the most recent information about regions, countries, national ideas, religions, etc.?

It's a huge amount of work to collect and then update the database every time a patch is released, so I would prefer not to do it if it's already done by someone else.

EDIT: Or, if it is possible to extract the information from the game files. Any suggestion would be much appreciated!

You could read all the needed values from the game files on each update to populate your own MySql database (could do it on server side with php/javascript code, but I'm not sure about possible copyright issues; better if you use local servers for php/MySql on your own machine to build a database then upload/update to the web server).

For example, you would need to read "history/provinces/*.txt" and parse data for every province, to get development values then add values from each province to get total development for every country. Same for every country, reading data from "history/countries/*.txt".

 

[Richard Karlsson]

You could read all the needed values from the game files on each update to populate your own MySql database (could do it on server side with php/javascript code, but I'm not sure about possible copyright issues; better if you use local servers for php/MySql on your own machine to build a database then upload/update to the web server).

For example, you would need to read "history/provinces/*.txt" and parse data for every province, to get development values then add values from each province to get total development for every country. Same for every country, reading data from "history/countries/*.txt".

Ok, that sounds really good! I will definitely have a look at that when I get home. Hopefully it will be possible to automatize the process to make the updating manageable in the future and result in a more correct Country Selection Tool.

 

[Richard Karlsson]

You could read all the needed values from the game files on each update to populate your own MySql database (could do it on server side with php/javascript code, but I'm not sure about possible copyright issues; better if you use local servers for php/MySql on your own machine to build a database then upload/update to the web server).

For example, you would need to read "history/provinces/*.txt" and parse data for every province, to get development values then add values from each province to get total development for every country. Same for every country, reading data from "history/countries/*.txt".

Ok, so I had a look at the provinces-files you suggested. So far I've been able to loop through all of them and extract the necessary information:

• Province Number
• Province Name
• Base Tax
• Base Production
• Base Manpower
• Owner
• Controller
• Trade Goods

I have a problem though, and hopefully someone will be able to help me with this. For some of the provinces, like The Canaries (example below) there is one set of base-information regarding controller, owner, trade_goods etc. But this is then changed from the year 1402 (as I understand the code).

Does this mean that the first lines of the file, with the basic information, is useless if the game start in 1444? I believe so, but I want to be sure before starting all over and re-writing my code for the data extraction.

-------------------------------------------------------------------

#366 - The Canarias

culture = berber
religion = animism
capital = "The Canarias"
trade_goods = unknown
hre = no
base_tax = 3
base_production = 3
base_manpower = 1
native_size = 5
native_ferocity = 1
native_hostileness = 0


1402.1.1 = {
discovered_by = western
discovered_by = muslim
discovered_by = ottoman
discovered_by = CAS

owner = CAS
controller = CAS
add_core = CAS
is_city = yes
religion = catholic
culture = castillian
trade_goods = sugar
} # Castilian control
1478.1.1 = { capital = "Las Palmas" }
1516.1.23 = {
controller = SPA
owner = SPA
add_core = SPA
} # King Fernando dies, Carlos inherits Aragon and becomes co-regent of Castille
1713.4.11 = { remove_core = CAS }

-------------------------------------------------------------------

 

[BSM]

Does this mean that the first lines of the file, with the basic information, is useless if the game start in 1444?

I think it is NOT useless, as each data is additive. You need to read all info from file start until a date where the game start, updating any field as its value is changed on certain dates. So you can represent the state of all provinces for different game start dates, or just get a single "screenshot" of how the game is at the usual start in 1444.
If you will use a single game state (Nov 1444), just read info from every file, sequentially from file beginning and until you reach a date posterior to the date at which the game start.
In the example about Canarias, read data until
} # Castilian control
and stop when reach
1478.1.1 = {
So you read all lines but when find a line starting with a date + '=' + '{', you need to check that date to decide if you stop or continue until closing bracket '}' then check next date.

For province names, some of the names included in "history/provinces/*.txt" could not match the name you see in game, so take a look at "common/province_names/*.txt" in order to replace each province name with the name used by the main culture of its country owner at the game start.
For example, for province #223 you could guess the name as "Granada" from first line in "history/provinces/223 - Granada.txt", but the game will show it as "Garnatah".
How to do?
1- Read the controller country in line 4: "controller = GRA". (otherwise the "owner = GRA").
2- Read "history/countries/GRA - Granada.txt" and find "primary_culture = andalucian" (line 2).
3- Read "common/province_names/andalucian.txt" and find
223 = "Garnatah"

 

[Richard Karlsson]

I think it is NOT useless. You need to read all info from file start until a date where the game start, updating any field as its value is changed on certain dates. So you can represent the state of all provinces for different game start dates, or just get a single "screenshot" of how the game is at the usual start in 1444.

Ok, well,... I'm sorry,.. I wasn't really clear about what I meant.

The thing is, I thought I was almost done with the data extraction, and then I realized that I have to check for all the dates as well and update the data (as you write). So what I meant was that the base information alone is quite useless. It has to be combined with (some of) the other data. But of course, it is not useless since it's absolutely necessary.

This, in combination with the fact that the files are quite inconsistent (at least to some extent) made it a much more time-consuming project than I had wished for. But I'm still working on it,..

What I've done so far is:

• Familiarized myself with the code structure.
  
• Read in all the 3248 province files and converted the txt-files to a format I'm more used to.
  
• Extracted and cleaned all the basic information that is not conditioned on date:
  • Controller
  • Owner
  • Culture
  • Religion
  • Base Tax
  • Base Production
  • Base Manpower
  • Trade Goods
  • Province Name
  • Province Number
• Saved the information in separate province files

The next steps are to update the files with information conditioned on date and then to merge the files together.

 

[BSM]

Province Name

Just in case you missed it: I have edited several times my last post, specially in regard of the province name.

 

[Richard Karlsson]

Just in case you missed it: I have edited several times my last post, specially in regard of the province name.

Thanks a lot for your help!

 

[Richard Karlsson]

What is done?
Finally the files are merged and the combined file has all the (hopefully) correct values for the starting date of 11th of November 1444.




Next Step
The next steps are the following:

• Checking the information to make sure the extraction of data from the files worked out as intended.
  
• Adjusting the names of the provinces according to their current owner (in the way BSM explained in a previous post).
• Extracting the information on a country level, technology group, geographical region, religion, etc.
• Plan the database, (number of tables, connections, etc.).

The New Country Selection Tool
I've decided to only include the existing countries on the 11th of November 1444. So, no formable or releasable countries will be included.

The new Country Selection Tool will be based on the current version available at: http://richardsvensson.se/eu4/ so the first filtering options will probably be the same, at least for a start.

• Government Type
• Religion
• Technology Group
• Geographical Region
• Number of Provinces
  
• Total Development
• Total Fort Level

Plus a Filter Reset button, and options to order the results by:

• Country Name
• Religion
• Government
• Technology Group

Let me know if you have any feedback regarding any of these things!

 

[BSM]

The top 10 looks nice.
Let me know if I can help in any way.
I don't know if it would mean a lot work, but if you want to get something like in game when selecting a country, you could do clickable each row in the list of results, then show in the right part some info like in game:

Maybe not needed some stuff like liberty desire, ideas monarch or heir, but some other stuff like alliances, vassals and even historic rivals/friends could be handy.
From "history/countries/*.txt" you can get (for "POL - Poland.txt"):

government = feudal_monarchy
primary_culture = polish
religion = catholic
technology_group = eastern
historical_friend = LIT
historical_rival = TEU
capital = 262 # Krakow
national_focus = ADM
1444.11.11 = {
    monarch = {
        name = "(Interregnum)"
        adm = 0
        mil = 0
        dip = 0
        regent = yes
    }
}
more data follows for posterior monarchs

From "history/diplomacy/*.txt" you can parse data for:
vassal, march, union, dependency (tributary states), alliance, guarantee, royal_marriage
then filter out data for end_date <= 1444.1.11 AND start_date >= 1444.1.11
and get the identifiers for both countries involved in each relation (first, second).

An idea suitable for another tool, just in case you want it:
Often in my games I get a single province from the enemy in a war, just to release a new vassal and after the truce use the re-conquest casus belli.
I would like to know the releasable countries from enemy provinces, and a list of provinces for each releasable vassal, to find the cheaper initial province (lower development/war score) and maximum total development or amount of provinces to get in next war.
For this task, I would like a tool where I could upload my save game and examine my possible goals.

 

[Richard Karlsson]

The top 10 looks nice.
Let me know if I can help in any way.
I don't know if it would mean a lot work, but if you want to get something like in game when selecting a country, you could do clickable each row in the list of results, then show in the right part some info like in game:

Maybe not needed some stuff like liberty desire, ideas monarch or heir, but some other stuff like alliances, vassals and even historic rivals/friends could be handy.
From "history/countries/*.txt" you can get (for "POL - Poland.txt"):

government = feudal_monarchy
primary_culture = polish
religion = catholic
technology_group = eastern
historical_friend = LIT
historical_rival = TEU
capital = 262 # Krakow
national_focus = ADM
1444.11.11 = {
    monarch = {
        name = "(Interregnum)"
        adm = 0
        mil = 0
        dip = 0
        regent = yes
    }
}
more data follows for posterior monarchs

From "history/diplomacy/*.txt" you can parse data for:
vassal, march, union, dependency (tributary states), alliance, guarantee, royal_marriage
then filter out data for end_date <= 1444.1.11 AND start_date >= 1444.1.11
and get the identifiers for both countries involved in each relation (first, second).

An idea suitable for another tool, just in case you want it:
Often in my games I get a single province from the enemy in a war, just to release a new vassal and after the truce use the re-conquest casus belli.
I would like to know the releasable countries from enemy provinces, and a list of provinces for each releasable vassal, to find the cheaper initial province (lower development/war score) and maximum total development or amount of provinces to get in next war.
For this task, I would like a tool where I could upload my save game and examine my possible goals.

Thank you! All your comments and suggestions are much appreciated! Besides that, I don't think there is much I need help with, at the moment. Of course, there will be a lot of testing and experimenting with the website as soon as I get something up there. At that point it will be much easier for you to check the information as well, to make sure that it's correct.

To get a small country presentation like you suggest would be ideal, and I will definitely try to implement that. Regarding what information to include, I will focus on the basic things for now and leave diplomatic status and relations for later.

Your tool suggestion sounds great, but I don't know how to create such a feature to allow for save files to be uploaded and analyzed.

 

[Richard Karlsson]

What is done?
The province names has been changed to reflect who the current owner in the date 11th of November 1444. So, at the moment there are two data files.

Countries
The country dataset consists of 666 countries and has the following information:

• country_prefix
• country_name
• government
• technology_group
• religion
• primary_culture
• capital

Provinces
The province dataset consists of 2460 provinces and has the following information:

• province_number
• province_name
• owner
• controller
• culture
• religion
• base_tax
• base_production
• base_manpower
• trade_goods

Next Step
The next steps are the following:

• Read in the information in the two datasets in two tables in the database. Probably the country_prefix in the country dataset and owner in the province dataset will be used to link the two tables together.
  
• Update the country selection tool website to use the new informaiton (and to calculate the necessary numbers).
• Check the information more carefully to make sure it's correct

 

[Richard Karlsson]

So,.... a long time has passed and nothing has happened to the Country Selection Tool. But finally, a new version is online: http://richardsvensson.se/eu4/

It's now updated with the most recent version of the game (1.24.1) and it should hopefully be correct.

Please report all mistakes/errors you encounter here in this thread, and let me know what improvements/new features you want to see in the future.

 

[defy]

Is BYZ missing or am I blind?

 

[Richard Karlsson]

Is BYZ missing or am I blind?

Hmm... no, you're correct. Thanks for pointing it out.

The reason seems to be that no provinces are associated with BYZ in the province-dataset I extracted from the game files. Obsiously an error from my side, I just have to find the source of the problem.

In the dataset for countries there is one observation:
BYZ - Byzantineempire - Feudal Monarchy - Eastern - Orthodox - Greek - 151