file "db/unitnames.csv" localization does not work

[OLDodin]

I noticed that in versions 1.10+ the file "db/unitnames.csv" can no longer be localized. Works correctly only with text encoding win-1252. The remaining files (airnames.csv, armynames.csv, navynames.csv) support localization as in the version 1.07

 

[Czarina Julie]

Hi OL Dodin,

I'm a new C++ developer on the 1.13 release team (new to the team in the last year but old, like really old...retired) . I can tell you that all the .csv files in AoD directory (...\config) use an Encoding of Western European Windows-1252. There are 2 caveats to this:

• The Russian Cyrillic alphabet isn't part of Windows-1252 and AoD doesn't use UTF-8. So to get Cyrillic to work in AoD the code "translates" Windows-1252 to Cyrillic Windows-1251 Encoding.
• The Polish language has a similar issue that the Russian Cyrillic alphabet has, that it "translates" Windows-1252 Encoding to another Encoding. That Encoding is Central Europe, Windows-1250.

The ...\db\unitnames.csv file (not to be confused with the ...\config\unit_names.csv file), doesn't do any Encoding "translations." So, what you see in the unitnames.csv is WYSIWYG (what you see is what you get). If the expectation is to see Russian units using Cyrillic, or Arabic countries using Arabic, that can't be currently done. So for an example, I took a screen shot of Statistics, Divisions to show how the unitnames.scv works:

You will what gets "translated" form one encoding to another encoding. The unitnames, armynames, airnames, and navynames will not be translated.

As for when this change, I can't tell you since I've only know that it's been Windows-1252 on my (US) English laptop ever wince I've had the game.

I'm going to try something, look at the code for the amount of effort, and get back to you later this week.

 

[OLDodin]

Just in the version 1.07 it worked for all files (airnames.csv, armynames.csv, navynames.csv, unitnames.csv) - text displayed correctly for win-1250 and win-1251.
Since version 1.10 only one file unitnames.csv stopped displaying correctly text.

All of these encodings are single byte and are simply read byte by byte. And then the value of the byte is interpreted according to the selected language in the launcher. For example, if Russian is selected, another font (gfx\fonts\kyr\ - for 1251) is used. But for the db/unitnames.csv file, the launcher settings are ignored or something wrong.

Through disassembling, I saw that the processing for the db/unitnames.csv file for some reason became different from the other three files. I would like to avoid patching the exe file if it is possible to restore the work as in version 1.07.

 

[Czarina Julie]

From the actual source code I have now (the 1.12 release), there is only one encoding, but there are many cpps (with their headers) and I'm probably haven't dug much into the language support. I know that years ago there was some talk about going UTF-8 but don't know the story behind that. They are very protective of the source code and even though I have access to it now, it's code I didn't do. We I go to add some functionality or troubleshoot a bug, I usually have to educated guesses on what to search through the entire project for, add some breakpoints, then run the game stepping through it until I get that "Ah Ha" moment. Then fix it.

When I get back to the code (currently doing the new technologies, their components (research), etc.) then I'll be able to spend more time looking for a better answer. I'll try to go back to previous developer's versions to see where it was changed and what (if any) notes made why it was changed.