A Validator For The Glory

[Jamie550]

By the way, there is a misconception regarding the roles of colonynames.txt and the new_colony entry in countries.txt.

The names in colonynames.txt are always consulted; e.g. if your tag is BRE, it will first check the names in colonynames to see if there are any for BRE (which there are), and, only if those are already used, it will use the new_colony entry of GER to consult the list in countries_special.txt. There it will find that for GER, it should make a name starting with "Neu " and followed by the name of a city owned by BRE.

So the new_colony entries don't actually point to entries in colonynames.txt.

(This is actually what it says in the header of countries.txt.)

Fixed - now any tag is accepted in "new_colony".

 

[Andrei Gijgorev]

--- When starting the program on my system (WinXP pro) the window is out of the bounds of my desktop. It may have something to do with me using large font sizes, not sure. When I change the size of the program window the top bar appears and I can move the window around, so it's not a big problem. Just wanted to mention it...

--- AFAIK there's a limit on ID numbers, some 7 digit number IIRC. Using an ID higher than this number can lead to a CTD on loading, or the game complaining about attempting to create a monarch/leader/whatever with an already used ID. I don't know the exact number myself, but I'm sure someone with access to the sourcecode can provide it. Since the error message given by the game is incorrect, I think it's important to include this check.

--- The validator complains about multiple instances of country tags in the revolt.txt. I remember how a few years ago I successfully used a technique which involved multiple country tags with different dates to achieve a revolting nation with different revolting provinces depending on the year of the revolt. I'm not sure whether that was EU2, CK, or HoI2 though. MichaelM, could you comment on the possibility of this method in FTG? Because if that's a valid method, then I suggest checking the date spans of the respective revolter countries before throwing an exception.

--- Please include a validation that technology levels in the scenario file are within the bounds of said technology as defined in \db\technologies.

=o= The validator seems to complain about the "absolutestartyear" header key which is new in FTG. Also it complains about the absence of "startdate" which can be replaced by "startyear".

=o= The validator complains if it doesn't find a "flagname" struct in the header. AFAIK there's no difference between an empty flagname struct and none at all.

=o= The validator complains about NAT not being listed in the playable countries list.

=o= The validator doesn't seem to recognise AI structs directly written in scenario files and inc files, i.e. something like this:

country = {
	tag = MUS
	ai = {
		expansion = 0
		tradingpost = 100
		neighbour = 100
		enemies = -100
		traders = 100
		monopoly = 0
		war = -400.0
		ferocity = no
		sreligion = no
		naval = 0.0
		galleys = no
	}
	...
}

AFAIK this has been possible and done since EU2.

=o= The validator doesn't recognise "failure" keys in cot structs.

=o= Also, looking at the code I see the validator checks whether the total number of merchants in a cot is <=30, when it should be <=20. Furthermore the validator doesn't check whether a merchant's tag country exists at all.

=o= The validator thinks relation values have to be integers, which they don't. Same with army/navy numbers.

=o= The validator doesn't recognise the short forms art, inf, cav.



I've corrected most of the issues above (the ones marked with =o=) myself and uploaded the updated files here: happy little link

Further changes: Made all keys in AI files optional if debug mode. Added a few missing keys and created some types (FlagsList, ContinentList, RegionList, AreaList). You should probably check all files against yours with Winmerge or similar differencing tools to make sure I didn't do anything wrong.

BTW I think you should introduce a new type, an integer in float format (like 2.000), because that's the format the game stores most integers in. Or FTG needs to stop saving integers as floats, which would also save space... a bit.



EDIT: Am I correct that "\data\ftg\scenarios.pdox" isn't used anymore and has been superseded by the files in "\data\ftg\scenarios\"?



EDIT2: A few more fixes...

=o= Added a lot of keys to country definition such as stabilitypercentage, landmaintenance, estimation, basevp, loan, elector, ...
I saw you had some of those implemented as untyped childs, which I found very strange. I assume you wanted to include them at a later point and didn't get to it yet. Anyway, I killed these entries.

=o= Units now can have more than one leader assigned.

=o= Provinces in scenarios can now be land or sea provinces, since the "winter" and "storm" bools are valid for sea provinces. You should probably update the code to differentiate between land and sea though.

=o= Updated militaryalliances, wars, etc.

=o= Set geography and colour names to be case-insensitive (only in debug mode).

=o= Event actions can now have more than one "capital" command (only in debug mode).

=o= Added "hre" and "storm" keys to province definition.

=o= Added "running" and "datecompleted" to the city clause.

=o= Added "port" for navies.

=o= Set "natives" key in province clause to integer and added "tolerance" and "ferocity".


The happy little link above has been updated.

MichaelM, can you comment on what the "rank" key in monarch structs is for? It seems the game always sets it to 100 on saving.
Also, the game saves victory points twice, like this:

	vp = 122.790
	vp = 0.000

The second vaue is always zero. I doubt this is intended.
And is the "elector" parameter for countries obsolete? It's not in any Vanilla or AGCEEP files, but if you add it to a country in a scenario the game will pass it on to the savegame.
Is there still a limit of 30 COTs?



EDIT3: Oh and I think I also added something to the event/command files... triggers in actions and other things. You should really check it in Winmerge...



EDIT4: Is it possible that there are exactly three instances in exactly two files in exactly one game (FTG) where "strictness" makes any kind of difference regarding thrown exceptions and such?

 

[MichaelM]

--- The validator complains about multiple instances of country tags in the revolt.txt. I remember how a few years ago I successfully used a technique which involved multiple country tags with different dates to achieve a revolting nation with different revolting provinces depending on the year of the revolt. I'm not sure whether that was EU2, CK, or HoI2 though. MichaelM, could you comment on the possibility of this method in FTG? Because if that's a valid method, then I suggest checking the date spans of the respective revolter countries before throwing an exception.

No, FTG only allows one entry per country. But that is an interesting idea and could prove useful in cases like PRU (in vanilla).

--- Please include a validation that technology levels in the scenario file are within the bounds of said technology as defined in \db\technologies.

There is also a bug here, in that the game has an off-by-one error when it attempts to cap such an entry at the max tech level. Otherwise this wouldn't be much of a problem.

MichaelM, can you comment on what the "rank" key in monarch structs is for? It seems the game always sets it to 100 on saving.

I suppose it's left over from EU1. I know this gets saved, but I guess I never thought it was worth fixing.

Also, the game saves victory points twice, like this:

	vp = 122.790
	vp = 0.000

The second vaue is always zero. I doubt this is intended.

That's a side effect of removing the player rank system; the second one keeps track of what the score was last month so the game knows when to promote or demote the player. Now that system has been removed so it never gets updated.

And is the "elector" parameter for countries obsolete? It's not in any Vanilla or AGCEEP files, but if you add it to a country in a scenario the game will pass it on to the savegame.

It defaults to the value found in countries.txt, but can be overridden via scenario files or events.

Is there still a limit of 30 COTs?

The limit is currently *checks code* 69, but new COTs won't be created unless the total falls below (num-owned-provinces/provinces-per-COT). Provinces-per-COT is defined as 29 because no one has reworked the class to use a list instead of an array. It's now on my to-do list.


EDIT: @Jamie550: I just crashed the validator while attempting to validate WATKABAOI's 1492 scenario. Apparently it's somewhere in the COT definitions, though I don't know exactly what. The stack trace is:

ec) EXCEPTION CONTEXT
ec) Validation Task: Scenarios
Validation Key: Scenarios
Validation Type: Audax.Clausewitz.Validator.TypeExtension.DeclarativeValidatorJob
ec) Validating [document] with constraint [EegFile] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating element with constraint [EegFile] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating constraint [Inherits (IncFile)] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating constraint [literal "cot"] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating element with constraint [CotDefinition] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating element with constraint [CotDefinition] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating constraint [literal "merchant"] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating element with constraint [MerchantDef] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating element with constraint [MerchantDef] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating constraint [literal "level"] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating element with constraint [CotLevel] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Validating element with constraint [CotLevel] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 
ec) Running on element constraint [LocalOp] @ <mod>\Scenarios\1492 - The Age of Exploration.eeg ! cot[27]\merchant\level: 

dec) DEFAULT EXCEPTION CONTEXT


ERROR
System.FormatException: Input string was not in a correct format.
   at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
   at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
   at Audax.Clausewitz.Validator.TypeCore.Locals.LocalInt.OpIntIncrement.Run(Context c, String localName)
   at Audax.Clausewitz.Validator.TypeCore.Locals.LocalOpConstraint.Audax.Clausewitz.Validator.TypeCore.ICElementRunnable.InternalRunOnElement(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.RunOnElement(ICElementRunnable constraint, Context c)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ComplexConstraintWorker.ValidateInstanceAsBase(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ComplexConstraintWorker.ElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.Audax.Clausewitz.Validator.TypeCore.ICElementJudge.InternalElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c)
   at Audax.Clausewitz.Validator.TypeCore.PlaceholderConstraint.Audax.Clausewitz.Validator.TypeCore.ICElementJudge.InternalElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c, ElementNode node)
   at Audax.Clausewitz.Validator.TypeCore.NewChildConstraintSystem.ChildConstraint.Audax.Clausewitz.Validator.TypeCore.ICChildJudge.InternalValidate(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.Validate(ICChildJudge constraint, Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ComplexConstraintWorker.ValidateInstanceAsBase(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ComplexConstraintWorker.ElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.Audax.Clausewitz.Validator.TypeCore.ICElementJudge.InternalElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c)
   at Audax.Clausewitz.Validator.TypeCore.PlaceholderConstraint.Audax.Clausewitz.Validator.TypeCore.ICElementJudge.InternalElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c, ElementNode node)
   at Audax.Clausewitz.Validator.TypeCore.NewChildConstraintSystem.ChildConstraint.Audax.Clausewitz.Validator.TypeCore.ICChildJudge.InternalValidate(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.Validate(ICChildJudge constraint, Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ComplexConstraintWorker.ValidateInstanceAsBase(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ComplexConstraintWorker.ElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.Audax.Clausewitz.Validator.TypeCore.ICElementJudge.InternalElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c)
   at Audax.Clausewitz.Validator.TypeCore.PlaceholderConstraint.Audax.Clausewitz.Validator.TypeCore.ICElementJudge.InternalElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c, ElementNode node)
   at Audax.Clausewitz.Validator.TypeCore.NewChildConstraintSystem.ChildConstraint.Audax.Clausewitz.Validator.TypeCore.ICChildJudge.InternalValidate(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.Validate(ICChildJudge constraint, Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ComplexConstraintWorker.ValidateInstanceAsBase(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ValidateInstanceAsBase(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.InheritanceConstraint.Audax.Clausewitz.Validator.TypeCore.ICChildJudge.InternalValidate(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.Validate(ICChildJudge constraint, Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ComplexConstraintWorker.ValidateInstanceAsBase(Context c, NodeFlags flags)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.ComplexConstraintWorker.ElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ComplexConstraint.Audax.Clausewitz.Validator.TypeCore.ICElementJudge.InternalElementValidate(Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.ElementValidate(ICElementJudge constraint, Context c)
   at Audax.Clausewitz.Validator.TypeCore.ConstraintExtensions.DocumentValidate(ICElementJudge constraint, Context c, Document doc)
   at Audax.Clausewitz.Validator.TypeExtension.DeclarativeValidatorJob.DeclarativeValidatorWorker.Validate(Context c)
   at Audax.Clausewitz.Validator.TypeExtension.DeclarativeValidatorJob.ValidateOverride()
   at Audax.Clausewitz.Validator.ValidatorJobBase.Validate()
   at Audax.Clausewitz.Validator.ValidatorJob.Validate()
   at Audax.Clausewitz.Validator.GameValidator.Validate(Action onEachValidatorCompleteAction)

 

[Andrei Gijgorev]

That crash is due to merchants' "level" parameter written in float format (3.000), so the validator doesn't recognise them as integers. I replaced them with proper integers in my updated Watkabaoi scenarios so the validator doesn't crash. It's why I proposed the new type for floats with integer values btw.

It defaults to the value found in countries.txt, but can be overridden via scenario files or events.

I assume the "elector" parameter can only have the values 0 and 1 then, which is what I added to the code.

 

[MichaelM]

I assume the "elector" parameter can only have the values 0 and 1 then, which is what I added to the code.

No, it can have any positive value. I believe in AGCEEP they give all Germanic minors one vote (just to make them all worth something) and then give the actual electors two votes.

However, values larger than 2 are probably rare enough that a warning would be appropriate.

 

[Jamie550]

Andrei Gijgorev: Wow That is one of the most extensive examination of a game's validation files and one of the largest contributions to a game's validation I've seen. Very impressive, and thank you for all of the improvement's you've made. They're being integrated into the Validator now.

AI files
As all entries are optional, I'm not sure why they are declared as Single unless in debug. Also, AI flags seem to be in the form of flags = { [name] = yes/no }, not just a list of flags. Especially in the conquer and garrison clauses, many NonNegDbl were changed to Double - so they can take on negative values as well?

Commands
According to http://forum.paradoxplaza.com/forum/showthread.php?t=459110, "cedeprovince" is the proper command, and "secedeprovince" is deprecated.

Scenarios
- Out of curiosity, what does the 'failure' clause in a CoT do? Also, what does the 'value' clause do?
- Is it useful to have an startdate entry in an 'attackers' or 'defenders' clause?
- For province definitions, what entries can sea provinces have? Is it the case that a sea province may have storms or winters, but nothing else, while land provinces may have anything but storms or winters?
- Yes, Scenarios.pdox has been superseded by the Scenarios folder.

EDIT4: Is it possible that there are exactly three instances in exactly two files in exactly one game (FTG) where "strictness" makes any kind of difference regarding thrown exceptions and such?

I don't understand.

--- When starting the program on my system (WinXP pro) the window is out of the bounds of my desktop. It may have something to do with me using large font sizes, not sure. When I change the size of the program window the top bar appears and I can move the window around, so it's not a big problem. Just wanted to mention it...

Hmm, this will be investigated.

--- AFAIK there's a limit on ID numbers, some 7 digit number IIRC. Using an ID higher than this number can lead to a CTD on loading, or the game complaining about attempting to create a monarch/leader/whatever with an already used ID. I don't know the exact number myself, but I'm sure someone with access to the sourcecode can provide it. Since the error message given by the game is incorrect, I think it's important to include this check.

--- Please include a validation that technology levels in the scenario file are within the bounds of said technology as defined in \db\technologies.

BTW I think you should introduce a new type, an integer in float format (like 2.000), because that's the format the game stores most integers in. Or FTG needs to stop saving integers as floats, which would also save space... a bit.

These, too, will be considered. MichaelM, is there a certain upper limit for ID numbers?

The problem with WATKABAOI will also be investigated.

So far, the changelog is:

Commands: In debug mode, multiple instances of the "capital" command will be accepted.
Countries: In debug mode, political map color and leader name tag are case-insensitive.
Cultures: In debug mode, cultural map color is case-insensitive.
Events: Actions now accept triggers in their roots.
Geography: In debug mode, Region and Area are now case-insensitive.
Scenarios: NAT has been added to the list of valid tags in a scenario.
Scenarios: Added the 'sleepevent' clause in INC files.
Scenarios: The 'flagname' clause is now optional in the scenario header.
Scenarios: Added option 'absolutestartyear' clause in the scenario header.
Scenarios: In the scenario header, 'startyear' and 'startdate' are now optional, but at least one must be included.
Scenarios: Added optional 'failure' clause in a CoT definition.
Scenarios: The maximum number of merchants in a CoT has been corrected from 30 to 20.
Scenarios: Countries with merchants in a CoT now have their tags checked for existence.
Scenarios: Added an optional 'value' clause to CoT merchant definitions.
Scenarios: Added an optional 'startdate' clause to alliance declarations.
Scenarios: Added optional 'attackerscore' and 'defenderscore' clauses to war declarations.
Scenarios: Province definition native clauses will now accept any non-negative integer.
Scenarios: To province definitions, added optional 'tolerance', 'ferocity', and 'hre' clauses.
Scenarios: There can now be sea province definitions, with clauses of 'winter' or 'storm'.
Scenarios: The limit on CoT numbers has been changed from 30 to 69.
Scenarios: In debug mode, the 'treasury' clause has been made optional.
Scenarios: In a country definition, added 'elector', 'estimation', 'lastbankruptcy', 'vp', 'basevp', ledger', 'loan', 'landmaintenance', 'navalmaintenance', 'navalpercentage', 'landpercentage', 'tepercentage', 'pepercentage', 'stabilitypercentage', 'diplomaticdates', 'historicalmonarch', 'revoltrisk', 'combatlosses', attritionlosses', disbandlosses', 'warexhaustion', 'manpower', and 'treasurypercentage' clauses.
Scenarios: In a country definition, the AI clause may now hold an inline block of instructions, as well as point to a file.
Scenarios: In the city clause, added 'culture', 'running', and 'datecompleted'.
Scenarios: Added optional 'guarenteed' clause to country relations.
Scenarios: 'value' clause of country relations is now a double.
Scenarios: The 'badboy' clause of a country definition now accepts any non-negative double.
Scenarios: Army and navy numbers in units are now non-negative doubles.
Scenarios: In unit clauses, 'inf', 'art', and 'cav' are now accepted.
Scenarios: In unit clauses, added 'movetime' and 'movement' clauses.
Scenarios: Units can now have multiple leaders assigned.
Scenarios: For navies, added an optional 'port' clause.

 

[Andrei Gijgorev]

AI files
As all entries are optional, I'm not sure why they are declared as Single unless in debug.

The problem is that unlike in HoI2, a country can have only one assigned AI file, so any parameter that isn't in that file will take default values which may not be wanted.
Although if you support me over at the 1.3 wishlist thread this may change.

Also, AI flags seem to be in the form of flags = { [name] = yes/no }, not just a list of flags.

Ah yes you're right about that, sorry.

Especially in the conquer and garrison clauses, many NonNegDbl were changed to Double - so they can take on negative values as well?

The version that I downloaded (v0.8, which I think is the newest one available two days ago) has all of these values as doubles, and I didn't change anything there. You're right though, they should be NonNegDbl.

Commands
According to http://forum.paradoxplaza.com/forum/showthread.php?t=459110, "cedeprovince" is the proper command, and "secedeprovince" is deprecated.

Sorry, that was a major screwup. I replaced "secede" with "cede" in all Watkabaoi event files, and it seems I accidently had the commands file open as well at that time so it got replaced there too... twice in that line actually...

Scenarios
- Out of curiosity, what does the 'failure' clause in a CoT do? Also, what does the 'value' clause do?

"failure" increases over time if the COT lacks competition of merchants. If the value is high enough (100 I think) the COT will disappear.
"value" is the last month's income of that country's merchants AFAIK.

- Is it useful to have an startdate entry in an 'attackers' or 'defenders' clause?

It exists in savegames, so I don't think the validator should complain about it.

- For province definitions, what entries can sea provinces have? Is it the case that a sea province may have storms or winters, but nothing else, while land provinces may have anything but storms or winters?

Yes, except that land provinces may have winter as well.

I don't understand.

Well, your validator has options like "debug" <-> "end user" and "strict" <-> "loose". Now while there are a lot of instances where the program performs differently in debug mode, I only found three instances where the "strict" bool was checked, all in FTG files.
If strictness is indeed largely irrelevant, I suggest replacing it with a "case sensitive" option.

BTW I recently learned that the "dateCompleted" key is valid in a monarch clause. If it exists, there must also be at least one of "oDIP", "oMIL" and "oADM", which work the same as DIP, MIL, ADM except for that they can take negative numbers as well.
What it does is altering the monarch's stats for a certain period of time.
I'm not sure if there can be more than one "dateCompleted" key, and subsequently other "oDIP" or whatever keys.

MichaelM verified in the Watkabaoi thread that unit IDs have a maximum of 4,294,967,295 (uInt32) while personally I'm pretty sure that monarch and leader IDs are uInt16 (max 65535).
MichaelM, is it possible that ID types up to 9423 indicate uInt16 IDs, while 9424 and above indicate uInt32 IDs?


EDIT: I see now that on page two there's a v0.8.2 beta. All my changes are based on v0.8, so you'll need to reimplemented anything from the beta if you haven't already.

 

[Andrei Gijgorev]

I started working on a "Provinces.pdox" file to validate \db\map\provinces.txt; however I wasn't able to figure out how to elegantly make the type of an entry (Single/Optional) depend upon other entries of the same struct, so I'll leave that part to you. Here's a hopefully complete list of which kind of provinces may have which parameters:

• "id" -> Single. Unique Int. Must range from 0 to 2018.
  
  
• "name" -> Single. String.
  
  
• "continent" -> Single (land provinces), Optional (naval and river provinces). Valid continent name string.
  
  
• "region" -> Single if province has "continent" key. Valid region name string.
  
  
• "area" -> Single if province has "region" key. Valid area name string.
  
  
• "type" -> Single, land provinces. Either "inland" or "coastal".
  
  
• "sea_adjacency" -> Single, coastal provinces. Int, must be the ID of a naval province.
  
  
• "terrain" -> Single. Very crucial key. Terrain types are defined in \db\map\terrains.txt. For example, a SeaProvince is NOT a province with "terrain = sea", but rather a provicne with "terrain = x" where x in terrains.txt has the parameter "type = naval". Same for rivers with "type = river". Furthermore, "terra_incognita" is an additional hardcoded terrain type. The types "terra_incognita" and "river" are special in that these provinces should only have "id", "name", and "terrain" keys in the province clause. ID 0 must have terrain "terra_incognita".
  
  
• "size_modifier" -> Optional, land and naval provinces. Double.
  
  
• "climate" -> Optional, land and naval provinces. However I suggest setting it to Single for land provinces at least if not in debug mode. Can have the following (hardcoded) values: "desertic" "tropical" "temperate" "ncontinental" "scontinental" "tundra" "arctic".
  
  
• "ice" -> Optional, naval provinces. Bool.
  
  
• "storm" -> Optional, naval provinces. Bool.
  
  
• "religion" -> Single, land provinces. Religion as defined in the religions db file.
  
  
• "culture" -> Single, land provinces. Culture as defined in the cultures db file.
  
  
• "manpower" -> Single, land provinces. NonNegDbl.
  
  
• "income" -> Single, land provinces. NonNegDbl.
  
  
• "goods" -> Single, land provinces. Goods as defined in the goods db file. Note that "nothing" is a valid hardcoded goods type.
  
  
• "value" -> Single, only land provinces which produce goods that have the "inflation = yes" parameter. NonNegDbl.
  
  
• "city_name" -> Single, land provinces. String that should point to an existing csv entry in the localisation folder.
  
  
• "cot_modifier" -> Optional, land provinces. Int.
  
  
• "colonization_difficulty" -> Optional, land provinces. NonNegInt.
  
  
• "natives" -> Optional, land provinces. Natives Clause:
  • "combat" -> Optional. NonNegInt.
  • "ferocity" -> Optional. NonNegDbl.
  • "efficiency" -> Optional. NonNegDbl.
  • "tp_negotiation" -> Optional. NonNegInt.
  • "tolerance" -> Optional. NonNegInt.
  
  
• "gfx" -> Single (land provinces), Optional (naval provinces). Gfx Clause:
  • "army" -> Single (land), Optional (naval). GfxCoords.
  • "city" -> Single, land. GfxCoords.
  • "port" -> Single, coastal. GfxCoords.
  • "manufactory" -> Single, land. GfxCoords.
  • "center" -> Optional. GfxCoords.
  • "background" -> Optional. GfxCoords.
  • "terrain1" -> Optional, land (naval too if debug). GfxCoordsTerrain.
  • "terrain2" -> Same as "terrain1".
  • "terrain3" -> Same as "terrain1".
  • "terrain4" -> Same as "terrain1".
  • "river" -> Optional, land. List of up to three unique river prov IDs.
  
  
• "history" -> Optional, land provinces. Untyped since not yet implemented.

The code for GfxCoords and GfxCoordsTerrain:

(Type GfxCoords) = {
	Single = { Left = "x" Right = NonNegInt }
	Single = { Left = "y" Right = NonNegInt }
}

(Type GfxCoordsTerrain) = {
	Inherits = GfxCoords
	Optional = { Left = "variant" Right = NonNegInt }
}

All entries inside the natives clause should be (Optional If = Debug) / (Single IfNot = Debug).


I'm looking forward to the release of the next version of this wonderful program!

 

[Jamie550]

First, I'm sorry for taking so long to reply - I was caught up in something, and unfortunately soon will be again, as I am moving in less than a week. But changes have been made, and soon (within the next few days), another beta version will be released. I've tried to incorporate everything, Andrei Gijgorev, but most likely, my merging was imperfect, and there was some left out.

As for Strict, Loose, Debug, and End User:

When you are trying to validate a mod, you should always use "EndUser" mode. "Debug" mode is only for developing validation for a game. The purpose of debug mode is to hide errors that are very numerous, so that someone developing validation can ignore the errors that come up a thousand times, and see if there are other problems with validation.

Now, as for Loose vs Strict - Strict will find errors that, while not technically errors, make life difficult for the future. Such as the "special" clause for leaders - it does nothing, and it's harmless to keep, but it's also redundant and worthless. Hence, in Strict mode, the Validator will warn about "special", while in Loose mode, no warning will be made.

In summary: Use EndUser mode unless you are developing validation. Use strict if you want your files to be as good as possible.

 

[Andrei Gijgorev]

When you are trying to validate a mod, you should always use "EndUser" mode. "Debug" mode is only for developing validation for a game.

It was rather difficult for me during the last few days to do one without doing the other as well.

Now, as for Loose vs Strict - Strict will find errors that, while not technically errors, make life difficult for the future. Such as the "special" clause for leaders - it does nothing, and it's harmless to keep, but it's also redundant and worthless. Hence, in Strict mode, the Validator will warn about "special", while in Loose mode, no warning will be made.

I'm not questioning the purpose of Strict vs Loose. But since you already have that distinction, perhaps you should consider extending the concept to more files. I mean, for anyone who validates anything except for the FtG leaders and monarchs, Strict vs Loose currently doesn't make any kind of difference at all.
Or am I missing something important here?

"id" -> Single. Unique Int. Must range from 0 to 2018.

I think it's 2019, not 2018. Sorry.

 

[Jamie550]

I'm not questioning the purpose of Strict vs Loose. But since you already have that distinction, perhaps you should consider extending the concept to more files. I mean, for anyone who validates anything except for the FtG leaders and monarchs, Strict vs Loose currently doesn't make any kind of difference at all.
Or am I missing something important here?

Ah, I see. As of now there indeed are very few differences, but it doesn't seem like there's much to add only for Strict (though I added two in the latest round of changes).

I think it's 2019, not 2018. Sorry.

Got it, will fix. Also, the map folder should now have all required validation - though in some places imperfect.

 

[Andrei Gijgorev]

Well if I understand you correctly, "Loose" omits all errors that definitely won't cause the game to crash or otherwise behave unwantedly.
If that is the case, I think there are a lot of instances where this could be sensibly implemented, especially case sensitive types (AFAIK there is not a single case sensitive parameter or value in any P'dox game) and keys that are overriden with default values if they don't exist, which there are a lot of. Also it would be nice if the Strict vs Loose option would be individually adjustable for any of the validation checkboxes. Not that it makes much of a difference at the moment.

Another suggestion: you should check ID uniqueness only within a scenario and its associated files - note that "monarchs_tag.txt" will not be loaded by the game if country TAG has a "monarchtable" clause in the scenario, so even if there are duplicate IDs in these two files, it shouldn't be considered an error (same with leaders).
Duplicate IDs in different scenarios have no effect either. Currently I have to move all but one scenario out of the scenario folder and uncheck the "monarchs" and "leaders" checkboxes or otherwise I get spammed by the the hordes of duplicate IDs.

 

[Jamie550]

Well if I understand you correctly, "Loose" omits all errors that definitely won't cause the game to crash or otherwise behave unwantedly.
If that is the case, I think there are a lot of instances where this could be sensibly implemented, especially case sensitive types (AFAIK there is not a single case sensitive parameter or value in any P'dox game) and keys that are overriden with default values if they don't exist, which there are a lot of. Also it would be nice if the Strict vs Loose option would be individually adjustable for any of the validation checkboxes. Not that it makes much of a difference at the moment.

Another suggestion: you should check ID uniqueness only within a scenario and its associated files - note that "monarchs_tag.txt" will not be loaded by the game if country TAG has a "monarchtable" clause in the scenario, so even if there are duplicate IDs in these two files, it shouldn't be considered an error (same with leaders).
Duplicate IDs in different scenarios have no effect either. Currently I have to move all but one scenario out of the scenario folder and uncheck the "monarchs" and "leaders" checkboxes or otherwise I get spammed by the the hordes of duplicate IDs.

The duplicate ID problem has already been fixed, though imperfectly - monarchs/leaders in the db folder are checked for unique IDs, then for each scenario, monarch/leader IDs are checked. Though this will not find the problem where country AAA in the db folder and country BBB in the scenario folder have the same ID, it will at least prevent all false positives.

Regarding Loose vs Strict, certainly, much could be made optional, but IMO the Validator should conform to the "standard" dialect, where most things are considered case-sensitive, and most clauses are included even if they match their default values. Only rarely should "strict" and "loose" have differences, as it's good maintenance practice to disallow "hISToriCAlmonarcH", even if technically valid.

 

[Andrei Gijgorev]

True, but shouldn't at least "String" and "string" be allowed in all cases?
And isn't the whole point of "Loose" that you don't have to match "good maintenance practice"? It's not like there exists a style guide for P'dox modding. And sometimes you may not be validating your own files, but rather trying to find a CTD error in a mod you downloaded which is currently lacking support. In that case you may not want to be referenced to a hundred instances of "OMG AquaMarine is not a valid color!", but rather find the actual error as quickly as possible.

I'm only showing you aspects you may have not thought of yet, not trying to convince you. It's your program after all.

 

[Jamie550]

True, but shouldn't at least "String" and "string" be allowed in all cases?
And isn't the whole point of "Loose" that you don't have to match "good maintenance practice"? It's not like there exists a style guide for P'dox modding. And sometimes you may not be validating your own files, but rather trying to find a CTD error in a mod you downloaded which is currently lacking support. In that case you may not want to be referenced to a hundred instances of "OMG AquaMarine is not a valid color!", but rather find the actual error as quickly as possible.

I'm only showing you aspects you may have not thought of yet, not trying to convince you. It's your program after all.

Hmm, I see your point of needing to find that CtD. The problem is that usually, people should be shown as many errors as reasonable, and "Loose" is already established as the default. Perhaps another mode, "BareBones", which would ignore anything that's shouldn't cause the game to fail to load. In this case, the user could choose "Strict", "Loose", or "BareBones". In regards to casing, it would be easy to change the Validator to ignore casing when the "BareBones" flag is on.

If so, conditional clauses would become more difficult to write. As a help, the Validator could set flags Strict, BareBones, NotStrict, and/or NotBareBones (though the latter two could be replaced with IfNot), depending on the option the user selected. Also, the If clause would probably be much more used, as otherwise it could be very confusing. i.e. instead of

(Optional If = "BareBones") = { Left = "aaa" ... }
(Single IfNot = "BareBones") = { Left = "aaa" ... }
(Optional If = "BareBones") = { Left = "bbb" ... }
(Single IfNot = "BareBones") = { Left = "bbb" ... }
(Optional If = "BareBones") = { Left = "ccc" ... }
(Single IfNot = "BareBones") = { Left = "ccc" ... }

, it would be written as

(If If = BareBones) = {
Optional = { Left = "aaa" ... }
Optional = { Left = "bbb" ... }
Optional = { Left = "ccc" ... }
}
(If IfNot = BareBones) = {
Single = { Left = "aaa" ... }
Single = { Left = "bbb" ... }
Single = { Left = "ccc" ... }
}

Do you think that the third mode would be an acceptable method?

 

[Jamie550]

v0.8.3 beta is available for download

--> Download Validator v0.8.3 beta <--

As this is a beta version, it has not undergone the extensive testing of a full release. However, reasonable attempts to limit the number and severity of bugs have been made.

NOTE: This version requires the .NET Framework 4.0 Client Profile
Download the Framework here: http://www.microsoft.com/downloads/details.aspx?FamilyID=5765d7a8-7722-4888-a970-ac39b33fd8ab



Changelog

Current Version: 0.8.3 beta

v0.8.3 beta (released 2010 Aug 19)
General
Fixed a bug with node selection - when the target of a CriteriaYielder has no results, there will be no crash.
When the target of a CriteriaYielder has multiple results or no results, an error will be reported.
Added delayed corrections suggestions to Type Mismatch errors.
Some Victoria II support added.
Internals
Script node errors will display the location of the problem.
CSV Validator: New entry "AllowExtraColumns", which, when set to true, will allow extra columns without complaint.
Added an IconRefConstraint to package Core.
To ValidateFileConstraint, added hybrid Format attribute.
To ValidateFileConstraint, fixed bug where ConditionalName, as a named attribute, would not be recognized.
Created Constraint WithDicEnum.
Modified task Script to support DicEnum creation.
Improved flexibility and performance of node selection.
Locals: Int.Increment now accepts integers in the form X.000 (i.e. with a zeroed decimal).
Ftg
Events: "action" can be used in place of "action_[abcd]".
Map: Added adj-defs validation.
Map: Added geography validation.
Map: Added special province validation.
Map: Added province validation.
Countries: In the "new_colony" field, any tag is accepted now.
Commands: In debug mode, multiple instances of the "capital" command will be accepted.
Countries: In debug mode, political map color and leader name tag are case-insensitive.
Cultures: In debug mode, cultural map color is case-insensitive.
Events: Actions now accept triggers in their roots.
Geography: In debug mode, Region and Area are now case-insensitive.
Scenarios: NAT has been added to the list of valid tags in a scenario.
Scenarios: Added the 'sleepevent' clause in INC files.
Scenarios: The 'flagname' clause is now optional in the scenario header.
Scenarios: Added option 'absolutestartyear' clause in the scenario header.
Scenarios: In the scenario header, 'startyear' and 'startdate' are now optional, but at least one must be included.
Scenarios: Added optional 'failure' clause in a CoT definition.
Scenarios: The maximum number of merchants in a CoT has been corrected from 30 to 20.
Scenarios: Countries with merchants in a CoT now have their tags checked for existence.
Scenarios: Added an optional 'value' clause to CoT merchant definitions.
Scenarios: Added an optional 'startdate' clause to alliance declarations.
Scenarios: Added optional 'attackerscore' and 'defenderscore' clauses to war declarations.
Scenarios: Province definition native clauses will now accept any non-negative integer.
Scenarios: To province definitions, added optional 'tolerance', 'ferocity', and 'hre' clauses.
Scenarios: There can now be sea province definitions, with clauses of 'winter' or 'storm'.
Scenarios: The limit on CoT numbers has been changed from 30 to 69.
Scenarios: In debug mode, the 'treasury' clause has been made optional.
Scenarios: In a country definition, added 'elector', 'estimation', 'lastbankruptcy', 'vp', 'basevp', ledger', 'loan', 'landmaintenance', 'navalmaintenance', 'navalpercentage', 'landpercentage', 'tepercentage', 'pepercentage', 'stabilitypercentage', 'diplomaticdates', 'historicalmonarch', 'revoltrisk', 'combatlosses', attritionlosses', disbandlosses', 'warexhaustion', 'manpower', and 'treasurypercentage' clauses.
Scenarios: In a country definition, the AI clause may now hold an inline block of instructions, as well as point to a file.
Scenarios: In the city clause, added 'culture', 'running', and 'datecompleted'.
Scenarios: Added optional 'guarenteed' clause to country relations.
Scenarios: 'value' clause of country relations is now a double.
Scenarios: The 'badboy' clause of a country definition now accepts any non-negative double.
Scenarios: Army and navy numbers in units are now non-negative doubles.
Scenarios: In unit clauses, 'inf', 'art', and 'cav' are now accepted.
Scenarios: In unit clauses, added 'movetime' and 'movement' clauses.
Scenarios: Units can now have multiple leaders assigned.
Scenarios: For navies, added an optional 'port' clause.
Victoria II
Added bookmarks validation.
Added localization validation.
Added country colors validation.
Added religion validation.
Added country validation.
Added traits validation.
Added culture validation.
Added goods validation.
Eu3
GraphicalCultureType: The graphical culture type "" has been removed. Parsing of the file slightly improved.

 

[Andrei Gijgorev]

BareBones sounds great, but code looks awful. I see you have defined the template types in the binaries, so why not simply add a template that acts like Optional in BareBones mode and Single otherwise. Call it Conventional or something and you spare yourself a lot of redundancy and possible error sources.

Thank you for uploading the beta, I'll be sure to thoroughly test it the next days.

 

[Garbon]

Great tool! A few comments though:

I'm getting a fair amount of false positives with less common functions.

Events: actions can now have triggers so this shouldn't pop up
Scenario files: country definitions can support bits like flagname, badboy, revoltrisk and wardates.

 

[Jamie550]

Great tool! A few comments though:

I'm getting a fair amount of false positives with less common functions.

Events: actions can now have triggers so this shouldn't pop up
Scenario files: country definitions can support bits like flagname, badboy, revoltrisk and wardates.

The download available three posts above this one should solve all of these problems.

BareBones sounds great, but code looks awful. I see you have defined the template types in the binaries, so why not simply add a template that acts like Optional in BareBones mode and Single otherwise. Call it Conventional or something and you spare yourself a lot of redundancy and possible error sources.

Thank you for uploading the beta, I'll be sure to thoroughly test it the next days.

Hmm, I don't much like my hardcoding of templates - hopefully they can be moved back to the text files in the future. I'm not sure how BareBones is any more messy than Debug, though, as they are essentially the same thing (though perhaps BareBones should be abbreviated to Bare or BB or B or similar).

 

[Andrei Gijgorev]

It's not the "BareBones" that I dislike, it's having to use at least two lines for a simple if/else statement. Of course I wouldn't complain if you'd introduce other templates as well, like StrictOnly or EndUserOnly or whatever. Personally I would find that much more elegant than all the "If this then that, IfNot this then yonder" redundancy.
Of course this is a rather technical issue, and it's your code anyway, so...

By the way, as far as I know there is not a single list in any P'dox game that is not technically allowed to contain duplicate items. Maybe you can define PdoLists to always ignore the uniqueness parameter in BareBones mode.