I don't understand goods substitution

  • We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

Googane

Recruit
16 Badges
Jan 23, 2023
9
23
  • Crusader Kings II
  • Europa Universalis IV
  • Stellaris
  • Hearts of Iron IV: Cadet
  • Hearts of Iron IV: Together for Victory
  • Stellaris - Path to Destruction bundle
  • Hearts of Iron IV: Death or Dishonor
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Apocalypse
  • Stellaris: Distant Stars
  • Hearts of Iron IV: Expansion Pass
  • Hearts of Iron IV: La Resistance
  • Stellaris: Federations
  • Battle for Bosporus
  • Hearts of Iron IV: By Blood Alone
  • Hearts of Iron IV: No Step Back
I don't know if people are already aware of this but I don't get why my pops are still overpaying for things like logs, fabric, grain, etc when it's objectively cheaper to buy oil, clothes, furniture or groceries to substitute their needs. This makes it so that I still have to keep importing or creating rural buildings for stuff that I technically already have to not lose SOL which makes no sense to me.

Is this intended?
 
  • 2Like
Reactions:
If you can understand the discussion happening in this thread, you'll be fine


I've been told it's very simple really
 
  • 12Haha
  • 1Like
  • 1
Reactions:
Essentially pops only look at production and not consumption when determining what to buy. We don't know if this is going to get fixed.
 
  • 9
  • 4
Reactions:
I don't know if people are already aware of this but I don't get why my pops are still overpaying for things like logs, fabric, grain, etc when it's objectively cheaper to buy oil, clothes, furniture or groceries to substitute their needs. This makes it so that I still have to keep importing or creating rural buildings for stuff that I technically already have to not lose SOL which makes no sense to me.

Is this intended?
Making a good algorithm to make them always buy the best possible way is hard
 
  • 5
  • 2
Reactions:
The biggest problem is goods that can be used for different needs, like wood being used for both heating and crude items. If it wasn't for that then I think it would be relatively straight forward.
 
  • 2
  • 2
Reactions:
Making a good algorithm to make them always buy the best possible way is hard
Is it really? You need X amount of, say, heating. Wood grants an "W" amount of heating and oil, in turn, an "O" amount. It doesn't sound too complicated to determine the result, the worst case scenario is probably a linear programming problem.
 
  • 2
  • 2
  • 1Haha
Reactions:
In real life people substitute goods based on price, which changes frequently. Doing so in-game would require many calculations that could cause feedback loops, so the devs opted substitution to use supply instead, which doesn't change as much.
 
  • 5
  • 2
Reactions:
In real life people substitute goods based on price, which changes frequently. Doing so in-game would require many calculations that could cause feedback loops, so the devs opted substitution to use supply instead, which doesn't change as much.
This is total speculation based on absolutely nothing.
 
  • 2Like
  • 1
Reactions:
This is total speculation based on absolutely nothing.
No it isn't, the price of the goods changes depending on the goods used by population to meet their demands, thus meaning that your calculations depend on themselves, making apriori calculation of the "correct" substitution difficult and time intensive.

As there are already enough people saying the game is moving too slowly, there is only a limited budget of operation time that can be used for calulating goods substitution, and thus they are using a method that they are able to calculate apriori, meaning they only have to do it once and the result doesn't depend on the result of the calculation.

I'm not saying this is the way it should be done, as I think I've already mentioned in the other thread about goods substitution, but I can at least understand why they've done it, even if I don't agree with it.

But designing an algorithm that provides better results without requiring multiple rounds of recalculation is not easy, as can be found if you read through the other thread. Fastest option would be to calculate substitution based on (goods production - non-population demand) as that removes the circularity of the calculations, and produces at least better results than the current version which is just based on goods production, but would add more calculations as first the non-population demand needs to be calculated. This likely wouldn't cause too much slow down, but then again, I don't have access to the game code nor proper debug timing tools. If we want better price fidelity we need to also take the population demand itself into account in the calculation, and here we get to problems with priority and precedence. If one pop or need is calculated before another, they might make wildly different substitution decisions as their demand will make things different for the following calculations, and this would also mean that the amount of calculations needed for substitution would skyrocket, since you'd need to re-do most of the calculations as many times as you have population fragments demanding things. One option would be to base the calculation on the previous round of calculation, so last weeks price information, where we solve the problem of demand changing price, as that has already been calculated, so we don't need to recalculate it on the fly, but then substitution will always drag a bit behind actual optimal, and if there is no proper damping on substitution we can get a wildly oscillating system that one week uses wood for heating fully since it was cheap last week, and then the next switches over to oil since it was cheap this week, and thus making havoc on the overall market pricing.

If you can show which part of that description is "total speculation", please do so. It's based on the basic functions of the economic simulation and how demand affects prices, and the fact that there are multiple actors creating demand.

If you know of a way of calculating optimal goods substitution for all populations even in one state in one go, without referencing the results of the goods substitution, please do tell, I'm sure the other thread would appreciate the input.
 
  • 5
  • 2Like
  • 2
  • 1Haha
Reactions:
No it isn't, the price of the goods changes depending on the goods used by population to meet their demands, thus meaning that your calculations depend on themselves, making apriori calculation of the "correct" substitution difficult and time intensive.

As there are already enough people saying the game is moving too slowly, there is only a limited budget of operation time that can be used for calulating goods substitution, and thus they are using a method that they are able to calculate apriori, meaning they only have to do it once and the result doesn't depend on the result of the calculation.

I'm not saying this is the way it should be done, as I think I've already mentioned in the other thread about goods substitution, but I can at least understand why they've done it, even if I don't agree with it.

But designing an algorithm that provides better results without requiring multiple rounds of recalculation is not easy, as can be found if you read through the other thread. Fastest option would be to calculate substitution based on (goods production - non-population demand) as that removes the circularity of the calculations, and produces at least better results than the current version which is just based on goods production, but would add more calculations as first the non-population demand needs to be calculated. This likely wouldn't cause too much slow down, but then again, I don't have access to the game code nor proper debug timing tools. If we want better price fidelity we need to also take the population demand itself into account in the calculation, and here we get to problems with priority and precedence. If one pop or need is calculated before another, they might make wildly different substitution decisions as their demand will make things different for the following calculations, and this would also mean that the amount of calculations needed for substitution would skyrocket, since you'd need to re-do most of the calculations as many times as you have population fragments demanding things. One option would be to base the calculation on the previous round of calculation, so last weeks price information, where we solve the problem of demand changing price, as that has already been calculated, so we don't need to recalculate it on the fly, but then substitution will always drag a bit behind actual optimal, and if there is no proper damping on substitution we can get a wildly oscillating system that one week uses wood for heating fully since it was cheap last week, and then the next switches over to oil since it was cheap this week, and thus making havoc on the overall market pricing.

If you can show which part of that description is "total speculation", please do so. It's based on the basic functions of the economic simulation and how demand affects prices, and the fact that there are multiple actors creating demand.

If you know of a way of calculating optimal goods substitution for all populations even in one state in one go, without referencing the results of the goods substitution, please do tell, I'm sure the other thread would appreciate the input.
Yes, that would be a disastrous scenario if the devs were so foolish as to not cap rate of change in consumption, the first thing you should do.

I won't pretend to know the behind-the-scenes challenges to the process, but using the previous week's price differential to drive a minor change in consumption patterns per week would result in pops gravitating towards the optimum and quietly oscillating around it.

Substitution dragging behind the optimal is also not a bad thing - we don't respond immediately to market fluctuations. To use the fuel example, we don't replace our heating system every time the price of gas drops enough to be cheaper than electricity.
 
  • 7Like
  • 1
Reactions:
Yes, that would be a disastrous scenario if the devs were so foolish as to not cap rate of change in consumption, the first thing you should do.

I won't pretend to know the behind-the-scenes challenges to the process, but using the previous week's price differential to drive a minor change in consumption patterns per week would result in pops gravitating towards the optimum and quietly oscillating around it.

Substitution dragging behind the optimal is also not a bad thing - we don't respond immediately to market fluctuations. To use the fuel example, we don't replace our heating system every time the price of gas drops enough to be cheaper than electricity.
Yep, precisely. But still it requires them to keep a hold of the old information, which of course they have somewhere since it can be shown in the UI, but that might not be in a usable format for the pop demand calculations, again a technical issue that can certainly be surpassed.

And yeah, I fully agree that substitution dragging a bit behind optimal is a good thing, but implementing such a damper is not without some operational cost in calculation time. Hopefully it is something that can be had without slowing down the simulation speed too much, but there is going to be a cost to implementing this kind of system. And without having access to proper debug timing tools, impossible to say how much of one.
 
  • 2
Reactions:
The optimal price is the one where supply/demand is the same for all the goods being used, or as close to it as possible. If you didn't need to account for the same good being possibly used for different goods then you could look at the supply for all goods that could meet a need, the demand from non-pop sources for those goods and how much demand pops would add for those goods in total. This demand from pops could then be distributed among the goods such that the ratio of supply to demand is the same for each.

Let's say that you have 120 wood production and 100 demand for wood from non-pop sources. 50 coal production and 20 coal demand from non-pop sources and 30 oil production with 0 oil demand from non-pop sources. Let's also say that your pops want 100 wood worth of heating, with each coal counting for 1.5 wood and each oil counting for 2 wood.

This would give a set of three linear equations to be solved to find the optimal distribution:
(100 + pop demand for wood)/120 = (20 + pop demand for coal)/50
(20 + pop demand for coal)/50 = (0 + pop demand for oil)/30
pop demand for wood + 1.5 * pop demand for coal + 2 * pop demand for oil = 100

Solving these tells us that the pops should buy 8.24 wood, 25.1 coal and 27.06 oil

If they did then demand/supply would be 108.24/120 for wood, 45.1/50 for coal and 27.06/30 for oil, balancing the demand to supply ratio with each having slightly less demand than supply.

But this simple formula would not account for limited market acces or for some goods being able to fulfill multiple needs.
 
  • 1
Reactions:
In real life people substitute goods based on price, which changes frequently. Doing so in-game would require many calculations that could cause feedback loops, so the devs opted substitution to use supply instead, which doesn't change as much.
Contrary to real life the good prices in Vic3 are 100% predictable
 
  • 1Like
Reactions:
There is a whole economic theory studying good substitution, there isn't a straightforward solution.


IMHO, the current implemented solution has a problem when goods are produced in one or two order of magnitude bigger than the other goods. For example, grain vs other basic food substitutes or wood vs oil for heating needs.

To solve this I have suggested to use log base 2 of global sell orders instead of global sell orders. More detailed info in the link provided by @Sbrubbles
https://forum.paradoxplaza.com/forum/threads/when-will-goods-substitution-be-fixed.1581857/
 
  • 3
  • 1
Reactions:
Yep, precisely. But still it requires them to keep a hold of the old information, which of course they have somewhere since it can be shown in the UI, but that might not be in a usable format for the pop demand calculations, again a technical issue that can certainly be surpassed.

And yeah, I fully agree that substitution dragging a bit behind optimal is a good thing, but implementing such a damper is not without some operational cost in calculation time. Hopefully it is something that can be had without slowing down the simulation speed too much, but there is going to be a cost to implementing this kind of system. And without having access to proper debug timing tools, impossible to say how much of one.
The current price is already used elsewhere in the game and is even all over the UI. If what you're saying is true then trade wouldn't work either. Stop with this nonsense already.