• We have updated our Community Code of Conduct. Please read through the new rules for the forum that are an integral part of Paradox Interactive’s User Agreement.

EU4 - Development Diary - 19th of February 2019

Fantastic bits and where to find them
Hi everyone, I am Lorenzo aka Duplo aka The Battlepope aka The Caped Crusader and I am programmer on our beloved Europa Universalis 4. You may have seen me in the Dharma Dev Clash, struggling to spread Catholicism and occasionally getting betrayed by fellow Italians and wrecked up by Venice AI :confused:

In these last weeks we announced that 1.28.3 was the last Europa Universalis 4 release supporting 32-bit, since we are moving toward 64-bit. I thought it would be nice to give to all of you an insight of what this means, what to genuinely expect in the near future and what to definitely not expect.

There's been a lot of fuss about 64 vs 32-bit apps lately. But what is all of this about?

It is no mystery that Apple decided to deprecate 32-bit apps in MacOS 10.14 "Mojave" (2018) in order to drop the support completely in the future: they are doing an amazing job reminding us about this every time we launch Steam or our beloved Europa Universalis 4.

What may be less known is that while 32-bit apps may still run on the latest MacOS release the new development environment was stripped out of all the fancy 32-bit support, making it painful for the poor developers to even compile the game. They are trying hard to make everyone move to 64. But why?

If I got your attention so far, we can finally dive into the topic. It's going to be great.

The fellowship of the bits
When we say 64-bit, we are talking about x86-64, a CPU architecture designed by AMD of which the specifications were released in the year of our Lord 2000.

In the year 2001, the Linux kernel started supporting this new architecture, even if there were no processors available on the market yet.

In the year 2003, the very first x86-64 processor is released: the AMD64 Opteron. Several Linux distributions were already supporting x86-64.

In the year 2005, Microsoft discontinued the IA-64 (another 64-bit architecture from Intel) version of Windows XP, and released Windows XP Professional x64 Edition.

In the year 2009, Apple latest state of the of art system, Mac OS X 10.6 "Snow Leopard", was released with full support for 64 bits on the x86-64 platform. Also Windows 7 started to be loaded by default in his 64 bit version on most new computers.

In the year 2011, with Mac OS X 10.7 "Lion", Apple decided to drop the support for 32-bit CPUs, effectively starting that modernization process of their platform that last year resulted in the deprecation of 32-bit apps.

Nice. But still, why should I care?
Well, there are several reasons to actually move to 64-bit. The looming threat of deprecation on MacOS is maybe the most evident, but there is much more. The x86-64 architecture is better than its older brother, by far.

Without going too much into technical details, the advantages of going full 64-bit can be summarized in three main points:

Extended address space
64-bit applications can break the hard limit of 4 GB of RAM. By several orders of magnitude. Do you like blobbing in Europa Universalis 4? Think about how happy the CPUs would be if they could do the same with RAM.
extended-address-space.jpg


Capacity increase
64-bit registers and 64-bit enabled operands that will allow the CPU to perform trivially operations that were costly in the 32-bit architecture. Didn’t you want to find your way from Paris to Beijing in the blink of an eye?
capacity-increase.jpg


Larger number of general-purpose registers
In compute-intensive code, the 64-bit compiler will make use of these additional registers to better optimize the generated program. Russia will be able to recruit Streltsy even faster!
number-of-general-purpose-registers.jpg


There is also another, non-technical, advantage: support. While Microsoft is not as zealous as Apple in its crusade against legacy technology, they are obviously putting most of their efforts in the new - and definitely more common - architecture. Of course the 32-bit tools are still doing an honest job, but having the chance to move to the new 64-bit ones will allow us to get advantage of the new hardware features, getting less bugs and better performances.

Wow! Great!
That said, moving to 64-bits won’t magically make the game run twice as fast, nor it will make more content appear out of nothing. While the migration to the new architecture itself will have little impact on Europa Universalis 4 at the beginning, it will put the foundation for the future development, allowing us - the programmers - to gradually try to squeeze the most out of the CPUs! But important questions first...

Multithreading when?
Really? C’mon… The lack of multithreading in Europa Universalis 4 has been a common misconception for a long time.

What is multithreading about? Multithreading involves taking small pieces of logic and potentially execute them at the same time, getting advantage of the multiple cores of modern CPUs. The advantage of multithreading is that we can squeeze the most of the processor and get things done faster, because executed at the same time. The main problem of multithreading is that it’s impossible to predict when the Operating System is going to let them run, as the execution of threads is completely out of the application control. Working with multithreading is basically finding the proper balance between the potentially better performances given by running tasks at the same time and the performance loss due to the synchronization.

This tradeoff is especially noticeable in Europa Universalis 4 as the game has tons of small pieces of information to calculate, and each of them is heavily influencing the AI behavior.

There are things that can be safely executed in parallel - and indeed they are executed that way - like loading and processing data, updating independent pieces of the gamestate, cache calculations, most of AI behaviors… But there are also things that definitely cannot: pieces of gamestate dependent on other pieces to be calculated before and so on... Trying to force those interdependent operations to be executed in parallel would produce non-deterministic results - different results even with exactly the same initial state - and inconsistent data. This non-deterministic behavior can lead to the game crashing, because the data dependent for an operation might be still in an incomplete state.

A mysterious and shady voice I can hear behind me explains this problem this way:
Imagine the data as existing as a quantum wave function. Until observed we can not know what state the data is in. When one thread needs to access and change that data the wave function collapses as it's being observed. If there were an expectation of that the data would be in a specific state before the wave function collapses it will just be garbled mess and the universe crashes.
multithreading.jpg


One of the focus of us, programmers, is to try to get the best performances possible, while adding new cool features. Multithreading is already a powerful tool in our toolbox. Multithreading is a thing in Europa Universalis 4, and it’s here to stay ;)

This old, but still valid Development Diary is an example of our effort to get the best performances out of our game, and the charts show how multithreading is actually a fundamental part of this effort!

If you have any question, please write them in this thread. I'll follow the discussion and try to answer all of you!
 

LordKensis

Sergeant
84 Badges
Mar 13, 2016
74
10
  • Crusader Kings II: Jade Dragon
  • Europa Universalis IV: Third Rome
  • Tyranny: Archon Edition
  • Stellaris: Humanoids Species Pack
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Conclave
  • Stellaris
  • Stellaris: Galaxy Edition
  • Surviving Mars: First Colony Edition
  • Hearts of Iron IV: Expansion Pass
  • Hearts of Iron IV: Cadet
  • Europa Universalis IV: Rights of Man
  • Tyranny: Archon Edition
  • Pillars of Eternity
  • Hearts of Iron IV: Together for Victory
  • Steel Division: Normandy 44
  • Europa Universalis IV: Mandate of Heaven
  • PDXCON 2017 Standard Ticket holder
  • Surviving Mars
  • Hearts of Iron IV: Death or Dishonor
  • Stellaris: Synthetic Dawn
  • Tyranny - Tales from the Tiers
  • Tyranny - Bastards Wound
  • Europa Universalis IV: Cradle of Civilization
  • Age of Wonders: Planetfall
  • Stellaris: Lithoids
  • Crusader Kings II: Holy Fury
  • Europa Universalis IV: Dharma
  • Imperator: Rome - Magna Graecia
  • Hearts of Iron IV: La Resistance
  • Cities: Skylines - Parklife
  • Europa Universalis IV: Golden Century
  • Stellaris: Distant Stars
  • Age of Wonders: Planetfall - Revelations
  • Europa Universalis IV
  • Victoria 3 Sign Up
  • Stellaris: Federations
  • BATTLETECH - Digital Deluxe Edition
  • Europa Universalis IV: Rule Britannia
  • Magicka
  • Stellaris: Megacorp
  • Imperator: Rome Sign Up
  • Victoria 2
  • Imperator: Rome Deluxe Edition
  • Stellaris: Apocalypse
  • Cities: Skylines
  • Stellaris: Ancient Relics
  • Europa Universalis IV: El Dorado
This isn't a dev diary, this is how a year one CS student would explain the advantages of 64-bit architecture last year's memes.
This is so annoyng, why are you complaining, they are switching from 32 to 64 bit and to make people like you not complaining of " omg omg no 34253453245 province added to my country, this patch suck " they have explained ( very well I would say ) what they are working on and why.
 

Fekke

Nice Guy Extraordinaire
91 Badges
Mar 3, 2014
36
13
fekke.dk
  • Victoria: Revolutions
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Heir to the Throne
  • Leviathan: Warships
  • Magicka
  • Majesty 2 Collection
  • Europa Universalis IV: Res Publica
  • Crusader Kings II
  • Semper Fi
  • Victoria 2
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Europa Universalis IV: Pre-order
  • Cities: Skylines - After Dark
  • Europa Universalis IV: Cossacks
  • Europa Universalis IV: Wealth of Nations
  • Crusader Kings II: Monks and Mystics
  • Cities in Motion 2
  • Hearts of Iron IV: Field Marshal
  • Europa Universalis IV: Third Rome
  • Europa Universalis III
  • Stellaris - Path to Destruction bundle
  • Europa Universalis III: Chronicles
  • Europa Universalis III Complete
  • Divine Wind
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Hearts of Iron IV Sign-up
  • Magicka 2
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Way of Life
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Rights of Man
  • Crusader Kings II: Conclave
  • Hearts of Iron IV: Cadet
  • Stellaris
  • Stellaris: Galaxy Edition
  • Magicka: Wizard Wars Founder Wizard
  • Europa Universalis IV: El Dorado
  • Cities: Skylines
  • War of the Roses
  • Europa Universalis 4: Emperor
  • Stellaris: Leviathans Story Pack
  • The Showdown Effect
  • Stellaris: Digital Anniversary Edition
  • Europa Universalis III Complete
  • Europa Universalis III Complete
This is so annoyng, why are you complaining, they are switching from 32 to 64 bit and to make people like you not complaining of " omg omg no 34253453245 province added to my country, this patch suck " they have explained ( very well I would say ) what they are working on and why.
I am applauding the move to 64-bit architecture. This is just a bad DD. End of story.
 

Malecord

Graf
65 Badges
May 13, 2016
801
932
  • Cities in Motion 2
  • Europa Universalis IV: Third Rome
  • Crusader Kings II: Jade Dragon
  • Europa Universalis IV: Rule Britannia
  • Stellaris: Leviathans Story Pack
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Conclave
  • Cities: Skylines - Snowfall
  • Stellaris: Megacorp
  • Stellaris
  • Stellaris: Galaxy Edition
  • Crusader Kings II: Reapers Due
  • Europa Universalis IV: Rights of Man
  • Stellaris: Digital Anniversary Edition
  • Stellaris: Humanoids Species Pack
  • Cities: Skylines - Natural Disasters
  • Crusader Kings II: Monks and Mystics
  • Stellaris - Path to Destruction bundle
  • Cities: Skylines - Mass Transit
  • Europa Universalis IV: Mandate of Heaven
  • Stellaris: Apocalypse
  • Stellaris: Synthetic Dawn
  • Age of Wonders III
  • Cities: Skylines - Green Cities
  • Europa Universalis IV: Cradle of Civilization
  • Stellaris: Nemesis
  • Imperator: Rome
  • Cities: Skylines - Campus
  • Stellaris: Ancient Relics
  • Cities: Skylines Industries
  • Crusader Kings II: Holy Fury
  • Cities: Skylines - Parklife
  • Stellaris: Distant Stars
  • Europa Universalis IV: Dharma
  • Europa Universalis IV
  • Europa Universalis IV: Golden Century
  • Cities: Skylines - After Dark
  • Europa Universalis 4: Emperor
  • Stellaris: Federations
  • Stellaris: Lithoids
  • Warlock: Master of the Arcane
  • Cities: Skylines
  • Cities: Skylines Deluxe Edition
  • Europa Universalis IV: El Dorado
  • Crusader Kings II: Way of Life
  • Magicka 2
  • Europa Universalis IV: Common Sense
  • Crusader Kings II
  • Europa Universalis IV: Art of War
So it took several weeks to produce a dev diary that's explains what 64-bit means, but doesn't give any technical details relating to EU4?

He said that it won't change anything by itself and the game will run at the same speed (and possibly consume more memory).

Eventually it will lead to better performances as the game will be refactored to exploit the larger RAM and registers. I think that when they'll figure out how to do this we will have a DD explaining what they did and how they did it. But for now it's understandable that there isn't much to say.
 

steveh11

Games Player
116 Badges
Jun 9, 2001
2.410
418
www.asstr.org
  • Cities: Skylines Deluxe Edition
  • Victoria: Revolutions
  • Rome Gold
  • Semper Fi
  • Sengoku
  • Teleglitch: Die More Edition
  • Victoria 2
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Warlock: Master of the Arcane
  • Warlock 2: The Exiled
  • 500k Club
  • Cities: Skylines
  • Europa Universalis IV: Res Publica
  • Crusader Kings II: Holy Knight (pre-order)
  • Europa Universalis III: Collection
  • Europa Universalis IV: El Dorado
  • Europa Universalis IV: Pre-order
  • Crusader Kings II: Way of Life
  • Pillars of Eternity
  • Europa Universalis IV: Common Sense
  • Crusader Kings II: Horse Lords
  • Paradox Order
  • Cities: Skylines - After Dark
  • Europa Universalis IV: Cossacks
  • Victoria 3 Sign Up
  • Europa Universalis IV: Art of War
  • Crusader Kings II
  • Crusader Kings II: Charlemagne
  • Crusader Kings II: Legacy of Rome
  • Crusader Kings II: The Old Gods
  • Crusader Kings II: Rajas of India
  • Crusader Kings II: The Republic
  • Crusader Kings II: Sons of Abraham
  • Crusader Kings II: Sunset Invasion
  • Crusader Kings II: Sword of Islam
  • Europa Universalis III
  • Divine Wind
  • Europa Universalis IV
  • Ancient Space
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Call to arms event
  • For the Motherland
  • Hearts of Iron III
  • Hearts of Iron III: Their Finest Hour
  • Hearts of Iron III Collection
  • Heir to the Throne
  • Europa Universalis III Complete
  • March of the Eagles
So:- Just EU4, or will this be rolled out to the other Clausewitz games?
(Imagine the huge galaxies under 64-bit Stellaris. IMAGINE!)
 

Zwirbaum

(Formerly known as Zwireq)
103 Badges
Jun 2, 2011
4.451
2.248
34
  • Europa Universalis IV: Mare Nostrum
  • Hearts of Iron III
  • Heir to the Throne
  • Magicka
  • Sengoku
  • Victoria: Revolutions
  • Rome Gold
  • Victoria 2: A House Divided
  • Victoria 2: Heart of Darkness
  • Europa Universalis III Complete
  • Europa Universalis IV: Cossacks
  • Crusader Kings II: Conclave
  • Teleglitch: Die More Edition
  • Victoria 2
  • Europa Universalis IV: Common Sense
  • Stellaris
  • Stellaris: Galaxy Edition
  • Hearts of Iron IV Sign-up
  • Stellaris Sign-up
  • 500k Club
  • Hearts of Iron IV: Cadet
  • Europa Universalis IV: El Dorado
  • Hearts of Iron IV: Colonel
  • Pride of Nations
  • Crusader Kings II: Way of Life
  • Pillars of Eternity
  • Surviving Mars: First Colony Edition
  • Europa Universalis IV: Golden Century
  • Crusader Kings II: Horse Lords
  • Europa Universalis IV: Third Rome
  • BATTLETECH
  • Surviving Mars
  • Steel Division: Normand 44 Sign-up
  • Hearts of Iron IV: Death or Dishonor
  • Age of Wonders III
  • Crusader Kings II: Jade Dragon
  • Hearts of Iron IV: Expansion Pass
  • Stellaris: Humanoids Species Pack
  • Crusader Kings II: Holy Fury
  • Steel Division: Normandy 44
  • Europa Universalis IV
  • PDXCON 2018 "The Emperor"
  • BATTLETECH - Digital Deluxe Edition
  • Cities: Skylines - Parklife
  • Europa Universalis 4: Emperor
  • Stellaris: Distant Stars
  • Europa Universalis IV: Dharma
  • Europa Universalis III Complete
  • Shadowrun Returns
  • Shadowrun: Dragonfall
81922807.jpg


/joke I actually get most of the jargon as I did Computer Science at Uni. :)

QUESTION

What percentage of the EU4 community is still using 32bit OS?

IIRC it is ~1.1%
 

Zak Preston

Zakharia
74 Badges
Aug 16, 2014
1.668
2.159
  • Crusader Kings III
  • Knights of Honor
  • Europa Universalis IV
  • Europa Universalis IV: Call to arms event
  • Europa Universalis IV: Pre-order
  • Stellaris
  • Stellaris: Galaxy Edition
  • Stellaris Sign-up
  • Hearts of Iron IV: Cadet
  • Stellaris: Megacorp
  • Imperator: Rome Deluxe Edition
  • Imperator: Rome Sign Up
  • Mount & Blade: Warband
  • Europa Universalis III: Collection
  • Victoria 2
  • Stellaris: Galaxy Edition
  • Crusader Kings II
  • Imperator: Rome
  • War of the Roses
@Duplo DirectX-12/Vulkan API when? :rolleyes:

I've noticed that some graphical settings like shadows, trees and rivers are quite CPU intensive (which may be a harsh legacy of DX9), and EU4 doesn't utilize much of my Radeon's 290x resources at all.
 

artemis667

Field Marshal
63 Badges
Apr 30, 2002
3.427
703
  • Hearts of Iron IV: Cadet
  • Cities: Skylines
  • Crusader Kings II: Holy Knight (pre-order)
  • Europa Universalis III: Collection
  • Europa Universalis IV: El Dorado
  • Europa Universalis IV: Pre-order
  • Pillars of Eternity
  • Europa Universalis IV: Common Sense
  • Europa Universalis IV: Cossacks
  • Europa Universalis IV: Mare Nostrum
  • Stellaris
  • Stellaris: Galaxy Edition
  • Stellaris: Galaxy Edition
  • 500k Club
  • Europa Universalis IV: Rights of Man
  • Stellaris: Digital Anniversary Edition
  • Stellaris: Leviathans Story Pack
  • Stellaris - Path to Destruction bundle
  • Europa Universalis IV: Mandate of Heaven
  • Europa Universalis IV: Third Rome
  • Stellaris: Synthetic Dawn
  • Age of Wonders III
  • Europa Universalis IV: Cradle of Civilization
  • Stellaris: Humanoids Species Pack
  • Stellaris: Apocalypse
  • Stellaris: Nemesis
  • Heir to the Throne
  • Cities in Motion 2
  • Crusader Kings II
  • Deus Vult
  • Europa Universalis III
  • Europa Universalis III Complete
  • Divine Wind
  • Europa Universalis IV
  • Europa Universalis IV: Art of War
  • Europa Universalis IV: Conquest of Paradise
  • Europa Universalis IV: Wealth of Nations
  • Europa Universalis IV: Call to arms event
  • Hearts of Iron III
  • Cities in Motion
  • Europa Universalis III Complete
  • Knights of Pen and Paper +1 Edition
  • Magicka
  • Europa Universalis III Complete
  • Europa Universalis IV: Res Publica
  • Victoria: Revolutions
  • Europa Universalis: Rome
  • Semper Fi
  • Victoria 2
  • Victoria 2: A House Divided
As someone who's been in IT for nearly thirty years, done a bit of machine language programming, and understands a bit of what CPUs are doing each clock cycle and why bits are important... this DD is damn cool :)