[Tool] JoroDox mod making tool

[Joror]

Hi CK2 modders, this is a crosspost from This EU4 User mod topic, I will keep both first posts of this topic up to date. Most progress reports will probably be in that topic. Will monitor both for questions / feedback.

--------------

I've started preliminary work on a full conversion mod for EU4 last december, and made a tool for myself to assist in that.

It can read a mod (or base game) dir and show info contained in the various files.

Current version: v0.7.0 - 05-05-2015: Chrome App

• Reads script data files: http://i.imgur.com/OeY2aQ9.png
• View / analyze map data: http://i.imgur.com/T9z6Wo7.png
  (zoomable, draggable map a-la google maps)
• PdxMesh Model viewer & converter! - Converts Collada models to Paradox model files to be used in CK2/EU4! [Work in Progress]
• Export models from EU4/CK2 for tweaking & animation reuse.
  
  See 2nd post for how-to & examples.

EU4 just got spacey~

Spoiler

CK2 is in on it too

Spoiler

It is still in a very very early state, and I am adding features primarily as I need them in my own modding process.
But I thought sharing early would be good to get feedback, and maybe could assist some other modders.

In the progress of researching the possibilities for my mod I wanted to see if I could add my own models, and found out that you couldn't in EU4/CK2.
But I really wanted it for my mod, so I did some basic reverse engineering of the '.mesh' and '.anim' file format (pdxmesh).
(note: I have asked and received permission from Paradox to share this tool & specifically this mesh reader and potential converter)

Programmed entirely in javascript (angularjs + lots of javascript components) & html as a chrome app. (why a chrome app? because it is crossplatform & I wanted to experiment with it )

---------------

Get it as a Google chrome browser app:
JoroDox Tools
WARNING ALPHA STATE, please back up your mods/work/saves - as you should do anyhows.

Source files (GitHub):
https://github.com/DaanBroekhof/JoroDox
(feel free to fork/add pull requests)

---------------

Changelog:
v0.7.0 (2015-05-05):
- Export PdxMesh models from EU4/CK2 to Collada format.
- Experimental CSV import of data
- Editing of province data, and multiple at once

v0.6.0 (23-02-2015):
- Map province select, data view (EU4)
- More mod data loading (EU4: countries, localisation)
- Experimental localisation save
- CK2 map-shield model changing (see CK2 example below for special shaders)

v0.5.1 (09-02-2015):
- Support for multiple UV texture mappings (for shaders like PdxMeshTextureAtlas)

v0.5.0 (08-02-2015):
- PDXmesh shader & texture configuration added
- Better import of collada animations

v0.4.0 (05-02-2015):
- Added pdx animation reading & export from collada
- Can select pdx animations from same dir or subdirs

v0.3.1 (02-02-2015):
- Fixed bug in model converter where last triangle in model would have incorrect texture UVs
- Redirect on mesh conversion
- Updated about & version info

v0.3 (31-01-2015):
- Fixed texture mapping for multi-UV-per-vertex models
- Added data cache view & management in 'settings' tab

v0.2 (31-01-2015):
- Basic static Collada-to-pdxmesh converter added
- Pimped model viewer
- Refresh directory button

v0.1 (23-01-2015):
- Browse mod file structure
- Change? I just started!
- Read PDX asset files
- Read pdxmesh file format
- Read assorted image files (also .TGA)
- Load & analyze provinces.bmp, compare with definitions.csv, merge changes with existing definitions.csv
- If you have a 'map/seas.png' file with only your sea provinces, you can auto-generate the 'sea_starts' values in map/default.map
Known issues:
- Slow trees when they contain lots of data
- Some operations may hang if your mod does not contain the files the tool expects (like map files)

 

[Joror]

How to convert models

For now, the tool only supports basic models, with animations with NO extra bells & whistles like: mixed materials, transparencies or weapon attachment points.

1. Have a model in the 'Collada' (.dae) model format
2. Have the textures of the model (diffuse, normal and/or specular) as separate, single files, in the .DDS texture format, and name them {name-of-modelfile}_diffuse.dds, {name-of-modelfile}_normal.dds and {name-of-modelfile}_spec.dds.
3. (Your specular files (shinyness) should not be black-to-white, but instead an alpha channel - convert like this: http://superuser.com/questions/173194/converting-grayscale-shades-into-alpha-channel. No shinyness wanted? Use a fully transparent image.)
4. Install the JoroDox chrome app (see link above) and open it
5. In the top 'Settings' tab, select your mod dir (or just any dir containing your model files)
6. In the 'Explore' tab select your collada model file. You should get a basic 3D model viewer, with textures working
7. Press the 'Convert to .mesh' file
8. Open the created '.mesh' file in the JoroDox, you should see a similar model. (textures might be crappier, DDS support is so-so)
9. Voila, you got a model!
10. Configure the use the model in EU4/CK2 (see examples below)
11. IMPORTANT CK2 NOTE - The current CK2 version has a bug in the 'standard' pdxmesh renderer - make sure to include the 'gfx/FX/pdxmesh.lua' file from the demo mod below which has a fix

How to use models

• Download Example CK2 Mod v1.1 - with imported Collada as static models, animated models and custom shields.
• Download Example EU4 Mod v0.4.0 - with imported Collada as static map decoration & land unit & ship.

Animation import notes:

• The Collada import requires 30 frames per second for animation. 3DMax: Use 'sample frames' when exporting or, Blender: sample frames in between (use sample keyframes action).
• If your animation file cannot be found, or you added a model with bones, but did not configure an animation, the game will copy another semi-random animation to your model. Always look in 'Documents\Paradox Interactive\Europa Universalis IV\logs\error.log' if you see weird animations.

Tips & tricks:

• You can tweak the scale of models in the config files on various places - your model might be too small to see
• Keep your models simple before you export them to Collada
• I haven't tried this with may types of models, so bugs probably galore. Provide feedback where you encounter things.

 

[Joror]

Wait so you found a way to include your own models?

Indeed. Currently only static models (things like static map objects & buildings), but working on the animation files too.

This tool can read and partially write the .mesh and .anim files used in EU4, and CK2 (since Charlemagne).
(The .xac files used in CK2 before that it cannot read or write)

Imports Collada (.dae) model files and exports .mesh files at the moment.

 

[Joror]

About the seas.png... I think that CK2 doesn't uses sea_starts since 2.1

Otherwise, youmight be a mesiah, indeed!

Yeah the seas.png (an provinces.png with only the seas) is more a thing for my own mod, where I was playing with flexible seas (long story) - you can ignore that button for most normal mods.

 

[Joror]

The majority of folks are probably going to push you and push you and push you to get Animation files going as fast as you can, so that can start adding new military units for historical mods or in the case of some more ambitious mods, so they can change sailing ships to space ships.

However, even this development will have huge ripples in what people are modding. I, for example, am working towards a Cyberpunk mod for CK2, and possible ability to change the vanilla medieval towns into skyscrapers will make a work of difference.

Good work man; this has the makings of a game-changing development for Paradox mod makers, which are already at a very high standard.

I kind of figured that people would like to use the unit models too, and that is also something I will do for my own mod so it is on the top of the list. Will first explain how to do the static models before doing the rest tho.

Luckily I got some time on my hands next week to do some extra work.

 

[Joror]

New version + howtos in top 2 posts.

New developments: Static unit models also possible with current state:

Spoiler

 

[Joror]

New version 0.3.1 available with a conversion bugfix. (see first post)

 

[Joror]

Does this mean the shields with the coat of arms are replaceable too? I have zero 3d skills as of yet, but this might just spur me..

Eventually, yes. But at this time only simple models without 'bones' or animations can be converted. And shield objects would be probably have bones to attach to things. (I haven't examined those in detail yet)

 

[Joror]

New version v0.4.0, animations are in:

(animations untested in CK2, but most likely works there too)

 

[Joror]

I have a problem, where the .mesh file is never made. The cog spins for hours.

Are there conditions on the .dae being convertible?

The cog spinning forever is a sign that there was an internal bug that stopped the process.
At the moment most Collada files should load, but since this is still a very fresh app which has not been tested broadly, there will probably be bugs/unforseen combinations.

Simplifying your model (just one merged mesh, no animations, no bones/skeleton) might fix it.

Also, if you send me the scene in question, I could take a look at what might be going wrong. (post as zip or email me at daan.broekhof <at > gmail.com if you don't want to share publicly)

 

[Joror]

Have you tested to see if it possible to make custom Shield icons for Map COA frames?

That would be moddable I think yes. But it would require some extra work.

A current model is defined in gfx/models/Shields/count_shield.xac

Which is configured in interface/mapitems.gfx:

EMFXActorType = {
		name = "westerngfx_count_shield"
		actorfile = "gfx\models\Shields\count_shield.xac"
		scale = 0.5f
		use_animation = no
	}

That can be replaced with:

pdxmesh = {
		name = "westerngfx_count_shield"
		file = "gfx\models\Shields\count_shield.mesh"
		scale = 0.5f
		use_animation = no
	}

Where the .mesh is the location of a .mesh file created by the tool.

That would get you a new frame, but it would not have the flag of the nation in it.
To do that the modeller would have to define a separate part in the model with a specific shader: 'PdxMeshTextureAtlas' (which is a shader that I believe paints the flag over something)

That last bit is not something that I have added support for yet. And it will require some testing to get the configuration of the mesh with that specific shader correct.

 

[Joror]

So, any news on the above? Also, the example mod is crashing for me.

If by the above you mean the custom shields - it would be possible with the last version of the tool. I have added support for configuring the 'shaders' for models in the last version.
I have successfully tested this in EU4 (cars with flags-on-em : http://i.imgur.com/ybOOQP1.jpg), but I still need to test this in CK2. I test less with CK2 since I am primarily working on an EU4 mod.

Will also have a look at the CK2 example mod this weekend (maybe add a demo custom shield too) - it was working for me in the past but there might an issue.

If people have specific problems / questions on how to do specific things I am always willing to answer em as best as I can.

 

[Joror]

Just experimented with some custom shields:

These needed some extra shader configuration. (the original shield 'always faces you' logic was only added to the old 'xac' model shaders)

Will add this to the example mod files, which you can get in the second post of this topic.

 

[Joror]

New version: 0.6.0, with updated & working CK2 mod example, custom shields example, map data viewing (EU4 mostly).

 

[Joror]

Wait do I need chrome just to use this?

That is indeed the case. The application is a cross-platform javascript Chrome app.

 

[SaintDaveUK]

Just looks to me like you need to map UVs to fill the 0.0 - 1.0 square in the first UV set.

This is how it should look in Maya, but with your shield shape, not a square:

 

[Joror]

I love you. Now I just need to figure out a bit of 3D stuffs. Are you any good with Blender at all? Having trouble exporting the texture with a collada file.

Edit:

And do you know why it looks like this?

Ah hmm not exactly sure, but have an inkling. I think you are either missing a UV-mapping or have one mixed up.

To get technical: The mesh (object) that you want to have the flag printed on should have 2 UV-mapping sets.

One set (the first) is the UV for the flag. The flag UV is a regular one where 0,0 is the top left of the flag and 1,1 the bottom right.
The second UV set is the UV for the texture assigned to the mesh (in the tool). The result of this UV-mapping & texture is blended into the flag. This way you can get 'flag creases' and other effects. You can have a white texture configured to just get the unadulterated flag.

Also be sure to use the /gfx/FX/pdxmesh.lua file from my CK2 mod example in your project. (I assume you are working from that example mod? I have updated it yesterday so be sure to get the 1.1 version)

If you look at my example 'shield', the center square is a separate object with two UVs, both of which are mapped 0,0 to 1,1.
The 'sources' dir has the .blend for this if you want to inspect it.

 

[Joror]

Just looks to me like you need to map UVs to fill the 0.0 - 1.0 square in the first UV set.

This is how it should look in Maya, but with your shield shape, not a square:

Oh aye indeed people should not use exactly 0;0 and 1;1.

For best effect, use 0,05;0,05 to 0,95;0,95 UV values as SaintDaveUK says - otherwise the colors of the flag next to the current flag might bleed into this one at the edges.

 

[SaintDaveUK]

Edit: Making a lot of progress, I've 2 UV maps now and while I still can't figure out much from the example all I need now I think is to draw that damned box but I can't find out how to make 4 vertices in the UV mapping field to place in the corners.

My image is only a square shape because that is the shape of the flag mesh I was making.

Yours shouldn't look like a box, it should just look like the front view of your shield, just the size and width should be similar to my box.

 

[Joror]

Now I just need to get rid of the exaggerated reflection and get the export to pop out the texture and normal maps and I've done it <3

Looking goood! Blender is indeed a bit of a bleep UI wise, seemingly ignoring most common UI logic I know of. (not an 3D modelling expert either, just know how to connect the dots on the programming side)

I think the shinyness/reflection can be reduced by using the 'nospec.dds' for the specular texture, and a better normal map probably also helps.