+ Reply to Thread
Results 1 to 17 of 17

Thread: Dev Dairy #2

  1. #1
    Kerberos Productions castewar's Avatar
    Sword of the StarsSword of the Stars II

    Join Date
    Jan 2010
    Location
    Vancouver
    Posts
    250

    Dev Dairy #2

    by Lead Programmer, Darren Grant

    Over the last five years–through two expansions, a weapons pack and a tech pack–the graphics engine that fueled the original Sword of the Stars game (MARS) has done everything we asked of it. More than we’d originally intended, really. Sword of the Stars II seemed like the perfect time and opportunity to look back at our previous engine and build the future from the ground up.

    Part of the process of designing a new engine architecture is identifying exactly what can be improved over the previous iteration. For MARS2, we took a number of new approaches to accommodate our goals for the sequel: provide the player with the capability for additional immersion and interaction with the universe. Below, I break down the different elements that we’ve added/upgraded in order to bring our full vision of SotS II into existence. It can get a bit technical at times, but I hope you find it enlightening!

    Direct 3D 10
    For most gamers, the benefits of stepping up to D3D10 is shrouded in mystery. That’s partly because Microsoft’s main goal with this version is mostly to ease the lives of OEMs and developers, by forcing adopters to agree on competent standards. What this means is that, unlike D3D9 (where hardware and drivers delivered a hodge-podge of mixed and matched features for developers to wade through–a compatibility nightmare), D3D10 guarantees a very powerful common baseline that stays the same. For example, as a graphics programmer, I know with near certainty that if the shadow rendering works correctly on my work station, it will behave correctly (albeit at different speeds) on any DX10- or DX11-era hardware and software. There were no such certainties in D3D9, which added to lost development time.

    Shader Model 4
    One of the more exciting features that D3D10 adopters get is the common Shader Model 4 baseline. A D3D9 game is limited to Shader Model 2.0 on average (which was the shader model target for SotS Prime). Sadly, the lack of basic computing power on the GPUs of that era severely limit the number of features that could be achieved concurrently.

    That bottleneck caused us a lot of problems. You want skinned models? OK, the programmer would come back instantly with a solution! Now you want to add 2 dynamic light sources per mesh? Hmm, it’s going to take a day for your programmer to rewrite the shaders to fit both features in the given space available. Now you want shadows? Your programmer grumbles, goes away for a couple weeks and comes back with a rewrite of the renderer that jumps through hoops to solve the problem. Just don’t try to fit more in after that.

    One of the things we learned developing SotS Prime is that if you don’t set up the proper tools for the artists initially, our programmers would spend a lot of time attached to the artists, tweaking parameters. And toward the end of a project, when things are getting tight, we’d be forced to drop certain polish features in favor of actually going final. Upgrading our engine let’s us avoid that this time around.

    Render stack
    Render stacks handle and sort the drawing instructions for each frame and optimize them before they reach Direct3D. There are two major categories of 3D rendering in games: Forward and deferred. Forward rendering is a technique where each pixel is drawn to the screen once, with all of the shading values already accumulated. This can be very efficient in situations where there are very few light sources, and is optimal in genres like FPS, where the world geometry is largely static and can therefore be heavily optimized. It doesn’t work well with dynamic lighting, though, so there’s also deferred rendering, which separates the lighting and material rendering passes, making it optimal for situations with lots of relatively small dynamic light sources.

    Our render stack uses a deferred technique to produce some brilliant dynamic lighting and post-process effects, many of which can already be seen in various screenshots and videos. These effects can be tuned based on the class of video card available. For instance, the depth of field blur radius may be reduced, or flat-out disabled based on the player’s preferences of image quality vs. performance. The gamut of effects that can be disabled include full scene shadowing, hundreds of dynamic light sources, decaling, refraction, local ambient occlusion, full HDR bright/bloom and image based ambient lighting.

    Particle effects
    Including the rich particle effects of the previous game in SotS II is big deal for us. MARS2’s particle-authoring system has been greatly revised to enable integration with material rendering and control of dynamic lighting systems. With material integration, some complex and dramatic new effects are possible, and with control over dynamic lights, the particles can convey a much greater presence as they skirt past the hulls of nearby ships or fade into scorch marks.

    Thanks to our deferred rendering system, the engine an handle hundreds of simultaneous light sources. There’s no reason that a particle system can’t be authored to provide a beautiful animated glow that casts light on surrounding geometry in the game. Having direct control over the lighting inside the particle authoring tool gives artists very direct control over the integration of emitted light and the particles they want to draw. In other engines this might have to be done by going to two separate tools, which disrupts work flow.

    Shared material library
    Another key component to ensuring our artists’ efficiency is our shared material library, which enables artists to create and change the appearance of multiple models at once. Each material is based on existing shader class, which means that artists do not need to write code or cut and paste parts of cryptic shader trees–with the support of an in-house model viewer, the shared material library lets artists fine-tune the parameters only once and see the effect on all models that use that same resource.

    For instance, an artist might create a material called CruiserGlass, which calls for a green-glass shader with all the fancy refractive and reflective trimmings, tailor the colors and light levels, and then share those settings with all of the glass panels in all Tarka cruisers. Without the help of a single programmer or having to tweak the models at all, an artist can change one paramater on the CruiserGlass model and see the changes across all Tarka cruisers in the game simultaneously.

    Asset pipeline and multithreading
    In order to further reduce load times between major state changes and improve the performance of the game in general, we built a massive asset pipeline. This pipeline combines a new offline build process with an engine-integrated background loader and cache. Most assets are mapped directly from their files by the runtime, and once they’re in the RAM, they stick around unless the cache needs to recover space. This eliminates most of the long loading time problems that can happen when moving between encounters and the star map.

    MARS2 had been designed to work with modern multithreaded architecture, which will also greatly help loading times. We’re using a hybrid approach to target the 2-8 core PC’s that are common today, with an eye to 16+ core desktop systems that are promising to emerge. One primary CPU thread does the heavy lifting and secondary threads are assigned to specific support jobs such as loading and processing assets, building render lists, or executing high level game logic. Using this logic, for example, the engine will prepare for an upcoming combat encounter.

    Bullet Physics and 64-bit

    Another way we’re aiming to make the wide array of weapons and ships feel more alive is by adding Bullet Physics, a superb physics engine that has seen successful application in many major games and movies. It lets us model much more complex collision geometry than in SotS Prime. This greater level of detail goes a long way towards conveying the much larger range of scale in the game.

    One of the questions I get asked the most is, “Will Sword of the Stars II ship with native support for 64-bit OS?”. The answer is most assuredly, “Yes!” In fact, we aimed the MARS2 engine to target both 32 and 64-bit builds since the first lines of code went in, a little over a year ago.

    I hope this overview of the new game engine answers a lot of the questions people have been asking about SotS2 and maybe even a few that haven’t been asked yet. As we head towards completion of the game in the coming weeks, we’ll get into more of the fine details about the game itself, so stay tuned for our future dev diary entries!
    Last edited by castewar; 18-03-2011 at 06:00.
    Kerberos Productions Forums
    "Repensum est Canicula" - unofficial SolForce motto
    Follow us on Twitter!

  2. #2
    The Last Eurofederalist LeeDub's Avatar
    Achtung PanzerArsenal of DemocracyHearts of Iron 2: ArmageddonCrusader Kings IIDarkest Hour
    Deus VultEuropa Universalis 3Divine WindFor The GloryFor the Motherland
    Hearts of Iron IIIHOI3: Their Finest HourHeir to the ThroneEuropa Universalis III: In NomineMagicka
    Majesty 2EU3 Napoleon's AmbitionVictoria: RevolutionsEuropa Universalis: RomeSemper Fi
    Sword of the StarsSword of the Stars IISupreme Ruler 2020 GoldVictoria 2Victoria II: A House Divided
    Victoria II: Heart of DarknessRome: Vae VictisMount & Blade: WarbandCK2: Holy KnightKnights of Pen and Paper +1 Edition
    500k clubEuropa Universalis IV: Pre-order

    Join Date
    Oct 2008
    Posts
    1,356
    The graphics are really good so far. That Morrigi Dreadnought looks great both in design and in execution. That small craft also looks cool, if not very Morrigi-like.

    Great news on even better physics, too.

  3. #3
    Duke of Miža King_Richard_XI's Avatar
    200k clubCities in MotionCrusader Kings IIDeus VultEuropa Universalis 3
    Europa Universalis: ChroniclesEU3 CompleteFor The GloryHearts of Iron III CollectionHeir to the Throne
    Majesty II CollectionNaval War: Arctic CircleVictoria: RevolutionsSword of the StarsVictoria 2
    Victoria II: A House DividedMount & Blade: WarbandMount & Blade: With Fire and Sword500k clubEuropa Universalis IV: Pre-order
    EUIV: Call to arms event

    Join Date
    Nov 2008
    Location
    Location: Location
    Posts
    2,545
    Blog Entries
    4
    Very nice looking, I can tell this is going to be a good, fun, realistic (physics-wise at least) game.
    All hail Richard XI, King of the Mod, Grand Duke of Simcity 4, and Tsar of All the Awesomes - The_Shepherd

  4. #4
    Ill Growl of Talking Polar Mongoose's Avatar
    Hearts of Iron 2: ArmageddonCities in MotionCrusader Kings IIDeus VultDivine Wind
    Hearts of Iron IIIHeir to the ThroneEuropa Universalis III: In NomineLost Empire - ImmortalsMagicka
    Majesty 2EU3 Napoleon's AmbitionVictoria: RevolutionsEuropa Universalis: RomeSword of the Stars
    Sword of the Stars IIVictoria 2Victoria II: Heart of DarknessRome: Vae VictisMount & Blade: Warband

    Join Date
    Mar 2006
    Location
    London, UK
    Posts
    732
    Interesting insights there into something I've never really thought about that much. Looking very good.
    ˇViva ArgentinAAR! - My Vicky 2 AAR (1836-1880)

  5. #5
    Oberkommando HOI4 Paradox Dev Team podcat's Avatar
    200k clubHoI AnthologyDiplomacyFor the MotherlandHearts of Iron III
    HOI3: Their Finest HourEuropa Universalis: RomeSemper FiSengokuVictoria II: A House Divided
    Victoria II: Heart of DarknessMount & Blade: WarbandEuropa Universalis IV: Pre-order

    Join Date
    Jul 2007
    Location
    Werfing Nebels
    Posts
    8,223
    Sounds really cool. Could you say anything about how you ended up handling transparency which is usually an issue in deferred renderers? Separate passes with forward rendering, or some more exotic solution?

    Totally agree on DX10/11 and higher shader models, while they do enable you to do more complex effects the big advantage is how much less of a headache they are to work with for developers
    Reject reason to make the impossible possible!

  6. #6
    Very interesting technical details, and it is good to see a new dev diary... the game is indeed looking very good and I am anxious to see more of it.

    Actually, I must say the Morrigi Dreadnought looks a bit clunky, but that may be the result of too many colours - or their alliance with the Zuul. I would rather have a cleaner looking, more elegant design, but I understand this is part of the whole concept of the new game. And besides, that's a matter of taste. I personally prefer simpler lines, like the Imperial ships of Star Wars, or at least something more utilitarian, like the Human ships in Babylon 5. The only ships I used to like in SOTS were the Tarka, and maybe the Hiver... but as I played the game the awkward design of the Human ships started to look more and more attractive... and now I like them too.

  7. #7
    10+ Years on the Forum Demi Moderator SAS's Avatar
    98 games registered

    98

    200k clubHoI AnthologyCrusader Kings IIDeus VultDiplomacy
    For the MotherlandHearts of Iron IIIHearts of Iron III CollectionHOI3: Their Finest HourIron Cross
    Lost Empire - ImmortalsMarch of the EaglesVictoria: RevolutionsSemper FiSengoku
    Victoria 2Victoria II: A House DividedVictoria II: Heart of DarknessRome: Vae VictisCK2: Holy Knight
    EU Rome Collectors EditionEU3 Collectors EditionKnights of Pen and Paper +1 Edition500k clubEuropa Universalis IV: Pre-order
    EUIV: Call to arms eventMagicka: Wizard Wars Founder WizardWarlock 2: The ExiledEUIV: Wealth of NationsEUIV: Conquest of Paradise
    EUIV: Res PublicaAncient SpaceCrusader Kings II: Legacy of RomeCrusader Kings II: Sword of IslamCrusader Kings II: Sunset Invasion
    Crusader Kings II: The RepublicCrusader Kings II: The Old GodsCrusader Kings II: Sons of AbrahamCrusader Kings II: Rajas of IndiaCrusader Kings II: Charlemagne
    Warlock 2: Wrath of the NagasEUIV: Art of WarParadoxian Order - 2015Crusader Kings II: Limited Collectors EditionCrusader Kings II: Way of Life

    Join Date
    Jan 2002
    Location
    Icon Collecting
    Posts
    6,439
    Blog Entries
    1
    Love the detailed info, looking better and better!
    - March of the Eagles mod: The Gods

    - When I speak in Cyan its as a Demi-Moderator. Please review the following links: FAQ / Forum Rules / General Usage

  8. #8
    Is this going to become a regular/weekly developer diary? I can't wait for the SotS sequel and very DD will be read espcially! Also, will this be multi-threaded?
    1 Clerk
    Nationality: European/British
    Religion: Atheist
    Ideology: Liberalism/European Federalist
    Issues: Free Trade/Atheism
    Militancy: 2 (-0.01)
    Consciousness: 10 (+0.12)

  9. #9
    Quote Originally Posted by KommanderKhaki View Post
    Is this going to become a regular/weekly developer diary? I can't wait for the SotS sequel and very DD will be read espcially! Also, will this be multi-threaded?
    Yes it is multi-threaded, says so right in the section on multi-threading. They will use 1 HW and several MW that will spawn LW helper threads. They are shooting at the 2-8 core processors with an eye out for 16+ cores in the future.

  10. #10
    Games Player steveh11's Avatar
    Crusader Kings IIDivine WindFor the MotherlandHearts of Iron IIIHOI3: Their Finest Hour
    Heir to the ThroneEuropa Universalis III: In NomineEU3 Napoleon's AmbitionRome GoldSemper Fi
    SengokuVictoria 2Victoria II: A House DividedVictoria II: Heart of DarknessCK2: Holy Knight
    EU3 Collectors Edition500k clubEuropa Universalis IV: Pre-orderAncient SpaceParadoxian Order - 2015

    Join Date
    Jun 2001
    Location
    EU, EU2, CK, Vicky, HOI, HOI2, et al...
    Posts
    1,960
    I guess this means I'll have to buy *yet another* upgraded video card, or do without a game... Can you give definitive minimum requirements yet?
    "Nature always obeys her own laws"
    - Leonardo da Vinci

  11. #11
    Quote Originally Posted by RobAK801 View Post
    Yes it is multi-threaded, says so right in the section on multi-threading. They will use 1 HW and several MW that will spawn LW helper threads. They are shooting at the 2-8 core processors with an eye out for 16+ cores in the future.
    DOH, yeah, sorry, I read that and just read it as something completely different. Don't konw why, late night reading, lol. Cheers, but 16+?! Is that even on PC markets? I have an AMD Phenom X6 1055T but damn, 16? The other one works like a charm but I think Windows 7 might of ruined it lol.

    Just went back and checked, I thought it was still talking about DirectX10/11 and GPUs as opposed to CPU. My bad! Cheers though, also, is it confirmed this will bea regular diary (i.e. every Friday)?
    1 Clerk
    Nationality: European/British
    Religion: Atheist
    Ideology: Liberalism/European Federalist
    Issues: Free Trade/Atheism
    Militancy: 2 (-0.01)
    Consciousness: 10 (+0.12)

  12. #12
    And I came here looking for milk ...

    but now i read it and it did make for some interesting technical reading even though i have no clue about stuff like that.... So... nice work!

  13. #13
    Doubt it will be weekly, probably monthly at best. It would take a lot of effort to do a weekly diary for the next 5 months for such a small development team, this isn't a big developer like EA with 100+ people per project, there are only a couple dozen people and many do double duty as writers/coders/artists/etc.

  14. #14
    Kerberos Productions castewar's Avatar
    Sword of the StarsSword of the Stars II

    Join Date
    Jan 2010
    Location
    Vancouver
    Posts
    250
    Quote Originally Posted by podcat View Post
    Sounds really cool. Could you say anything about how you ended up handling transparency which is usually an issue in deferred renderers? Separate passes with forward rendering, or some more exotic solution?
    Nothing exotic - seperate passes for forward rendering.

    And as for how often, we'll be doing one a month, though the next one is coming up fast. Stay tuned.
    Kerberos Productions Forums
    "Repensum est Canicula" - unofficial SolForce motto
    Follow us on Twitter!

  15. #15
    General ashbery76's Avatar
    Crusader Kings IIHearts of Iron IIIVictoria: RevolutionsSword of the StarsSword of the Stars II
    Victoria 2Warlock: Master of the ArcaneEU Rome Collectors EditionEU3 Collectors Edition500k club
    Europa Universalis IV: Pre-order

    Join Date
    Oct 2000
    Location
    Rugby,England
    Posts
    1,757
    Hopefully the next diary will be for gamers and not engineers.
    Last edited by ashbery76; 23-03-2011 at 00:00. Reason: Typo.

  16. #16
    Kerberos Productions castewar's Avatar
    Sword of the StarsSword of the Stars II

    Join Date
    Jan 2010
    Location
    Vancouver
    Posts
    250
    Quote Originally Posted by ashbery76 View Post
    Hopefully the next diary with be for gamers and not engineers.
    Well, it's a personal preference I guess. Martin's first diary was more gameplay focused. This time, we let the lead engineer talk about his side of things, and those with a technical mind seemed to like it. A lot of people seemd to like it actually. Next up is Chris, our art lead, who will talk about the graphics from an art angle. So stay tuned.
    Kerberos Productions Forums
    "Repensum est Canicula" - unofficial SolForce motto
    Follow us on Twitter!

  17. #17
    that sounds cool, when are we going to get more gameplay elements though, like getting into the guts of research and ship design, of how these Provinces work and so on?

+ Reply to Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts