2014 seems a very long time ago, but that’s when Microsoft first unveiled news that it was working on releasing DirectX 12, to serve as the companies answer to developers asking for a low level API. DX12 was designed from the ground up to tackle the issues and constraints plaguing PC game development at the time. Microsoft’s DX12 (along with Khronos Groups Vulkan) can thank AMD’s Mantle API for laying much of the ground-work for the then new APIs. AMD’s Mantle spurred development (and in the case of Vulkan, actually was used as loose core of the API; albeit with the AMD specific optimizations removed and other tweaks) and demonstrated just what PC hardware was truly capable of. Developers were finally given access to the tools to properly access the true performance of both the CPU and GPU of PCs of the era.
*EDIT* we’ve also got an updated version of this article and video which you can find here.
We here at RedGamingTech, extensively covered Vulkan, DX12 and Mantle during their initial announcements (we actually have a few exclusive interviews with AMD’s Robert Hallock regarding Vulkan, linked in the description if you’d like to check them out). It was very easy to see the very impressive Witch Demo and have open mouths, drooling at what was to come. DirectX 12 and Vulkan allowed much gave games the ability to run over more threads, allowing the GPU to be better fed with data, and furthermore gave developers much better control over GPU resources.
And as I’m writing this, Nvidia, AMD and Microsoft (and about everyone else in the industry) are waxing lyrically about the benefits of RayTracing – and DXR (DirectX RayTracing) is yet another extension of how the industry will evolve, pushing the graphical envelope of future games even more. The titles such as Metro Exodus look incredible, and Nvidia are currently perfecting Adaptive Temporal Antialiasing, a sure sign of the future of graphics.
It was easy to assume that the vast majority of PC games would be DX12 or Vulkan by now, and DX11 would be relegated to smaller indie studios or games which didn’t push the visual envelope. But this hasn’t been the case, and games such Resident Evil 7, Final Fantasy 15 and even Black Ops 4 are all firmly stuck on DX11. Reasons of the slower adoption are numerous – it’s harder to develop games using DX12 or Vulkan (if you’re really pushing optimization boat out), older hardware typically performed better with DX11 (although this is starting to change, as people are upgrading their machines naturally) and drivers and SDK’s were immature (and this is another area that’s rapidly evolving).
Microsoft’s adoption of DX12 to both the Xbox platform and PC, and the dizzying array of platforms you can target with Vulkan have seen the release rates for games using these APIs speed up. Now ID software’s Doom (and Doom Eternal), Wolfenstein games, Microsoft’s own titles and Square Enix titles are particularly easy to point to when it comes to pushing the new APIs.
All of this is all well and good – but of course, ultimately gamer’s want to know what type of performance benefits you can see from using the API and what type of hardware best takes advantage of it.
We’ll be using an Intel I7-8700 processor for the CPU, testing with different CPU core count configurations and disabling Hyper Threading to see how the different APIs scale. For graphics, we’ll be using Nvidia’s GeForce GTX 1080 Ti, and test across the 3 most popular resolutions – 1080P, 1440P and finally 4K. Oh – and we’ll be using a combination of manual runs along with the built in benchmarks, and this is alongside Crucial Ballistix RAM and an MSI B360 Gaming Pro Carbon motherboard. For software we’ll be using Windows 10 with all software patched to the latest retail versions via Steam.
Before we continue – we’d like to thank MSI for providing the motherboard and processor used in this review; and if you’d like to check out the motherboard review please find it embedded in the article too.
If you’d like more results for games across different resolutions, lower core counts and testing with a GTX 1080 as well as the 1080 Ti, and with different games (including FF15), you can checkout our CPU and Resolution scaling video, which is embedded below too.
https://www.youtube.com/watch?v=AI8go-mYf-w
One of our favorite benchmarks is rise of the tomb raider, and this set of testing is no different. We run the game using both the built in benchmark and a manual run through the games GeoThermal Valley level.
Starting things out with the built-in benchmark, which shows a rolling demo of several of the games locations. 1080P shows a slight nod towards DX12 – but ‘slight’ is the operative word. DX12 enjoys just a 10FPS lead with the I7-8700 running with all its cores and threads. Disabling HT shows a slight improvement to both DX12 and DX11 results, but DX12 still brings home the victory between Microsoft’s two APIs. The only anomaly here was 4 cores with 4 threads, which performed better than all the other tests combined. We ran the test a few times and came away with the same result each time.
Naturally, higher resolutions shift more of the responsibility onto the GPU, and even at 1440P you’ll find yourself frequently GPU bound, despite our settings of only ‘high’ and only SMAA.
By the time 4K rolls around, and there’s very little in the DX11 vs DX12 results, and the performance falls almost within margin of error. Don’t forget, we have actually taken ROTR to just a few threads in our CPU scaling video (seen above) and the results were quite similar.
So; there we go then right – DX12 is a little faster than DX11 in these benchmarks, but nothing super impressive? Well, let’s try some manual runs – once again, using GeoThermal Valley. This is a great area in the game because lots of ‘stuff’ is happening, and its easily repeatable to do manual runs without much variance. It also pushes the CPU and GPU, and thus makes it a fantastic worst case scenario for the game engine, as we’re asking it to push water, physics, lots of draw calls, large numbers of NPCs and vast swathes of land, thus constantly swapping data in and out of both GPU and main system RAM.
The manual run did the trick, and the difference between the two APIs becomes incredibly obvious. Focusing in on the 12 thread DX12 vs DX 11 for a moment, and we can see a huge gap between the ‘average’ frame rate of about 40 percent. But truly, the real test comes from minimum frame rate – which is almost double the difference. Think of this – the DX12 4 core and 4 thread tests are very similar in performance (aside the max FPS) to the 12 thread DX11 tests.
Alright, alright – compelling data – but we’re giving the GPU a lot of leeway here. Let’s increase the pixel count by 2.25x (AKA, 1440P) – what happens now? Well, the average FPS isn’t really the story – about 8FPS separates the best performance (DX12 with 12 threads) and the worst performance (DX11 with 4 threads). But…. the minimum, and 1 percent frame rates and even the 0.1 percent all have a startling difference. Once again, 4 threads and DX12 out pace 12 thread DX11. At the worst case scenario, with just 4 physical cores, average FPS is only slightly better on DX12, BUT the 0.1, 1 percent, minimum and max are all better. Basically speaking, you’ll get a smoother and more consistent experience with the game running in DX12 mode, even with just 4 cores to throw at the problem.
Hitman 2016 is yet another title which firmly embraces DX12, and the results here are almost pretty clear too. There’s a 20 percent difference between DX12 and DX11 with all 12 threads, although DX11 does do slightly better without HT in our 6 physical core tests, although going further down the CPU core stack and yeah, DX12 continues to beat the competition, finishing 6 percent faster with just 4 cores… although it’s clear with just 4 cores, the tests really start becoming CPU bound.
At 1440P it’s a similar situation, with DX12 coming out on top – but at 4K (where you’re GPU bound) and we run into a consistent situation where DX12 loses out to DX11. We’ll investigate this further in part 2 of our analysis when we bring more games and graphics cards into the mix (and of course, it goes without saying that if you want to see this stuff, please support the channel!)
Deus Ex is a bit of a strange one – because we clearly can see that the game, even with only the high settings and 1080P is having our reference design GTX 1080 Ti run against the GPU bound wall. The results really close and there’s not much to say other than DX11 is ever so slightly faster here, other than with HT off with all 6 cores.
But as the resolution increases though, and the situation changes totally, with DX12 being ever so slightly ahead, and at 4K we get results which are pretty much identical. There are slightly better minimums and high FPS in play here, but the results aren’t that big of a deal – particularly at higher resolutions and graphical settings.
AOTS Escalation requires little introduction, and Brad Wardell and his team at Stardock were a staunch proponent of DX12 and low level APIs from the get go. There’s little I can say here other than DX12 continues to dominate the performance across the gamut versus DirectX 11.
Quite simply put, AOTS does better with additional threads and the ability to use those threads. The games engine is designed to show massive battles, with large numbers of units and calculating all of the AI, weapon fire trajectories, keeping track of all what’s going on, and of course actually drawing that just really does a number of the CPU.
What we have here is a perfect scenario for Low Level APIs to strut their stuff, and DirectX 12 totally dominates DirectX 11.
Another popular benchmark for DX12 is Civilization 6, and we use the AI benchmark with DX12 vs DX11. Rather than frame rate, this measures the time it takes for the AI to run a series of turns.
Finally, we’ll use Doom, which was used as a public demo back in the day to show off Vulkan vs OpenGL, running on an Nvidia GeForce GTX 1080. We run through the game (as there’s no built in benchmark) and look at the averages. We decided to stick to 1080P, because it’s a great example where the legacy OpenGL is clearly at a disadvantage compared to Khronos Groups latest and greatest API.
As a quick aside, we’ll of course be expanding our testing in the new Wolfenstein and also the upcoming Doom Eternal, plus other Vulkan games – so if that’s something you’re interested in, stick around and help make our testing even better.
We decided to take the Intel I7-8700, running its full might of 6 core and 12 threads and then reducing it to just 4 physical cores. The maximum frame rate isn’t the issue here, clearly with the game engine clearly being the limiting factor (Doom 2016 is limited to just 200 FPS). But the averages… the averages are super interesting. OpenGL clearly does terribly with the 12 threads available, and this can be seen immediately with low FPS of just 139 FPS for the average. That’s considerably worse than 4 cores and OpenGL, and 4 core with Vulkan. But… but, Vulkan is a completely different story.
https://www.youtube.com/watch?v=YxOhhhieKAY
For a start, Vulkan on either 4 cores or 12 threads essentially locks to 200 FPS (with a texture / area load here or there, or when we’re really having a lot of stuff happening on screen thus for a moment or two the 1080 Ti is the problem). The low FPS is also startling, with the 1 percent low FPS jumping almost 40 percent, and the minimum frame rate doubling from 12 threads with OpenGL to 12 threads with Vulkan. We see over 66 FPS increase in the minimum from OpenGL to Vulkan. There’s not much I can say other than what the numbers show – Vulkan decimates OpenGL, and the performance increase isn’t just tangible, it’s an entirely new world.
Conclusions
So there we have it then – part one of our look of modern apis vs the legacy APIs. And we hope you have enjoyed it; this has been very much a labor of love and don’t worry; there’s a lot more to come. We’re aiming to look at this with mid range hardware soon, and also continue coverage with the newest games, more resolutions and the bleeding edge hardware.
If you’d like to see more of that and you’re not a subscriber to our YouTube channel, well you know what to do! And if you are a subscriber then thanks very much for all of your support; we’re very grateful!
We are on Patreon, so if you’d like to check us out – well, feel free! Of course no pressure, but do know that your contributions really do help make content like this possible.
