We’re finally getting to see more of the launch wave of PlayStation 5 titles, with Insomniac revealing a first look at hardware accelerated ray tracing effects in Marvel’s Spider-Man Remastered and Marvel’s Spider-Man: Miles Morales. A huge improvement is clearly evident in reflections from the tallest skyscraper to the smallest puddle, but bafflingly, the reveal was met with some vitriol on social media. Artefacts and compromises were highlighted, with some even suggesting that we weren’t seeing ray tracing at all. The reality is clear though: hardware RT is an emerging feature on brand new consoles developers are just getting to know, it’s computationally expensive so compromises are inevitable – and what we’re seeing here is very impressive stacked up against similar hybrid RT implementations seen in the PC space.
In terms of Marvel’s Spider-Man on PS5, there are two rendering modes – a 4K quality mode pegged at 30fps, alongside a 60fps performance mode. The former shows obvious RT reflections but ray traced ambient occlusion is also mooted for inclusion. This is harder to identify in screenshots or in videos, but in general, ambient occlusion adds shadows into scenes to simulate the darker effect in nooks and crannies where light bounce is more muted. From what we can tell from the released footage, the 60fps performance mode sticks with the screen-space reflections seen in Spider-Man on PS4 – the key limitation being that only detail in the camera view can be reflected, leading to many visual discontinuities. Skyscraper reflections drop back to much rougher approximations based on captured cube maps that change according to height.
RT is a revelatory improvement here, but some people don’t seem to be impressed. It’s true that the resolution of the RT reflections is lower – perhaps quarter resolution compared to the main image. Others weren’t impressed that certain objects seen on-screen aren’t represented in the reflections, leading to suggestions that what we’re seeing can’t be ray tracing. Meanwhile, the leaves in trees appear much reduced, while reflections also miss some shadows. As I see it, all of these characteristics make a lot of sense and are unsurprising. What we’re seeing in Spider-Man is the simple result of the fact that RT has multiple performance sensitive steps and developers have a limited rendering budget for ray tracing. I’ve tried to rationalise this into four distinct steps.
The first step acknowledges that RT reflections are based on light that can be coming from any direction – from areas not rendered on-screen and can even be very far away. Already the performance implications are building up: the game needs to generate an easily readable version of the game scene to test ray tracing against. The larger and more detailed it is, the more GPU time is required to generate it.
The second step is the actual ray tracing, where tays are shot into the structure defined in the first step. To accurately map collisions against geometry, all of the objects in the game scene have multiple invisible boxes of increasing size around them. The rays shot into the scene move through these boxes until they hit one that contains geometry. The more rays, the more collisions, the more detailed the geometry, the longer this process takes. It’s this step that is accelerated via hardware.
The next step is about shading pixels where a hit was registered. Simply put this means shading the hit or choosing the colour for a pixel – like painting the glass side of a skyscraper accurately, for example. The more hits there are to shade or the more more complex that shading is, the longer it takes the GPU to complete. Because ray count is limited, the end result looks very noisy and clean-up is required. This leads us to the final step, where the reflection is denoised – essentially, smart algorithms are used to fill in the blanks, giving a more convincing reflection.
Even with hardware acceleration on the ray tracing step itself, this is still hugely stressful on the GPU and compromises are required. So that’s why Spider-Man’s reflections are of a lower resolution – it cuts down the ray count dramatically, giving a huge speed-up to the whole process. In the video embedded on this page, I use the excellent RT implementation in Ghostrunner operating on Unreal Engine 4 to give some idea of why these compromises are in place. The bottom line is that even in a simple scene, dropping RT reflections to quarter resolution increases performance by 28 per cent on an RTX 2060 Super, rising to 58 per cent in a more reflection heavy scene. This explains why Spider-Man’s reflections aren’t running at the same resolution.