Deep Learning Super Sampling (DLSS 2.0) Explained
DLSS or Deep Learning Super Sampling is Nvidia’s technique for smart upscaling, which can take an image rendered at a lower resolution and upscale it to a higher resolution display, thus providing more performance than native rendering. Nvidia introduced this technique with the first generation of the RTX series of graphics cards. DLSS is not just a technique for ordinary upscaling or supersampling, rather it uses AI to smartly increase the quality of the image that was rendered at a lower resolution in order to preserve the image quality. This can, in theory, provide the best of both worlds as the image displayed would still be high quality while the performance will also be improved over native rendering.
Need for DLSS
So why do we need such fancy upscaling techniques to squeeze out more performance? Well, the reality is that the technology of newer monitors is developing at a much faster rate than the technology of our PC components. The newest monitors can provide crisp 4K resolution with up to 144 or even 165Hz refresh rates. Most gamers nowadays consider 1440p 144Hz to be the sweet spot for high-end gaming. Driving these kinds of resolutions at these refresh rates takes a lot of graphical horsepower. In modern games, only the best of the best GPUs may be able to handle 4K 60 FPS gaming with everything set to Ultra. This means that if you want to improve performance but don’t want to compromise on image quality that much, upscaling or DLSS supersampling technique may come in handy.
DLSS can also be important for those gamers who want to target 4K resolution but do not have quite the graphical horsepower to do so. These gamers can turn to DLSS for this task, as it would render the game at a lower resolution (say 1440p) and then smartly upscale it to 4K for a crisp image but still higher performance. DLSS can come in quite handy more mid-range and entry-level RTX graphics cards and enable the users to play at higher resolutions at comfortable framerates without compromising too much on quality.
Raytracing
Another big feature that is being pushed to the forefront of PC gaming is Real-time Raytracing. Nvidia announced support for raytracing with their new RTX series of graphics cards. Raytracing is a rendering technique that provides accurate light path rendering in games and other graphical applications, resulting in much higher graphical fidelity especially in shadows, reflections, and global illumination. While it does provide some stunning visuals, Raytracing has a big impact on performance. In many games, it can actually cut the framerate in half, as compared to traditional rendering. Enter DLSS.
Using the power of DLSS (and now the much improved DLSS 2.0) gamers with RTX series of graphics cards can alleviate much of the performance loss that comes with Raytracing, and can enjoy a higher fidelity raytraced image while retaining a higher framerate. This technique is deemed extremely impressive by reviewers and the general public due to the fact that it can make raytracing actually playable at high resolutions, and it retains almost exactly the same picture quality as the traditionally rendered image. DLSS is an absolute necessity with Raytracing and Nvidia did a fine job developing and releasing these two techniques simultaneously.
Traditional Upscaling
Upscaling and supersampling techniques have existed in the past too. In fact, these are built into almost every modern game and even the control panels of both Nvidia and AMD. These techniques also implement the same basic upscaling method as DLSS; they take a lower resolution image and upscale it to fit a higher resolution display. So what makes them any different? The answer basically comes down to two things.
- Output Quality: The output image quality of traditionally upscaled games is generally lower than that with DLSS. This is because DLSS uses AI to calculate and adjust the image quality so that the difference between native and upscaled images can be minimized. There is no such processing in traditional upscaling techniques, so the output image quality is lower than both traditional rendering and DLSS.
- Performance Hit: Another big drawback of traditional supersampling is the performance hit over DLSS. This upscaling can render the image at a lower resolution, but it does not provide nearly enough performance improvement to justify the loss of image quality. DLSS mitigates this issue by providing a massive performance boost, while still retaining image quality extremely close to native quality. This is why DLSS is being labeled as “The next big thing” by many tech experts and reviewers.
What makes DLSS Unique
DLSS is a technology that has been developed by Nvidia, who is the world leader in groundbreaking work like Deep Learning and Artificial Intelligence. It is understandable that DLSS has a few tricks up its sleeve that elude traditional upscaling techniques.
AI Upscaling
DLSS harnesses the power of AI to smartly calculate how to render the image at a lower resolution while keeping maximum quality intact. It uses the power of the new RTX cards to perform complex computations and then uses that data to adjust the final image to make it look as close to native rendering as possible. This is an extremely impressive technology that we hope continues to develop further as many have even dubbed DLSS to be the “future of gaming”.
Tensor Cores
Nvidia has put dedicated processing cores on the RTX series of graphics cards which are known as Tensor Cores. These cores act as computational sites for deep learning and AI calculations. These fast and highly advanced cores are also used for DLSS calculations. The technology of DLSS uses deep learning features of these cores in order to preserve quality and provide maximum performance while gaming. However, this also means that DLSS is only limited to the RTX suite of graphics cards with Tensor cores, and cannot be utilized on older GTX series of cards, or cards from AMD for that matter.
No Hit to Visual Quality
The hallmark feature of DLSS is its extremely impressive conservation of quality. Using traditional upscaling using the game menus, players can definitely notice a lack in sharpness and crispness of the game after it has been rendered at a lower resolution. This is a non-issue while using DLSS. Although it does render the image at a lower resolution (often as much as 66% of the original resolution), the resulting upscaled image is far far better than what you would get out of traditional upscaling. It is so impressive that most players cannot tell the difference between an image natively rendered at the higher resolution, and an image upscaled by DLSS. This is a groundbreaking feat in gaming since gamers are always looking for a balance between quality and performance. With DLSS, they have a chance of getting both.
Significant Performance Gains
The most notable advantage of DLSS and arguably the entire incentive behind its development is the significant uplift in performance while DLSS is turned on. This performance comes from the simple fact that DLSS is rendering the game at a lower resolution, and then upscaling it using AI in order to match the monitor’s output resolution. Using the deep learning features of the RTX series of graphics cards, DLSS can output the image in a quality that matches the natively rendered image.
Makes Raytracing playable
Raytracing emerged out of nowhere in 2018 and suddenly became the forefront of PC Gaming with Nvidia pushing this feature hard and even branding their new graphics cards as “RTX” instead of their usual GTX naming scheme. While Raytracing is an interesting and unique feature that does increase the visual quality of the game, the gaming industry is still not ready to shift completely to raytraced rendering over traditional rasterized rendering yet.
A big reason for this is the performance hit that comes with Raytracing. By simply turning on Raytracing, some games can experience performance loss of up-to HALF of the original framerate. This means that you are significantly compromising on performance even on the most high-end graphics cards.
This is where DLSS comes in. DLSS can actually make this new feature playable in even the most demanding games. By rendering the image at a lower resolution and later upscaling it without any loss to visual quality, DLSS can compensate for the performance hit that Raytracing normally brings to games. This is why most games that support Raytracing also have support for DLSS so that they can be used together for a near-perfect experience.
Customizable presets
DLSS 2.0 improves further upon the framework laid by DLSS and introduces more customizable presets. Now users can choose from 3 presets called Quality, Balanced, and Performance. All 3 presets improve the performance in some ways, while the Quality preset can even improve the image quality over native rendering! DLSS 2.0 has now also introduced an Ultra Performance preset for 8K gaming with the GeForce RTX 3090 that actually makes 8K gaming possible.
Under the Hood
Nvidia has explained the mechanics behind its DLSS 2.0 technology on its official website. We know that Nvidia is using a system called the Neural Graphics Framework or NGX, which uses the ability of an NGX-powered supercomputer to learn and get better at AI computations. DLSS 2.0 has two primary inputs into the AI network:
- Low resolution, aliased images rendered by the game engine
- Low resolution, motion vectors from the same images — also generated by the game engine
Nvidia then uses a process known as temporal feedback to “estimate” what the frame will look like. Then, a special type of AI autoencoder takes the low-resolution current frame, and the high-resolution previous frame to determine on a pixel-by-pixel basis how to generate a higher quality current frame. Nvidia is also simultaneously taking steps to improve the supercomputer’s understanding of the process:
During the training process, the output image is compared to an offline rendered, ultra-high quality 16K reference image, and the difference is communicated back into the network so that it can continue to learn and improve its results. This process is repeated tens of thousands of times on the supercomputer until the network reliably outputs high quality, high-resolution images.
Once the network is trained, NGX delivers the AI model to your GeForce RTX PC or laptop via Game Ready Drivers and OTA updates. With Turing’s Tensor Cores delivering up to 110 teraflops of dedicated AI horsepower, the DLSS network can be run in real-time simultaneously with an intensive 3D game. This simply wasn’t possible before Turing and Tensor Cores.
Support
DLSS is a relatively new technology that is still in its infancy. While more and more games are starting to support this feature, there is still a huge catalog of older games that probably will never support it. However, we can expect huge investment in DLSS and Raytracing moving forward since both Nvidia and AMD now have support for these features (AMD is supposed to announce a DLSS competitor soon), as well as the next-gen consoles, the PlayStation 5 and the Xbox Series X.
Recently with the release of the RTX 3000 series, Nvidia has expanded their catalog of games that support this feature. DLSS 2.0 is now coming to Cyberpunk 2077, Call of Duty: Black Ops Cold War, Fortnite, Watch Dogs Legion, Boundary, and Bright Memory: Infinite. Other notable titles that already have support for DLSS 2.0 include Death Stranding, Anthem, F1 2020, Control, Deliver Us The Moon, MechWarrior 5, and Wolfenstein: Youngblood.
While this library is not gigantic by any means, one should keep in mind the future potential of a technology as impressive as DLSS. With its massive performance improvement and diverse feature set, DLSS can be the centerpiece of gaming in the near future, especially with groundbreaking technologies like Raytracing pushing to the forefront. Nvidia also claims that its DLSS technology continues to learn and improve through AI which is a good thing for all PC gamers eager to enjoy stunning visuals at high framerates.
Conclusion
DLSS or Deep Learning Super Sampling is an incredibly impressive technology developed by Nvidia. It delivers a big performance improvement over traditional native rendering, while not compromising at all on image quality. This is possible through extensive work in the fields of AI and deep learning by Nvidia.
Harnessing the power of the RTX series of graphics cards, DLSS can provide almost indistinguishable image quality to native resolution, while providing a big framerate bump that can make Raytracing and higher resolutions like 4K playable. DLSS continues to expand its library of supported games, and we hope that it continues to get better as well so that gamers can enjoy the visuals they love at the framerates they desire.
