Intel could be the third major PC player to debut its frame generation technology for its XeSS framework known as ExtraSS as showcased during SIGGRAPH Asia 2023 in Sydney.
Intel XeSS To Join The Frame Generation Bandwagon With ExtraSS Frame Extrapolation Tech, Competing Against NVIDIA DLSS 3 and AMD FSR 3
Intel's XeSS technology is an upscaling technology that is hardware agnostic, meaning that it works with GPUs from NVIDIA and AMD too. It is also often comparable to NVIDIA's DLSS upscaling tech & sits ahead of AMD's FSR. Like NVIDIA's DLSS, Intel's XeSS leverages AI accelerators to deliver better image quality than other upscaling methods and the results speak for themselves which can be seen in the several XeSS titles that have been released to date. XeSS is also an open-source tech which means that it can easily be integrated within games and almost all games that feature XeSS support other upscaling methods too such as FSR and DLSS.
However, NVIDIA's DLSS and AMD's FSR technologies have further been extended with support for frame generation, a method that inserts a frame using interpolation technology. This can be used to boost the FPS while maintaining good image quality. NVIDIA's Frame Generation techniques are getting updated with DLSS 3.5 further enhancing the image quality while AMD just released its new FSR 3.03 update which addresses frame pacing issues and also offers more fidelity than before.
This is one area where Intel XeSS has so far lacked but it looks like Intel has its solution in the works as an extension to XeSS known as ExtraSS. In the "ExtraSS: A Framework for Joint Spatial Super Sampling and Frame Extrapolation" presentation, Intel's VP of graphics research, Anton Kaplanyan, & various other authors propose the idea of frame extrapolation to be used for frame generation. The description of the presentation is listed below:
We introduce ExtraSS, a novel framework that combines spatial super sampling and frame extrapolation to enhance real-time rendering performance. By integrating these techniques, our approach achieves a balance between performance and quality, generating temporally stable and high-quality, high-resolution results.
Leveraging lightweight modules on warping and the ExtraSSNet for refinement, we exploit spatial-temporal information, improve rendering sharpness, handle moving shadings accurately, and generate temporally stable results. Computational costs are significantly reduced compared to traditional rendering methods, enabling higher frame rates and alias-free high resolution results.
Evaluation using Unreal Engine demonstrates the benefits of our framework over conventional individual spatial or temporal super sampling methods, delivering improved rendering speed and visual quality. With its ability to generate temporally stable high-quality results, our framework creates new possibilities for real-time rendering applications, advancing the boundaries of performance and photo-realistic rendering in various domains.
What's interesting about Intel's XeSS "ExtraSS" is that the frame generation technique being used is "Frame Extrapolation" instead of "Frame Interpolation". We know that both NVIDIA's DLSS 3 & AMD FSR 3 used "Frame Interpolation" to generate and insert frames within a specific scene, offering higher FPS. Both Interpolation and Extrapolation methods are pretty much the same but as per BlurBusters, the main difference lies in how the generated frame is produced.
- NVIDIA DLSS Frame-Gen - Interpolation
- AMD FSR Frame-Gen - Interpolation
- Intel XeSS Frame-Gen - Extrapolation?
The Interpolation method uses multiple samples to produce an approximation of the frame that is to be inserted whereas the Extrapolation method uses information beyond the bounds of the input sample to produce an approximation of the frame. It is stated that Extrapolation might produce less reliable results, and add more artifacts but we have seen similar issues with interpolation in the start so with a few tweaks and optimizations, XeSS "ExtraSS" may find a middle ground in offering good quality with higher FPS.
The research paper itself also highlights the differences between Interpolation and Extrapolation. It says that while Frame Interpolation generates better results, it also introduces higher latency when generating frames which is why NVIDIA and AMD have latency-reducing technologies such as Reflex and Anti-Lag required to deliver a smooth frame-generation experience. Extrapolation on the other hand doesn't produce very high latency but has difficulties due to lacking key information to produce a new frame. XeSS "ExtraSS" aims to solve this by using a new warping method that helps produce better qualities versus the previous frame generation methods and with lower latencies.
Frame extrapolation is another way to increase the framerate by only using the information from prior frames. Li et al.[2022] proposed an optical flow-based method to predict flow based on previous flows and then warp the current frame to the next frame. ExtraNet [Guo et al .2021] uses occlusion motion vectors with neural networks to handle dis-occluded areas and shading changes with G-buffers information. Their methods fail when the scene becomes complex and generate artifacts in the disoccluded areas. Furthermore, it requires higher resolution inputs since they only generate new frames. We are the first ones to propose a joint framework to solve both spatial supersampling and frame extrapolation together while staying efficient and high quality.
Interpolation vs. Extrapolation
Frame interpolation and extrapolation are two key methods of Temporal Super Sampling. Usually frame interpolation generates better results but also brings latency when generating the frames. Note that there are some existing methods such as NVIDIA Reflex [NVIDIA2020 ]decreasing the latency byusing a better scheduler for the inputs, but they cannot avoid the latency introduced from the frame interpolation and is orthogonal to the interpolation and extrapolation methods.
The interpolation methods still have larger latency even with those techniques. Frame extrapolation has less latency but has difficulty handling the disoccluded areas because of lacking information from the input frames. Our method proposes a new warping method with a lightweight flow model to extrapolate frames with better qualities to the previous frame generation methods and less latency comparing to interpolation based methods.
Intel Research Paper (ExtraSS: A Framework for Joint Spatial Super Sampling and Frame Extrapolation)
You can also note some comparisons between Intel's XeSS "ExtraSS" Frame Extrapolation method against other aliasing technologies such as SSAA, TAA, and neural rendering frameworks such as ExtraNet and IFRnet below:
Comparison #1:
Comparison #2:
In some render time performance tests, Intel showcased a system running an AMD Ryzen 9 5950X CPU with an NVIDIA GeForce RTX 3090 GPU. The RTX 3090 GPU was also running the same Intel XeSS Frame-Generation (Extrapolation) method which means that this would be the second frame-gen technology besides AMD's FSR 3 to feature support across all vendors which once again shows the commitment of Intel being open-source friendly.
It's great news for the PC gaming segment that Intel is also working on its Frame Generation tech for games. So far, it looks like the technology is still some time away from introduction but we may get to hear more about it in 2024. As we have seen with XeSS, it was late compared to others but in terms of quality and support, it was worth the wait. So Intel might have some surprises for us when they introduce their own XeSS "ExtraSS" Frame Generation technology. We are looking forward to it.
