-
GFFE: G-buffer Free Frame Extrapolation for Low-latency Real-time Rendering
Authors:
Songyin Wu,
Deepak Vembar,
Anton Sochenov,
Selvakumar Panneer,
Sungye Kim,
Anton Kaplanyan,
Ling-Qi Yan
Abstract:
Real-time rendering has been embracing ever-demanding effects, such as ray tracing. However, rendering such effects in high resolution and high frame rate remains challenging. Frame extrapolation methods, which don't introduce additional latency as opposed to frame interpolation methods such as DLSS 3 and FSR 3, boost the frame rate by generating future frames based on previous frames. However, it…
▽ More
Real-time rendering has been embracing ever-demanding effects, such as ray tracing. However, rendering such effects in high resolution and high frame rate remains challenging. Frame extrapolation methods, which don't introduce additional latency as opposed to frame interpolation methods such as DLSS 3 and FSR 3, boost the frame rate by generating future frames based on previous frames. However, it is a more challenging task because of the lack of information in the disocclusion regions, and recent methods also have a high engine integration cost due to requiring G-buffers as input. We propose a \emph{G-buffer free} frame extrapolation, GFFE, with a novel heuristic framework and an efficient neural network, to plausibly generate new frames in real-time without introducing additional latency. We analyze the motion of dynamic fragments and different types of disocclusions, and design the corresponding modules of the extrapolation block to handle them. After filling disocclusions, a light-weight shading correction network is used to correct shading and improve overall quality. GFFE achieves comparable or better results compared to previous interpolation as well as G-buffer-dependent extrapolation methods, with more efficient performance and easier game integration.
△ Less
Submitted 23 May, 2024;
originally announced June 2024.
-
ENVIDR: Implicit Differentiable Renderer with Neural Environment Lighting
Authors:
Ruofan Liang,
Huiting Chen,
Chunlin Li,
Fan Chen,
Selvakumar Panneer,
Nandita Vijaykumar
Abstract:
Recent advances in neural rendering have shown great potential for reconstructing scenes from multiview images. However, accurately representing objects with glossy surfaces remains a challenge for existing methods. In this work, we introduce ENVIDR, a rendering and modeling framework for high-quality rendering and reconstruction of surfaces with challenging specular reflections. To achieve this,…
▽ More
Recent advances in neural rendering have shown great potential for reconstructing scenes from multiview images. However, accurately representing objects with glossy surfaces remains a challenge for existing methods. In this work, we introduce ENVIDR, a rendering and modeling framework for high-quality rendering and reconstruction of surfaces with challenging specular reflections. To achieve this, we first propose a novel neural renderer with decomposed rendering components to learn the interaction between surface and environment lighting. This renderer is trained using existing physically based renderers and is decoupled from actual scene representations. We then propose an SDF-based neural surface model that leverages this learned neural renderer to represent general scenes. Our model additionally synthesizes indirect illuminations caused by inter-reflections from shiny surfaces by marching surface-reflected rays. We demonstrate that our method outperforms state-of-art methods on challenging shiny scenes, providing high-quality rendering of specular reflections while also enabling material editing and scene relighting.
△ Less
Submitted 23 March, 2023;
originally announced March 2023.
-
Fast Monte Carlo Rendering via Multi-Resolution Sampling
Authors:
Qiqi Hou,
Zhan Li,
Carl S Marshall,
Selvakumar Panneer,
Feng Liu
Abstract:
Monte Carlo rendering algorithms are widely used to produce photorealistic computer graphics images. However, these algorithms need to sample a substantial amount of rays per pixel to enable proper global illumination and thus require an immense amount of computation. In this paper, we present a hybrid rendering method to speed up Monte Carlo rendering algorithms. Our method first generates two ve…
▽ More
Monte Carlo rendering algorithms are widely used to produce photorealistic computer graphics images. However, these algorithms need to sample a substantial amount of rays per pixel to enable proper global illumination and thus require an immense amount of computation. In this paper, we present a hybrid rendering method to speed up Monte Carlo rendering algorithms. Our method first generates two versions of a rendering: one at a low resolution with a high sample rate (LRHS) and the other at a high resolution with a low sample rate (HRLS). We then develop a deep convolutional neural network to fuse these two renderings into a high-quality image as if it were rendered at a high resolution with a high sample rate. Specifically, we formulate this fusion task as a super resolution problem that generates a high resolution rendering from a low resolution input (LRHS), assisted with the HRLS rendering. The HRLS rendering provides critical high frequency details which are difficult to recover from the LRHS for any super resolution methods. Our experiments show that our hybrid rendering algorithm is significantly faster than the state-of-the-art Monte Carlo denoising methods while rendering high-quality images when tested on both our own BCR dataset and the Gharbi dataset. \url{https://github.com/hqqxyy/msspl}
△ Less
Submitted 24 June, 2021;
originally announced June 2021.
-
Quantum aspects of chaos and complexity from bouncing cosmology: A study with two-mode single field squeezed state formalism
Authors:
Parth Bhargava,
Sayantan Choudhury,
Satyaki Chowdhury,
Anurag Mishara,
Sachin Panneer Selvam,
Sudhakar Panda,
Gabriel D. Pasquino
Abstract:
$Circuit~ Complexity$, a well known computational technique has recently become the backbone of the physics community to probe the chaotic behaviour and random quantum fluctuations of quantum fields. This paper is devoted to the study of out-of-equilibrium aspects and quantum chaos appearing in the universe from the paradigm of two well known bouncing cosmological solutions viz. $Cosine~ hyperboli…
▽ More
$Circuit~ Complexity$, a well known computational technique has recently become the backbone of the physics community to probe the chaotic behaviour and random quantum fluctuations of quantum fields. This paper is devoted to the study of out-of-equilibrium aspects and quantum chaos appearing in the universe from the paradigm of two well known bouncing cosmological solutions viz. $Cosine~ hyperbolic$ and $Exponential$ models of scale factors. Besides $circuit~ complexity$, we use the $Out-of-Time~ Ordered~ correlation~ (OTOC)$ functions for probing the random behaviour of the universe both at early and the late times. In particular, we use the techniques of well known two-mode squeezed state formalism in cosmological perturbation theory as a key ingredient for the purpose of our computation. To give an appropriate theoretical interpretation that is consistent with the observational perspective we use the scale factor and the number of e-foldings as a dynamical variable instead of conformal time for this computation. From this study, we found that the period of post bounce is the most interesting one. Though it may not be immediately visible, but an exponential rise can be seen in the $complexity$ once the post bounce feature is extrapolated to the present time scales. We also find within the very small acceptable error range a universal connecting relation between Complexity computed from two different kinds of cost functionals-$linearly~ weighted$ and $geodesic~ weighted$ with the OTOC. Furthermore, from the $complexity$ computation obtained from both the cosmological models and also using the well known MSS bound on quantum Lyapunov exponent, $λ\leq 2π/β$ for the saturation of chaos, we estimate the lower bound on the equilibrium temperature of our universe at late time scale. Finally, we provide a rough estimation of the scrambling time in terms of the conformal time.
△ Less
Submitted 15 September, 2021; v1 submitted 8 September, 2020;
originally announced September 2020.
