Showing 1–27 of 27 results for author: Huang, D Z

Fisher-Rao Gradient Flow: Geodesic Convexity and Functional Inequalities

Authors: José A. Carrillo, Yifan Chen, Daniel Zhengyu Huang, Jiaoyang Huang, Dongyi Wei

Abstract: The dynamics of probability density functions has been extensively studied in science and engineering to understand physical phenomena and facilitate algorithmic design. Of particular interest are dynamics that can be formulated as gradient flows of energy functionals under the Wasserstein metric. The development of functional inequalities, such as the log-Sobolev inequality, plays a pivotal role… ▽ More The dynamics of probability density functions has been extensively studied in science and engineering to understand physical phenomena and facilitate algorithmic design. Of particular interest are dynamics that can be formulated as gradient flows of energy functionals under the Wasserstein metric. The development of functional inequalities, such as the log-Sobolev inequality, plays a pivotal role in analyzing the convergence of these dynamics. The goal of this paper is to parallel the success of techniques using functional inequalities, for dynamics that are gradient flows under the Fisher-Rao metric, with various $f$-divergences as energy functionals. Such dynamics take the form of a nonlocal differential equation, for which existing analysis critically relies on using the explicit solution formula in special cases. We provide a comprehensive study on functional inequalities and the relevant geodesic convexity for Fisher-Rao gradient flows under minimal assumptions. A notable feature of the obtained functional inequalities is that they do not depend on the log-concavity or log-Sobolev constants of the target distribution. Consequently, the convergence rate of the dynamics (assuming well-posed) is uniform across general target distributions, making them potentially desirable dynamics for posterior sampling applications in Bayesian inference. △ Less

Submitted 22 July, 2024; originally announced July 2024.

Comments: 38 pages

Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows

Authors: Yifan Chen, Daniel Zhengyu Huang, Jiaoyang Huang, Sebastian Reich, Andrew M. Stuart

Abstract: In this paper, we study efficient approximate sampling for probability distributions known up to normalization constants. We specifically focus on a problem class arising in Bayesian inference for large-scale inverse problems in science and engineering applications. The computational challenges we address with the proposed methodology are: (i) the need for repeated evaluations of expensive forward… ▽ More In this paper, we study efficient approximate sampling for probability distributions known up to normalization constants. We specifically focus on a problem class arising in Bayesian inference for large-scale inverse problems in science and engineering applications. The computational challenges we address with the proposed methodology are: (i) the need for repeated evaluations of expensive forward models; (ii) the potential existence of multiple modes; and (iii) the fact that gradient of, or adjoint solver for, the forward model might not be feasible. While existing Bayesian inference methods meet some of these challenges individually, we propose a framework that tackles all three systematically. Our approach builds upon the Fisher-Rao gradient flow in probability space, yielding a dynamical system for probability densities that converges towards the target distribution at a uniform exponential rate. This rapid convergence is advantageous for the computational burden outlined in (i). We apply Gaussian mixture approximations with operator splitting techniques to simulate the flow numerically; the resulting approximation can capture multiple modes thus addressing (ii). Furthermore, we employ the Kalman methodology to facilitate a derivative-free update of these Gaussian components and their respective weights, addressing the issue in (iii). The proposed methodology results in an efficient derivative-free sampler flexible enough to handle multi-modal distributions: Gaussian Mixture Kalman Inversion (GMKI). The effectiveness of GMKI is demonstrated both theoretically and numerically in several experiments with multimodal target distributions, including proof-of-concept and two-dimensional examples, as well as a large-scale application: recovering the Navier-Stokes initial condition from solution data at positive times. △ Less

Submitted 25 June, 2024; originally announced June 2024.

Comments: 42 pages, 9 figures

Convergence Analysis of Probability Flow ODE for Score-based Generative Models

Authors: Daniel Zhengyu Huang, Jiaoyang Huang, Zhengjiang Lin

Abstract: Score-based generative models have emerged as a powerful approach for sampling high-dimensional probability distributions. Despite their effectiveness, their theoretical underpinnings remain relatively underdeveloped. In this work, we study the convergence properties of deterministic samplers based on probability flow ODEs from both theoretical and numerical perspectives. Assuming access to $L^2$-… ▽ More Score-based generative models have emerged as a powerful approach for sampling high-dimensional probability distributions. Despite their effectiveness, their theoretical underpinnings remain relatively underdeveloped. In this work, we study the convergence properties of deterministic samplers based on probability flow ODEs from both theoretical and numerical perspectives. Assuming access to $L^2$-accurate estimates of the score function, we prove the total variation between the target and the generated data distributions can be bounded above by $\mathcal{O}(d^{3/4}δでるた^{1/2})$ in the continuous time level, where $d$ denotes the data dimension and $δでるた$ represents the $L^2$-score matching error. For practical implementations using a $p$-th order Runge-Kutta integrator with step size $h$, we establish error bounds of $\mathcal{O}(d^{3/4}δでるた^{1/2} + d\cdot(dh)^p)$ at the discrete level. Finally, we present numerical studies on problems up to 128 dimensions to verify our theory. △ Less

Submitted 21 July, 2024; v1 submitted 15 April, 2024; originally announced April 2024.

Comments: 37 pages, 7 figures

An operator learning perspective on parameter-to-observable maps

Authors: Daniel Zhengyu Huang, Nicholas H. Nelsen, Margaret Trautner

Abstract: Computationally efficient surrogates for parametrized physical models play a crucial role in science and engineering. Operator learning provides data-driven surrogates that map between function spaces. However, instead of full-field measurements, often the available data are only finite-dimensional parametrizations of model inputs or finite observables of model outputs. Building on Fourier Neural… ▽ More Computationally efficient surrogates for parametrized physical models play a crucial role in science and engineering. Operator learning provides data-driven surrogates that map between function spaces. However, instead of full-field measurements, often the available data are only finite-dimensional parametrizations of model inputs or finite observables of model outputs. Building on Fourier Neural Operators, this paper introduces the Fourier Neural Mappings (FNMs) framework that is able to accommodate such finite-dimensional vector inputs or outputs. The paper develops universal approximation theorems for the method. Moreover, in many applications the underlying parameter-to-observable (PtO) map is defined implicitly through an infinite-dimensional operator, such as the solution operator of a partial differential equation. A natural question is whether it is more data-efficient to learn the PtO map end-to-end or first learn the solution operator and subsequently compute the observable from the full-field solution. A theoretical analysis of Bayesian nonparametric regression of linear functionals, which is of independent interest, suggests that the end-to-end approach can actually have worse sample complexity. Extending beyond the theory, numerical results for the FNM approximation of three nonlinear PtO maps demonstrate the benefits of the operator learning perspective that this paper adopts. △ Less

Submitted 6 June, 2024; v1 submitted 8 February, 2024; originally announced February 2024.

Comments: 63 pages, 10 figures, 1 table

MSC Class: 68T07; 62G20; 65J15

SPFNO: Spectral operator learning for PDEs with Dirichlet and Neumann boundary conditions

Authors: Ziyuan Liu, Yuhang Wu, Daniel Zhengyu Huang, Hong Zhang, Xu Qian, Songhe Song

Abstract: Neural operators have been validated as promising deep surrogate models for solving partial differential equations (PDEs). Despite the critical role of boundary conditions in PDEs, however, only a limited number of neural operators robustly enforce these conditions. In this paper we introduce semi-periodic Fourier neural operator (SPFNO), a novel spectral operator learning method, to learn the tar… ▽ More Neural operators have been validated as promising deep surrogate models for solving partial differential equations (PDEs). Despite the critical role of boundary conditions in PDEs, however, only a limited number of neural operators robustly enforce these conditions. In this paper we introduce semi-periodic Fourier neural operator (SPFNO), a novel spectral operator learning method, to learn the target operators of PDEs with non-periodic BCs. This method extends our previous work (arXiv:2206.12698), which showed significant improvements by employing enhanced neural operators that precisely satisfy the boundary conditions. However, the previous work is associated with Gaussian grids, restricting comprehensive comparisons across most public datasets. Additionally, we present numerical results for various PDEs such as the viscous Burgers' equation, Darcy flow, incompressible pipe flow, and coupled reactiondiffusion equations. These results demonstrate the computational efficiency, resolution invariant property, and BC-satisfaction behavior of proposed model. An accuracy improvement of approximately 1.7X-4.7X over the non-BC-satisfying baselines is also achieved. Furthermore, our studies on SOL underscore the significance of satisfying BCs as a criterion for deep surrogate models of PDEs. △ Less

Submitted 11 December, 2023; originally announced December 2023.

Sampling via Gradient Flows in the Space of Probability Measures

Authors: Yifan Chen, Daniel Zhengyu Huang, Jiaoyang Huang, Sebastian Reich, Andrew M Stuart

Abstract: Sampling a target probability distribution with an unknown normalization constant is a fundamental challenge in computational science and engineering. Recent work shows that algorithms derived by considering gradient flows in the space of probability measures open up new avenues for algorithm development. This paper makes three contributions to this sampling approach by scrutinizing the design com… ▽ More Sampling a target probability distribution with an unknown normalization constant is a fundamental challenge in computational science and engineering. Recent work shows that algorithms derived by considering gradient flows in the space of probability measures open up new avenues for algorithm development. This paper makes three contributions to this sampling approach by scrutinizing the design components of such gradient flows. Any instantiation of a gradient flow for sampling needs an energy functional and a metric to determine the flow, as well as numerical approximations of the flow to derive algorithms. Our first contribution is to show that the Kullback-Leibler divergence, as an energy functional, has the unique property (among all f-divergences) that gradient flows resulting from it do not depend on the normalization constant of the target distribution. Our second contribution is to study the choice of metric from the perspective of invariance. The Fisher-Rao metric is known as the unique choice (up to scaling) that is diffeomorphism invariant. As a computationally tractable alternative, we introduce a relaxed, affine invariance property for the metrics and gradient flows. In particular, we construct various affine invariant Wasserstein and Stein gradient flows. Affine invariant gradient flows are shown to behave more favorably than their non-affine-invariant counterparts when sampling highly anisotropic distributions, in theory and by using particle methods. Our third contribution is to study, and develop efficient algorithms based on Gaussian approximations of the gradient flows; this leads to an alternative to particle methods. We establish connections between various Gaussian approximate gradient flows, discuss their relation to gradient methods arising from parametric variational inference, and study their convergence properties both theoretically and numerically. △ Less

Submitted 9 March, 2024; v1 submitted 5 October, 2023; originally announced October 2023.

Comments: Related and text overlap with arXiv:2302.11024

AI-aided Geometric Design of Anti-infection Catheters

Authors: Tingtao Zhou, Xuan Wan, Daniel Zhengyu Huang, Zongyi Li, Zhiwei Peng, Anima Anandkumar, John F. Brady, Paul W. Sternberg, Chiara Daraio

Abstract: Bacteria can swim upstream due to hydrodynamic interactions with the fluid flow in a narrow tube, and pose a clinical threat of urinary tract infection to patients implanted with catheters. Coatings and structured surfaces have been proposed as a way to suppress bacterial contamination in catheters. However, there is no surface structuring or coating approach to date that thoroughly addresses the… ▽ More Bacteria can swim upstream due to hydrodynamic interactions with the fluid flow in a narrow tube, and pose a clinical threat of urinary tract infection to patients implanted with catheters. Coatings and structured surfaces have been proposed as a way to suppress bacterial contamination in catheters. However, there is no surface structuring or coating approach to date that thoroughly addresses the contamination problem. Here, based on the physical mechanism of upstream swimming, we propose a novel geometric design, optimized by an AI model predicting in-flow bacterial dynamics. The AI method, based on Fourier neural operator, offers significant speedups over traditional simulation methods. Using Escherichia coli, we demonstrate the anti-infection mechanism in quasi-2D micro-fluidic experiments and evaluate the effectiveness of the design in 3Dprinted prototype catheters under clinical flow rates. Our catheter design shows 1-2 orders of magnitude improved suppression of bacterial contamination at the upstream end of the catheter, potentially prolonging the in-dwelling time for catheter use and reducing the overall risk of catheter-associated urinary tract infections. △ Less

Submitted 27 April, 2023; originally announced April 2023.

Comments: maint text 4 figures, SI 5 figures

Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance

Authors: Yifan Chen, Daniel Zhengyu Huang, Jiaoyang Huang, Sebastian Reich, Andrew M. Stuart

Abstract: Sampling a probability distribution with an unknown normalization constant is a fundamental problem in computational science and engineering. This task may be cast as an optimization problem over all probability measures, and an initial distribution can be evolved to the desired minimizer dynamically via gradient flows. Mean-field models, whose law is governed by the gradient flow in the space of… ▽ More Sampling a probability distribution with an unknown normalization constant is a fundamental problem in computational science and engineering. This task may be cast as an optimization problem over all probability measures, and an initial distribution can be evolved to the desired minimizer dynamically via gradient flows. Mean-field models, whose law is governed by the gradient flow in the space of probability measures, may also be identified; particle approximations of these mean-field models form the basis of algorithms. The gradient flow approach is also the basis of algorithms for variational inference, in which the optimization is performed over a parameterized family of probability distributions such as Gaussians, and the underlying gradient flow is restricted to the parameterized family. By choosing different energy functionals and metrics for the gradient flow, different algorithms with different convergence properties arise. In this paper, we concentrate on the Kullback-Leibler divergence after showing that, up to scaling, it has the unique property that the gradient flows resulting from this choice of energy do not depend on the normalization constant. For the metrics, we focus on variants of the Fisher-Rao, Wasserstein, and Stein metrics; we introduce the affine invariance property for gradient flows, and their corresponding mean-field models, determine whether a given metric leads to affine invariance, and modify it to make it affine invariant if it does not. We study the resulting gradient flows in both probability density space and Gaussian space. The flow in the Gaussian space may be understood as a Gaussian approximation of the flow. We demonstrate that the Gaussian approximation based on the metric and through moment closure coincide, establish connections between them, and study their long-time convergence properties showing the advantages of affine invariance. △ Less

Submitted 2 November, 2023; v1 submitted 21 February, 2023; originally announced February 2023.

Comments: 82 pages, 8 figures (Welcome any feedback!)

Bayesian Spline Learning for Equation Discovery of Nonlinear Dynamics with Quantified Uncertainty

Authors: Luning Sun, Daniel Zhengyu Huang, Hao Sun, Jian-Xun Wang

Abstract: Nonlinear dynamics are ubiquitous in science and engineering applications, but the physics of most complex systems is far from being fully understood. Discovering interpretable governing equations from measurement data can help us understand and predict the behavior of complex dynamic systems. Although extensive work has recently been done in this field, robustly distilling explicit model forms fr… ▽ More Nonlinear dynamics are ubiquitous in science and engineering applications, but the physics of most complex systems is far from being fully understood. Discovering interpretable governing equations from measurement data can help us understand and predict the behavior of complex dynamic systems. Although extensive work has recently been done in this field, robustly distilling explicit model forms from very sparse data with considerable noise remains intractable. Moreover, quantifying and propagating the uncertainty of the identified system from noisy data is challenging, and relevant literature is still limited. To bridge this gap, we develop a novel Bayesian spline learning framework to identify parsimonious governing equations of nonlinear (spatio)temporal dynamics from sparse, noisy data with quantified uncertainty. The proposed method utilizes spline basis to handle the data scarcity and measurement noise, upon which a group of derivatives can be accurately computed to form a library of candidate model terms. The equation residuals are used to inform the spline learning in a Bayesian manner, where approximate Bayesian uncertainty calibration techniques are employed to approximate posterior distributions of the trainable parameters. To promote the sparsity, an iterative sequential-threshold Bayesian learning approach is developed, using the alternative direction optimization strategy to systematically approximate L0 sparsity constraints. The proposed algorithm is evaluated on multiple nonlinear dynamical systems governed by canonical ordinary and partial differential equations, and the merit/superiority of the proposed method is demonstrated by comparison with state-of-the-art methods. △ Less

Submitted 14 October, 2022; originally announced October 2022.

Comments: 28 pages, 11 figures

Fourier Neural Operator with Learned Deformations for PDEs on General Geometries

Authors: Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, Anima Anandkumar

Abstract: Deep learning surrogate models have shown promise in solving partial differential equations (PDEs). Among them, the Fourier neural operator (FNO) achieves good accuracy, and is significantly faster compared to numerical solvers, on a variety of PDEs, such as fluid flows. However, the FNO uses the Fast Fourier transform (FFT), which is limited to rectangular domains with uniform grids. In this work… ▽ More Deep learning surrogate models have shown promise in solving partial differential equations (PDEs). Among them, the Fourier neural operator (FNO) achieves good accuracy, and is significantly faster compared to numerical solvers, on a variety of PDEs, such as fluid flows. However, the FNO uses the Fast Fourier transform (FFT), which is limited to rectangular domains with uniform grids. In this work, we propose a new framework, viz., geo-FNO, to solve PDEs on arbitrary geometries. Geo-FNO learns to deform the input (physical) domain, which may be irregular, into a latent space with a uniform grid. The FNO model with the FFT is applied in the latent space. The resulting geo-FNO model has both the computation efficiency of FFT and the flexibility of handling arbitrary geometries. Our geo-FNO is also flexible in terms of its input formats, viz., point clouds, meshes, and design parameters are all valid inputs. We consider a variety of PDEs such as the Elasticity, Plasticity, Euler's, and Navier-Stokes equations, and both forward modeling and inverse design problems. Geo-FNO is $10^5$ times faster than the standard numerical solvers and twice more accurate compared to direct interpolation on existing ML-based PDE solvers such as the standard FNO. △ Less

Submitted 2 May, 2024; v1 submitted 11 July, 2022; originally announced July 2022.

Journal ref: Journal of Machine Learning Research (2023) Volume 24, Issue 1, Article No. 388, pp 18593-18618

Efficient Derivative-free Bayesian Inference for Large-Scale Inverse Problems

Authors: Daniel Zhengyu Huang, Jiaoyang Huang, Sebastian Reich, Andrew M. Stuart

Abstract: We consider Bayesian inference for large scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model. This renders most Markov chain Monte Carlo approaches infeasible, since they typically require $O(10^4)$ model runs, or more. Moreover, the forward model is often given as a black box or is impractical to differentiate. Therefor… ▽ More We consider Bayesian inference for large scale inverse problems, where computational challenges arise from the need for repeated evaluations of an expensive forward model. This renders most Markov chain Monte Carlo approaches infeasible, since they typically require $O(10^4)$ model runs, or more. Moreover, the forward model is often given as a black box or is impractical to differentiate. Therefore derivative-free algorithms are highly desirable. We propose a framework, which is built on Kalman methodology, to efficiently perform Bayesian inference in such inverse problems. The basic method is based on an approximation of the filtering distribution of a novel mean-field dynamical system into which the inverse problem is embedded as an observation operator. Theoretical properties of the mean-field model are established for linear inverse problems, demonstrating that the desired Bayesian posterior is given by the steady state of the law of the filtering distribution of the mean-field dynamical system, and proving exponential convergence to it. This suggests that, for nonlinear problems which are close to Gaussian, sequentially computing this law provides the basis for efficient iterative methods to approximate the Bayesian posterior. Ensemble methods are applied to obtain interacting particle system approximations of the filtering distribution of the mean-field model; and practical strategies to further reduce the computational and memory cost of the methodology are presented, including low-rank approximation and a bi-fidelity approach. The effectiveness of the framework is demonstrated in several numerical experiments, including proof-of-concept linear/nonlinear examples and two large-scale applications: learning of permeability parameters in subsurface flow; and learning subgrid-scale parameters in a global climate model from time-averaged statistics. △ Less

Submitted 11 August, 2022; v1 submitted 9 April, 2022; originally announced April 2022.

Comments: 44 pages, 15 figures

The Cost-Accuracy Trade-Off In Operator Learning With Neural Networks

Authors: Maarten V. de Hoop, Daniel Zhengyu Huang, Elizabeth Qian, Andrew M. Stuart

Abstract: The term `surrogate modeling' in computational science and engineering refers to the development of computationally efficient approximations for expensive simulations, such as those arising from numerical solution of partial differential equations (PDEs). Surrogate modeling is an enabling methodology for many-query computations in science and engineering, which include iterative methods in optimiz… ▽ More The term `surrogate modeling' in computational science and engineering refers to the development of computationally efficient approximations for expensive simulations, such as those arising from numerical solution of partial differential equations (PDEs). Surrogate modeling is an enabling methodology for many-query computations in science and engineering, which include iterative methods in optimization and sampling methods in uncertainty quantification. Over the last few years, several approaches to surrogate modeling for PDEs using neural networks have emerged, motivated by successes in using neural networks to approximate nonlinear maps in other areas. In principle, the relative merits of these different approaches can be evaluated by understanding, for each one, the cost required to achieve a given level of accuracy. However, the absence of a complete theory of approximation error for these approaches makes it difficult to assess this cost-accuracy trade-off. The purpose of the paper is to provide a careful numerical study of this issue, comparing a variety of different neural network architectures for operator approximation across a range of problems arising from PDE models in continuum mechanics. △ Less

Submitted 11 August, 2022; v1 submitted 24 March, 2022; originally announced March 2022.

Comments: 48 pages, 19 figures

Long Random Matrices and Tensor Unfolding

Authors: Gérard Ben Arous, Daniel Zhengyu Huang, Jiaoyang Huang

Abstract: In this paper, we consider the singular values and singular vectors of low rank perturbations of large rectangular random matrices, in the regime the matrix is "long": we allow the number of rows (columns) to grow polynomially in the number of columns (rows). We prove there exists a critical signal-to-noise ratio (depending on the dimensions of the matrix), and the extreme singular values and sing… ▽ More In this paper, we consider the singular values and singular vectors of low rank perturbations of large rectangular random matrices, in the regime the matrix is "long": we allow the number of rows (columns) to grow polynomially in the number of columns (rows). We prove there exists a critical signal-to-noise ratio (depending on the dimensions of the matrix), and the extreme singular values and singular vectors exhibit a BBP type phase transition. As a main application, we investigate the tensor unfolding algorithm for the asymmetric rank-one spiked tensor model, and obtain an exact threshold, which is independent of the procedure of tensor unfolding. If the signal-to-noise ratio is above the threshold, tensor unfolding detects the signals; otherwise, it fails to capture the signals. △ Less

Submitted 19 October, 2021; originally announced October 2021.

Comments: 29 pages, 4 figures

Bayesian Calibration for Large-Scale Fluid Structure Interaction Problems Under Embedded/Immersed Boundary Framework

Authors: Shunxiang Cao, Daniel Zhengyu Huang

Abstract: Bayesian calibration is widely used for inverse analysis and uncertainty analysis for complex systems in the presence of both computer models and observation data. In the present work, we focus on large-scale fluid-structure interaction systems characterized by large structural deformations. Numerical methods to solve these problems, including embedded/immersed boundary methods, are typically not… ▽ More Bayesian calibration is widely used for inverse analysis and uncertainty analysis for complex systems in the presence of both computer models and observation data. In the present work, we focus on large-scale fluid-structure interaction systems characterized by large structural deformations. Numerical methods to solve these problems, including embedded/immersed boundary methods, are typically not differentiable and lack smoothness. We propose a framework that is built on unscented Kalman filter/inversion to efficiently calibrate and provide uncertainty estimations of such complicated models with noisy observation data. The approach is derivative-free and non-intrusive, and is of particular value for the forward model that is computationally expensive and provided as a black box which is impractical to differentiate. The framework is demonstrated and validated by successfully calibrating the model parameters of a piston problem and identifying the damage field of an airfoil under transonic buffeting. △ Less

Submitted 30 December, 2021; v1 submitted 19 May, 2021; originally announced May 2021.

Comments: 24pages, 14 figures

Unscented Kalman Inversion: Efficient Gaussian Approximation to the Posterior Distribution

Authors: Daniel Z. Huang, Jiaoyang Huang

Abstract: The unscented Kalman inversion (UKI) method presented in [1] is a general derivative-free approach for the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategies, convergence property, and speed-up strategies [1,2] of the UKI are thoroughly studied, and the method is capable of… ▽ More The unscented Kalman inversion (UKI) method presented in [1] is a general derivative-free approach for the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategies, convergence property, and speed-up strategies [1,2] of the UKI are thoroughly studied, and the method is capable of handling noisy observation data and solving chaotic inverse problems. In this paper, we study the uncertainty quantification capability of the UKI. We propose a modified UKI, which allows to well approximate the mean and covariance of the posterior distribution for well-posed inverse problems with large observation data. Theoretical guarantees for both linear and nonlinear inverse problems are presented. Numerical results, including learning of permeability parameters in subsurface flow and of the Navier-Stokes initial condition from solution data at positive times are presented. The results obtained by the UKI require only $O(10)$ iterations, and match well with the expected results obtained by the Markov Chain Monte Carlo method. △ Less

Submitted 21 April, 2021; v1 submitted 27 February, 2021; originally announced March 2021.

Comments: 23 pages, 10 figures

Improve Unscented Kalman Inversion With Low-Rank Approximation and Reduced-Order Model

Authors: Daniel Z. Huang, Jiaoyang Huang

Abstract: The unscented Kalman inversion (UKI) presented in [1] is a general derivative-free approach to solving the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategy and convergence property of the UKI are thoroughly studied, and the method is demonstrated effectively handling noisy o… ▽ More The unscented Kalman inversion (UKI) presented in [1] is a general derivative-free approach to solving the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategy and convergence property of the UKI are thoroughly studied, and the method is demonstrated effectively handling noisy observation data and solving chaotic inverse problems. In this paper, we aim to make the UKI more efficient in terms of computational and memory costs for large scale inverse problems. We take advantages of the low-rank covariance structure to reduce the number of forward problem evaluations and the memory cost, related to the need to propagate large covariance matrices. And we leverage reduced-order model techniques to further speed up these forward evaluations. The effectiveness of the enhanced UKI is demonstrated on a barotropic model inverse problem with O($10^5$) unknown parameters and a 3D generalized circulation model (GCM) inverse problem, where each iteration is as efficient as that of gradient-based optimization methods. △ Less

Submitted 23 February, 2021; v1 submitted 21 February, 2021; originally announced February 2021.

Comments: 27 pages, 9 figures

Iterated Kalman Methodology For Inverse Problems

Authors: Daniel Zhengyu Huang, Tapio Schneider, Andrew M. Stuart

Abstract: This paper is focused on the optimization approach to the solution of inverse problems. We introduce a stochastic dynamical system in which the parameter-to-data map is embedded, with the goal of employing techniques from nonlinear Kalman filtering to estimate the parameter given the data. The extended Kalman filter (which we refer to as ExKI in the context of inverse problems) can be effective fo… ▽ More This paper is focused on the optimization approach to the solution of inverse problems. We introduce a stochastic dynamical system in which the parameter-to-data map is embedded, with the goal of employing techniques from nonlinear Kalman filtering to estimate the parameter given the data. The extended Kalman filter (which we refer to as ExKI in the context of inverse problems) can be effective for some inverse problems approached this way, but is impractical when the forward map is not readily differentiable and is given as a black box, and also for high dimensional parameter spaces because of the need to propagate large covariance matrices. Application of ensemble Kalman filters, for example use of the ensemble Kalman inversion (EKI) algorithm, has emerged as a useful tool which overcomes both of these issues: it is derivative free and works with a low-rank covariance approximation formed from the ensemble. In this paper, we work with the ExKI, EKI, and a variant on EKI which we term unscented Kalman inversion (UKI). The paper contains two main contributions. Firstly, we identify a novel stochastic dynamical system in which the parameter-to-data map is embedded. We present theory in the linear case to show exponential convergence of the mean of the filtering distribution to the solution of a regularized least squares problem. This is in contrast to previous work in which the EKI has been employed where the dynamical system used leads to algebraic convergence to an unregularized problem. Secondly, we show that the application of the UKI to this novel stochastic dynamical system yields improved inversion results, in comparison with the application of EKI to the same novel stochastic dynamical system. △ Less

Submitted 28 April, 2022; v1 submitted 2 February, 2021; originally announced February 2021.

Comments: 56 pages, 24 figures

Power Iteration for Tensor PCA

Authors: Jiaoyang Huang, Daniel Z. Huang, Qing Yang, Guang Cheng

Abstract: In this paper, we study the power iteration algorithm for the spiked tensor model, as introduced in [44]. We give necessary and sufficient conditions for the convergence of the power iteration algorithm. When the power iteration algorithm converges, for the rank one spiked tensor model, we show the estimators for the spike strength and linear functionals of the signal are asymptotically Gaussian;… ▽ More In this paper, we study the power iteration algorithm for the spiked tensor model, as introduced in [44]. We give necessary and sufficient conditions for the convergence of the power iteration algorithm. When the power iteration algorithm converges, for the rank one spiked tensor model, we show the estimators for the spike strength and linear functionals of the signal are asymptotically Gaussian; for the multi-rank spiked tensor model, we show the estimators are asymptotically mixtures of Gaussian. This new phenomenon is different from the spiked matrix model. Using these asymptotic results of our estimators, we construct valid and efficient confidence intervals for spike strengths and linear functionals of the signals. △ Less

Submitted 25 December, 2020; originally announced December 2020.

Comments: Draft version, comments are welcome!

A Computationally Tractable Framework for Nonlinear Dynamic Multiscale Modeling of Membrane Fabric

Authors: Philip Avery, Daniel Z. Huang, Wanli He, Johanna Ehlers, Armen Derkevorkian, Charbel Farhat

Abstract: A general-purpose computational homogenization framework is proposed for the nonlinear dynamic analysis of membranes exhibiting complex microscale and/or mesoscale heterogeneity characterized by in-plane periodicity that cannot be effectively treated by a conventional method, such as woven fabrics. The framework is a generalization of the "finite element squared" (or FE2) method in which a localiz… ▽ More A general-purpose computational homogenization framework is proposed for the nonlinear dynamic analysis of membranes exhibiting complex microscale and/or mesoscale heterogeneity characterized by in-plane periodicity that cannot be effectively treated by a conventional method, such as woven fabrics. The framework is a generalization of the "finite element squared" (or FE2) method in which a localized portion of the periodic subscale structure is modeled using finite elements. The numerical solution of displacement driven problems involving this model can be adapted to the context of membranes by a variant of the Klinkel-Govindjee method[1] originally proposed for using finite strain, three-dimensional material models in beam and shell elements. This approach relies on numerical enforcement of the plane stress constraint and is enabled by the principle of frame invariance. Computational tractability is achieved by introducing a regression-based surrogate model informed by a physics-inspired training regimen in which FE$^2$ is utilized to simulate a variety of numerical experiments including uniaxial, biaxial and shear straining of a material coupon. Several alternative surrogate models are evaluated including an artificial neural network. The framework is demonstrated and validated for a realistic Mars landing application involving supersonic inflation of a parachute canopy made of woven fabric. △ Less

Submitted 27 January, 2021; v1 submitted 11 July, 2020; originally announced July 2020.

Comments: 29 pages, 12 figures

Journal ref: International Journal for Numerical Methods in Engineering, 2021

Learning Constitutive Relations using Symmetric Positive Definite Neural Networks

Authors: Kailai Xu, Daniel Z. Huang, Eric Darve

Abstract: We present the Cholesky-factored symmetric positive definite neural network (SPD-NN) for modeling constitutive relations in dynamical equations. Instead of directly predicting the stress, the SPD-NN trains a neural network to predict the Cholesky factor of a tangent stiffness matrix, based on which the stress is calculated in the incremental form. As a result of the special structure, SPD-NN weakl… ▽ More We present the Cholesky-factored symmetric positive definite neural network (SPD-NN) for modeling constitutive relations in dynamical equations. Instead of directly predicting the stress, the SPD-NN trains a neural network to predict the Cholesky factor of a tangent stiffness matrix, based on which the stress is calculated in the incremental form. As a result of the special structure, SPD-NN weakly imposes convexity on the strain energy function, satisfies time consistency for path-dependent materials, and therefore improves numerical stability, especially when the SPD-NN is used in finite element simulations. Depending on the types of available data, we propose two training methods, namely direct training for strain and stress pairs and indirect training for loads and displacement pairs. We demonstrate the effectiveness of SPD-NN on hyperelastic, elasto-plastic, and multiscale fiber-reinforced plate problems from solid mechanics. The generality and robustness of the SPD-NN make it a promising tool for a wide range of constitutive modeling applications. △ Less

Submitted 1 April, 2020; originally announced April 2020.

Comments: 31 pages, 20 figures

Modeling, simulation and validation of supersonic parachute inflation dynamics during Mars landing

Authors: Daniel Z. Huang, Philip Avery, Charbel Farhat, Jason Rabinovitch, Armen Derkevorkian, Lee D Peterson

Abstract: A high fidelity multi-physics Eulerian computational framework is presented for the simulation of supersonic parachute inflation during Mars landing. Unlike previous investigations in this area, the framework takes into account an initial folding pattern of the parachute, the flow compressibility effect on the fabric material porosity, and the interactions between supersonic fluid flows and the su… ▽ More A high fidelity multi-physics Eulerian computational framework is presented for the simulation of supersonic parachute inflation during Mars landing. Unlike previous investigations in this area, the framework takes into account an initial folding pattern of the parachute, the flow compressibility effect on the fabric material porosity, and the interactions between supersonic fluid flows and the suspension lines. Several adaptive mesh refinement (AMR)-enabled, large edge simulation (LES)-based, simulations of a full-size disk-gap-band (DGB) parachute inflating in the low-density, low-pressure, carbon dioxide (CO2) Martian atmosphere are reported. The comparison of the drag histories and the first peak forces between the simulation results and experimental data collected during the NASA Curiosity Rover's Mars atmospheric entry shows reasonable agreements. Furthermore, a rudimentary material failure analysis is performed to provide an estimate of the safety factor for the parachute decelerator system. The proposed framework demonstrates the potential of using Computational Fluid Dynamics (CFD) and Fluid-Structure Interaction (FSI)-based simulation tools for future supersonic parachute design. △ Less

Submitted 3 December, 2019; originally announced December 2019.

Comments: 24 pages, 12 figures

High-order partitioned spectral deferred correction solvers for multiphysics problems

Authors: Daniel Z. Huang, Will Pazner, Per-Olof Persson, Matthew J. Zahr

Abstract: We present an arbitrarily high-order, conditionally stable, partitioned spectral deferred correction (SDC) method for solving multiphysics problems using a sequence of pre-existing single-physics solvers. This method extends the work in [1, 2], which used implicit-explicit Runge-Kutta methods (IMEX) to build high-order, partitioned multiphysics solvers. We consider a generic multiphysics problem m… ▽ More We present an arbitrarily high-order, conditionally stable, partitioned spectral deferred correction (SDC) method for solving multiphysics problems using a sequence of pre-existing single-physics solvers. This method extends the work in [1, 2], which used implicit-explicit Runge-Kutta methods (IMEX) to build high-order, partitioned multiphysics solvers. We consider a generic multiphysics problem modeled as a system of coupled ordinary differential equations (ODEs), coupled through coupling terms that can depend on the state of each subsystem; therefore the method applies to both a semi-discretized system of partial differential equations (PDEs) or problems naturally modeled as coupled systems of ODEs. The sufficient conditions to build arbitrarily high-order partitioned SDC schemes are derived. Based on these conditions, various of partitioned SDC schemes are designed. The stability of the first-order partitioned SDC scheme is analyzed in detail on a coupled, linear model problem. We show that the scheme is conditionally stable, and under conditions on the coupling strength, the scheme can be unconditionally stable. We demonstrate the performance of the proposed partitioned solvers on several classes of multiphysics problems including a simple linear system of ODEs, advection-diffusion-reaction systems, and fluid-structure interaction problems with both incompressible and compressible flows, where we verify the design order of the SDC schemes and study various stability properties. We also directly compare the accuracy, stability, and cost of the proposed partitioned SDC solver with the partitioned IMEX method in [1, 2] on this suite of test problems. The results suggest that the high-order partitioned SDC solvers are more robust than the partitioned IMEX solvers for the numerical examples considered in this work, while the IMEX methods require fewer implicit solves. △ Less

Submitted 3 April, 2020; v1 submitted 3 September, 2019; originally announced September 2019.

Comments: 25 pages, 13 figures. arXiv admin note: text overlap with arXiv:1803.11372

Journal ref: Journal of Computational Physics 412 (2020): 109441

An Embedded Boundary Approach for Resolving the Contribution of Cable Subsystems to Fully Coupled Fluid-Structure Interaction

Authors: Daniel Z. Huang, Philip Avery, Charbel Farhat

Abstract: Cable subsystems characterized by long, slender, and flexible structural elements are featured in numerous engineering systems. In each of them, interaction between an individual cable and the surrounding fluid is inevitable. Such a Fluid-Structure Interaction (FSI) has received little attention in the literature, possibly due to the inherent complexity associated with fluid and structural semi-di… ▽ More Cable subsystems characterized by long, slender, and flexible structural elements are featured in numerous engineering systems. In each of them, interaction between an individual cable and the surrounding fluid is inevitable. Such a Fluid-Structure Interaction (FSI) has received little attention in the literature, possibly due to the inherent complexity associated with fluid and structural semi-discretizations of disparate spatial dimensions. This paper proposes an embedded boundary approach for filling this gap, where the dynamics of the cable are captured by a standard finite element representation $\mathcal C$ of its centerline, while its geometry is represented by a discrete surface $Σしぐま_h$ that is embedded in the fluid mesh. The proposed approach is built on master-slave kinematics between $\mathcal C$ and $Σしぐま_h$, a simple algorithm for computing the motion/deformation of $Σしぐま_h$ based on the dynamic state of $\mathcal C$, and an energy-conserving method for transferring to $\mathcal C$ the loads computed on $Σしぐま_h$. Its effectiveness is demonstrated for two highly nonlinear applications featuring large deformations and/or motions of a cable subsystem and turbulent flows: an aerial refueling model problem, and a challenging supersonic parachute inflation problem. The proposed approach is verified using numerical data, and validated using real flight data. △ Less

Submitted 6 November, 2019; v1 submitted 11 August, 2019; originally announced August 2019.

Comments: 23 pages, 11 figures

Homogenized Flux-Body Force Treatment of Compressible Viscous Porous Wall Boundary Conditions

Authors: Daniel Z. Huang, Man Long Wong, Sanjiva K. Lele, Charbel Farhat

Abstract: A homogenization approach is proposed for the treatment of porous wall boundary conditions in the computation of compressible viscous flows. Like any other homogenization approach, it eliminates the need for pore-resolved fluid meshes and therefore enables practical flow simulations in computational fluid domains with porous wall boundaries. Unlike alternative approaches however, it does not requi… ▽ More A homogenization approach is proposed for the treatment of porous wall boundary conditions in the computation of compressible viscous flows. Like any other homogenization approach, it eliminates the need for pore-resolved fluid meshes and therefore enables practical flow simulations in computational fluid domains with porous wall boundaries. Unlike alternative approaches however, it does not require prescribing a mass flow rate and does not introduce in the computational model a heuristic discharge coefficient. Instead, it models the inviscid flux through a porous wall surrounded by the flow as a weighted average of the inviscid flux at an impermeable surface and that through pores. It also introduces a body force term in the governing equations to account for friction loss along the pore boundaries. The source term depends on the thickness of the porous wall and the concept of an equivalent single pore. The feasibility of the latter concept is demonstrated using low-speed permeability test data for the fabric of the Mars Science Laboratory parachute canopy. The overall homogenization approach is illustrated with a series of supersonic flow computations through the same fabric and verified using supersonic, pore-resolved numerical simulations. △ Less

Submitted 11 December, 2020; v1 submitted 22 July, 2019; originally announced July 2019.

Comments: 25 pages, 16 figures

Learning Constitutive Relations from Indirect Observations Using Deep Neural Networks

Authors: Daniel Z. Huang, Kailai Xu, Charbel Farhat, Eric Darve

Abstract: We present a new approach for predictive modeling and its uncertainty quantification for mechanical systems, where coarse-grained models such as constitutive relations are derived directly from observation data. We explore the use of a neural network to represent the unknown constitutive relations, compare the neural networks with piecewise linear functions, radial basis functions, and radial basi… ▽ More We present a new approach for predictive modeling and its uncertainty quantification for mechanical systems, where coarse-grained models such as constitutive relations are derived directly from observation data. We explore the use of a neural network to represent the unknown constitutive relations, compare the neural networks with piecewise linear functions, radial basis functions, and radial basis function networks, and show that the neural network outperforms the others in certain cases. We analyze the approximation error of the neural networks using a scaling argument. The training and predicting processes in our framework combine the finite element method, automatic differentiation, and neural networks (or other function approximators). Our framework also allows uncertainty quantification in the form of confidence intervals. Numerical examples on a multiscale fiber-reinforced plate problem and a nonlinear rubbery membrane problem from solid mechanics demonstrate the effectiveness of our framework. △ Less

Submitted 25 February, 2020; v1 submitted 29 May, 2019; originally announced May 2019.

Comments: 40 pages, 21 figures

A high-order partitioned solver for general multiphysics problems and its applications in optimization

Authors: Daniel Z. Huang, Per-Olof Persson, Matthew J. Zahr

Abstract: A high-order accurate adjoint-based optimization framework is presented for unsteady multiphysics problems. The fully discrete adjoint solver relies on the high-order, linearly stable, partitioned solver introduced in [1], where different subsystems are modeled and discretized separately. The coupled system of semi-discretized ordinary differential equations is taken as a monolithic system and par… ▽ More A high-order accurate adjoint-based optimization framework is presented for unsteady multiphysics problems. The fully discrete adjoint solver relies on the high-order, linearly stable, partitioned solver introduced in [1], where different subsystems are modeled and discretized separately. The coupled system of semi-discretized ordinary differential equations is taken as a monolithic system and partitioned using an implicit-explicit Runge-Kutta (IMEX-RK) discretization [2]. Quantities of interest (QoI) that take the form of space-time integrals are discretized in a solver-consistent manner. The corresponding adjoint equations are derived to compute exact gradients of QoI, which can be solved in a partitioned manner, i.e. subsystem-by-subsystem and substage-by-substage, thanks to the partitioned primal solver. These quantities of interest and their gradients are then used in the context of gradient-based PDE-constrained optimization. The present optimization framework is applied to two fluid-structure interaction problems: 1D piston problem with a three-field formulation and a 2D energy harvesting problem with a two-field formulation. △ Less

Submitted 26 December, 2018; originally announced December 2018.

Comments: 20 pages, 8 figures. arXiv admin note: substantial text overlap with arXiv:1803.11372

High-order, linearly stable, partitioned solvers for general multiphysics problems based on implicit-explicit Runge-Kutta schemes

Authors: Daniel Z. Huang, Per-Olof Persson, Matthew J. Zahr

Abstract: This work introduces a general framework for constructing high-order, linearly stable, partitioned solvers for multiphysics problems from a monolithic implicit-explicit Runge-Kutta (IMEX-RK) discretization of the semi-discrete equations. The generic multiphysics problem is modeled as a system of n systems of partial differential equations where the ith subsystem is coupled to the other subsystems… ▽ More This work introduces a general framework for constructing high-order, linearly stable, partitioned solvers for multiphysics problems from a monolithic implicit-explicit Runge-Kutta (IMEX-RK) discretization of the semi-discrete equations. The generic multiphysics problem is modeled as a system of n systems of partial differential equations where the ith subsystem is coupled to the other subsystems through a coupling term that can depend on the state of all the other subsystems. This coupled system of partial differential equations reduces to a coupled system of ordinary differential equations via the method of lines where an appropriate spatial discretization is applied to each subsystem. The coupled system of ordinary differential equations is taken as a monolithic system and discretized using an IMEX-RK discretization with a specific implicit-explicit decomposition that introduces the concept of a predictor for the coupling term. We propose four coupling predictors that enable the monolithic system to be solved in a partitioned manner and preserve the IMEX-RK structure and therefore the design order of accuracy of the monolithic scheme. The four partitioned solvers that result from these predictors are high-order accurate, allow for maximum re-use of existing single-physics software, and two of the four solvers allow the subsystems to be solved in parallel at a given stage and time step. We also analyze the stability of a coupled, linear model problem and show that one of the partitioned solvers achieves unconditional linear stability, while the others are unconditionally stable only for certain values of the coupling strength. We demonstrate the performance of the proposed partitioned solvers on several classes of multiphysics problems including a simple linear system of ODEs, advection-diffusion-reaction systems, FSI problems, and particle-laden flows. △ Less

Submitted 1 September, 2018; v1 submitted 30 March, 2018; originally announced March 2018.

Comments: 36 pages, 5 tables, 15 figures