Showing 1–50 of 81 results for author: Bernstein, D

Adaptive Target Tracking Using Retrospective Cost Input Estimation

Authors: Shashank Verma, Sneha Sanjeevini, E. Dogan Sumer, Dennis S. Bernstein

Abstract: Target tracking of surrounding vehicles is essential for collision avoidance in autonomous vehicles. Our approach to target tracking is based on causal numerical differentiation on relative position data to estimate relative velocity and acceleration. Causal numerical differentiation is useful for a wide range of estimation and control problems with application to robotics and autonomous systems.… ▽ More Target tracking of surrounding vehicles is essential for collision avoidance in autonomous vehicles. Our approach to target tracking is based on causal numerical differentiation on relative position data to estimate relative velocity and acceleration. Causal numerical differentiation is useful for a wide range of estimation and control problems with application to robotics and autonomous systems. The present paper extends prior work on causal numerical differentiation based on retrospective cost input estimation (RCIE). Since the variance of the input-estimation error and its correlation with the state-estimation error (the sum of the variance and the correlation is denoted as $\widetilde{V}$) used in the Kalman filter update are unknown, the present paper considers an adaptive discrete-time Kalman filter, where $\widetilde{V}_k$ is updated at each time step $k$ to minimize the difference between the sample variance of the innovations and the variance of the innovations given by the Kalman filter. The performance of this approach is shown to reach the performance of numerical differentiation based on RCIE with the best possible fixed value of $\widetilde{V}_k$. The proposed method thus eliminates the need to determine the best possible fixed value for $\widetilde{V}_k$. Finally, RCIE with an adaptive Kalman filter is applied to target tracking of a vehicle using simulated data from CarSim. △ Less

Submitted 26 July, 2024; originally announced July 2024.

Report number: This work was done in September 2022

Adaptive Kalman Filtering Developed from Recursive Least Squares Forgetting Algorithms

Authors: Brian Lai, Dennis S. Bernstein

Abstract: Recursive least squares (RLS) is derived as the recursive minimizer of the least-squares cost function. Moreover, it is well known that RLS is a special case of the Kalman filter. This work presents the Kalman filter least squares (KFLS) cost function, whose recursive minimizer gives the Kalman filter. KFLS is an extension of generalized forgetting recursive least squares (GF-RLS), a general frame… ▽ More Recursive least squares (RLS) is derived as the recursive minimizer of the least-squares cost function. Moreover, it is well known that RLS is a special case of the Kalman filter. This work presents the Kalman filter least squares (KFLS) cost function, whose recursive minimizer gives the Kalman filter. KFLS is an extension of generalized forgetting recursive least squares (GF-RLS), a general framework which contains various extensions of RLS from the literature as special cases. This then implies that extensions of RLS are also special cases of the Kalman filter. Motivated by this connection, we propose an algorithm that combines extensions of RLS with the Kalman filter, resulting in a new class of adaptive Kalman filters. A numerical example shows that one such adaptive Kalman filter provides improved state estimation for a mass-spring-damper with intermittent, unmodeled collisions. This example suggests that such adaptive Kalman filtering may provide potential benefits for systems with non-classical disturbances. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Efficient Batch and Recursive Least Squares for Matrix Parameter Estimation

Authors: Brian Lai, Dennis S. Bernstein

Abstract: Traditionally, batch least squares (BLS) and recursive least squares (RLS) are used for identification of a vector of parameters that form a linear model. In some situations, however, it is of interest to identify parameters in a matrix structure. In this case, a common approach is to transform the problem into standard vector form using the vectorization (vec) operator and the Kronecker product,… ▽ More Traditionally, batch least squares (BLS) and recursive least squares (RLS) are used for identification of a vector of parameters that form a linear model. In some situations, however, it is of interest to identify parameters in a matrix structure. In this case, a common approach is to transform the problem into standard vector form using the vectorization (vec) operator and the Kronecker product, known as vec-permutation. However, the use of the Kronecker product introduces extraneous zero terms in the regressor, resulting in unnecessary additional computational and space requirements. This work derives matrix BLS and RLS formulations which, under mild assumptions, minimize the same cost as the vec-permutation approach. This new approach requires less computational complexity and space complexity than vec-permutation in both BLS and RLS identification. It is also shown that persistent excitation guarantees convergence to the true matrix parameters. This method can used to improve computation time in the online identification of multiple-input, multiple-output systems for indirect adaptive model predictive control. △ Less

Submitted 9 June, 2024; v1 submitted 16 April, 2024; originally announced April 2024.

Comments: Accepted to the IEEE Control Systems Letters

Convergence of Recursive Least Squares Based Input/Output System Identification with Model Order Mismatch

Authors: Brian Lai, Dennis S. Bernstein

Abstract: Discrete-time input/output models, also called infinite impulse response (IIR) models or autoregressive moving average (ARMA) models, are useful for online identification as they can be efficiently updated using recursive least squares (RLS) as new data is collected. Several works have studied the convergence of the input/output model coefficients identified using RLS under the assumption that the… ▽ More Discrete-time input/output models, also called infinite impulse response (IIR) models or autoregressive moving average (ARMA) models, are useful for online identification as they can be efficiently updated using recursive least squares (RLS) as new data is collected. Several works have studied the convergence of the input/output model coefficients identified using RLS under the assumption that the order of the identified model is the same as that of the true system. However, the case of model order mismatch is not as well addressed. This work begins by introducing the notion of \textit{equivalence} of input/output models of different orders. Next, this work analyzes online identification of input/output models in the case where the order of the identified model is higher than that of the true system. It is shown that, given persistently exciting data, the higher-order identified model converges to the model equivalent to the true system that minimizes the regularization term of RLS. △ Less

Submitted 16 April, 2024; originally announced April 2024.

SIFt-RLS: Subspace of Information Forgetting Recursive Least Squares

Authors: Brian Lai, Dennis S. Bernstein

Abstract: This paper presents subspace of information forgetting recursive least squares (SIFt-RLS), a directional forgetting algorithm which, at each step, forgets only in row space of the regressor matrix, or the \textit{information subspace}. As a result, SIFt-RLS tracks parameters that are in excited directions while not changing parameter estimation in unexcited directions. It is shown that SIFt-RLS gu… ▽ More This paper presents subspace of information forgetting recursive least squares (SIFt-RLS), a directional forgetting algorithm which, at each step, forgets only in row space of the regressor matrix, or the \textit{information subspace}. As a result, SIFt-RLS tracks parameters that are in excited directions while not changing parameter estimation in unexcited directions. It is shown that SIFt-RLS guarantees an upper and lower bound of the covariance matrix, without assumptions of persistent excitation, and explicit bounds are given. Furthermore, sufficient conditions are given for the uniform Lyapunov stability and global uniform exponential stability of parameter estimation error in SIFt-RLS when estimating fixed parameters without noise. SIFt-RLS is compared to other RLS algorithms from the literature in a numerical example without persistently exciting data. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Stability Analysis of Adaptive Model Predictive Control Using the Circle and Tsypkin Criteria

Authors: Juan A. Paredes, Dennis S. Bernstein

Abstract: Absolute stability is a technique for analyzing the stability of Lur'e systems, which arise in diverse applications, such as oscillators with nonlinear damping or nonlinear stiffness. A special class of Lur'e systems consists of self-excited systems (SES), in which bounded oscillations arise from constant inputs. In many cases, SES can be stabilized by linear controllers, which motivates the prese… ▽ More Absolute stability is a technique for analyzing the stability of Lur'e systems, which arise in diverse applications, such as oscillators with nonlinear damping or nonlinear stiffness. A special class of Lur'e systems consists of self-excited systems (SES), in which bounded oscillations arise from constant inputs. In many cases, SES can be stabilized by linear controllers, which motivates the present work, where the goal is to evaluate the effectiveness of adaptive model predictive control for Lur'e systems. In particular, the present paper considers predictive cost adaptive control (PCAC), which is equivalent to a linear, time-variant (LTV) controller. A closed-loop Lur'e system comprised of the positive feedback interconnection of the Lur'e system and the PCAC-based controller can thus be derived at each step. In this work, the circle and Tsypkin criteria are used to evaluate the absolute stability of the closed-loop Lur'e system, where the adaptive controller is viewed as instantaneously linear time-invariant. When the controller converges, the absolute stability criteria guarantee global asymptotic stability of the asymptotic closed-loop dynamics. △ Less

Submitted 5 April, 2024; originally announced April 2024.

Comments: 8 pages. arXiv admin note: text overlap with arXiv:2402.00346

Retrospective Cost-based Extremum Seeking Control with Vanishing Perturbation for Online Output Minimization

Authors: Juan A. Paredes, Jhon Manuel Portella, Dennis S. Bernstein, Ankit Goel

Abstract: Extremum seeking control (ESC) constitutes a powerful technique for online optimization with theoretical guarantees for convergence to the neighborhood of the optimizer under well-understood conditions. However, ESC requires a nonconstant perturbation signal to provide persistent excitation to the target system to yield convergent results, which usually results in steady state oscillations. While… ▽ More Extremum seeking control (ESC) constitutes a powerful technique for online optimization with theoretical guarantees for convergence to the neighborhood of the optimizer under well-understood conditions. However, ESC requires a nonconstant perturbation signal to provide persistent excitation to the target system to yield convergent results, which usually results in steady state oscillations. While certain techniques have been proposed to eliminate perturbations once the neighborhood of the minimizer is reached, system modifications and environmental perturbations can suddenly change the minimizer and nonconstant perturbations would once more be required to convergence to the new minimizer. Hence, this paper develops a retrospective cost-based ESC(RC/ESC) technique for online output minimization with a vanishing perturbation, that is, a perturbation that becomes zero as time increases independently from the state of the controller or the controlled system. The performance of the proposed algorithm is illustrated via numerical examples. △ Less

Submitted 6 February, 2024; originally announced February 2024.

A Data-Driven Autopilot for Fixed-Wing Aircraft Based on Model Predictive Control

Authors: Riley J. Richards, Juan A. Paredes, Dennis S. Bernstein

Abstract: Autopilots for fixed-wing aircraft are typically designed based on linearized aerodynamic models consisting of stability and control derivatives obtained from wind-tunnel testing. The resulting local controllers are then pieced together using gain scheduling. For applications in which the aerodynamics are unmodeled, the present paper proposes an autopilot based on predictive cost adaptive control… ▽ More Autopilots for fixed-wing aircraft are typically designed based on linearized aerodynamic models consisting of stability and control derivatives obtained from wind-tunnel testing. The resulting local controllers are then pieced together using gain scheduling. For applications in which the aerodynamics are unmodeled, the present paper proposes an autopilot based on predictive cost adaptive control (PCAC). As an indirect adaptive control extension of model predictive control, PCAC uses recursive least squares (RLS) with variable-rate forgetting for online, closed-loop system identification. At each time step, RLS-based system identification updates the coefficients of an input-output model whose order is a hyperparameter specified by the user. For MPC, the receding-horizon optimization can be performed by either the backward-propagating Riccati equation or quadratic programming. The present paper investigates the performance of PCAC for fixed-wing aircraft without the use of any aerodynamic modeling or offline/prior data collection. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Experimental Application of Predictive Cost Adaptive Control to Thermoacoustic Oscillations in a Rijke Tube

Authors: Juan A. Paredes, Dennis S. Bernstein

Abstract: Model predictive control (MPC) has been used successfully in diverse applications. As its name suggests, MPC requires a model for predictive optimization. The present paper focuses on the application of MPC to a Rijke tube, in which a heating source and acoustic dynamics interact to produce self-excited oscillations. Since the dynamics of a Rijke tube are difficult to model to a high level of accu… ▽ More Model predictive control (MPC) has been used successfully in diverse applications. As its name suggests, MPC requires a model for predictive optimization. The present paper focuses on the application of MPC to a Rijke tube, in which a heating source and acoustic dynamics interact to produce self-excited oscillations. Since the dynamics of a Rijke tube are difficult to model to a high level of accuracy, the implementation of MPC requires leveraging data from the physical setup as well as knowledge about thermoacoustics, which is labor intensive and requires domain expertise. With this motivation, the present paper uses predictive cost adaptive control (PCAC) for sampled-data control of an experimental Rijke-tube setup. PCAC performs online closed-loop linear model identification for receding-horizon optimization based on the backward propagating Riccati equation. In place of analytical modeling, open-loop experiments are used to create a simple emulation model, which is used for choosing PCAC hyperparameters. PCAC is applied to the Rijke-tube setup under various experimental scenarios. △ Less

Submitted 1 February, 2024; originally announced February 2024.

Maximum likelihood thresholds of Gaussian graphical models and graphical lasso

Authors: Daniel Irving Bernstein, Hayden Outlaw

Abstract: Associated to each graph G is a Gaussian graphical model. Such models are often used in high-dimensional settings, i.e. where there are relatively few data points compared to the number of variables. The maximum likelihood threshold of a graph is the minimum number of data points required to fit the corresponding graphical model using maximum likelihood estimation. Graphical lasso is a method for… ▽ More Associated to each graph G is a Gaussian graphical model. Such models are often used in high-dimensional settings, i.e. where there are relatively few data points compared to the number of variables. The maximum likelihood threshold of a graph is the minimum number of data points required to fit the corresponding graphical model using maximum likelihood estimation. Graphical lasso is a method for selecting and fitting a graphical model. In this project, we ask: when graphical lasso is used to select and fit a graphical model on n data points, how likely is it that n is greater than or equal to the maximum likelihood threshold of the corresponding graph? Our results are a series of computational experiments. △ Less

Submitted 5 December, 2023; originally announced December 2023.

Adaptive Output-Feedback Model Predictive Control of Hammerstein Systems with Unknown Linear Dynamics

Authors: Mohammadreza Kamaldar, Dennis S. Bernstein

Abstract: This paper considers model predictive control of Hammerstein systems, where the linear dynamics are a priori unknown and the input nonlinearity is known. Predictive cost adaptive control (PCAC) is applied to this system using recursive least squares for online, closed-loop system identification with optimization over a receding horizon performed by quadratic programming (QP). In order to account f… ▽ More This paper considers model predictive control of Hammerstein systems, where the linear dynamics are a priori unknown and the input nonlinearity is known. Predictive cost adaptive control (PCAC) is applied to this system using recursive least squares for online, closed-loop system identification with optimization over a receding horizon performed by quadratic programming (QP). In order to account for the input nonlinearity, the input matrix is defined to be control dependent, and the optimization is performed iteratively. This technique is applied to output stabilization of a chain of integrators with unknown dynamics under control saturation and deadzone input nonlinearity. △ Less

Submitted 28 September, 2023; originally announced September 2023.

Comments: arXiv admin note: text overlap with arXiv:2309.11589

Adaptive Real-Time Numerical Differentiation with Variable-Rate Forgetting and Exponential Resetting

Authors: Shashank Verma, Brian Lai, Dennis S. Bernstein

Abstract: Digital PID control requires a differencing operation to implement the D gain. In order to suppress the effects of noisy data, the traditional approach is to filter the data, where the frequency response of the filter is adjusted manually based on the characteristics of the sensor noise. The present paper considers the case where the characteristics of the sensor noise change over time in an unkno… ▽ More Digital PID control requires a differencing operation to implement the D gain. In order to suppress the effects of noisy data, the traditional approach is to filter the data, where the frequency response of the filter is adjusted manually based on the characteristics of the sensor noise. The present paper considers the case where the characteristics of the sensor noise change over time in an unknown way. This problem is addressed by applying adaptive real-time numerical differentiation based on adaptive input and state estimation (AISE). The contribution of this paper is to extend AISE to include variable-rate forgetting with exponential resetting, which allows AISE to more rapidly respond to changing noise characteristics while enforcing the boundedness of the covariance matrix used in recursive least squares. △ Less

Submitted 28 September, 2023; originally announced September 2023.

Journal ref: 2024 American Control Conference (ACC), 3103-3108

Output-Feedback Nonlinear Model Predictive Control with Iterative State- and Control-Dependent Coefficients

Authors: Mohammadreza Kamaldar, Dennis S. Bernstein

Abstract: By optimizing the predicted performance over a receding horizon, model predictive control (MPC) provides the ability to enforce state and control constraints. The present paper considers an extension of MPC for nonlinear systems that can be written in pseudo-linear form with state- and control-dependent coefficients. The main innovation is to apply quadratic programming iteratively over the horizo… ▽ More By optimizing the predicted performance over a receding horizon, model predictive control (MPC) provides the ability to enforce state and control constraints. The present paper considers an extension of MPC for nonlinear systems that can be written in pseudo-linear form with state- and control-dependent coefficients. The main innovation is to apply quadratic programming iteratively over the horizon, where the predicted state trajectory is updated based on the updated control sequence. Output-feedback control is facilitated by using the block-observable canonical form for linear, time-varying dynamics. This control technique is illustrated on various numerical examples, including the Kapitza pendulum with slider-crank actuation, the nonholonomic integrator, the electromagnetically controlled oscillator, and the triple integrator with control-magnitude saturation. △ Less

Submitted 28 September, 2023; v1 submitted 20 September, 2023; originally announced September 2023.

Comments: Submitted to 2024 American Control Conference

Real-Time Kinematics-Based Sensor-Fault Detection for Autonomous Vehicles Using Single and Double Transport with Adaptive Numerical Differentiation

Authors: Shashank Verma, Dennis S. Bernstein

Abstract: Sensor-fault detection is crucial for the safe operation of autonomous vehicles. This paper introduces a novel kinematics-based approach for detecting and identifying faulty sensors, which is model-independent, rule-free, and applicable to ground and aerial vehicles. This method, called kinematics-based sensor fault detection (KSFD), relies on kinematic relations, sensor measurements, and real-tim… ▽ More Sensor-fault detection is crucial for the safe operation of autonomous vehicles. This paper introduces a novel kinematics-based approach for detecting and identifying faulty sensors, which is model-independent, rule-free, and applicable to ground and aerial vehicles. This method, called kinematics-based sensor fault detection (KSFD), relies on kinematic relations, sensor measurements, and real-time single and double numerical differentiation. Using onboard data from radar, rate gyros, magnetometers, and accelerometers, KSFD uniquely identifies a single faulty sensor in real time. To achieve this, adaptive input and state estimation (AISE) is used for real-time single and double numerical differentiation of the sensor data, and the single and double transport theorems are used to evaluate the consistency of data. Unlike model-based and knowledge-based methods, KSFD relies solely on sensor signals, kinematic relations, and AISE for real-time numerical differentiation. For ground vehicles, KSFD requires six kinematics-based error metrics, whereas, for aerial vehicles, nine error metrics are used. Simulated and experimental examples are provided to evaluate the effectiveness of KSFD. △ Less

Submitted 1 July, 2024; v1 submitted 10 September, 2023; originally announced September 2023.

Comments: 30 pages, 29 figures, submitted to Control Engineering Practice

Real-Time Numerical Differentiation of Sampled Data Using Adaptive Input and State Estimation

Authors: Shashank Verma, Sneha Sanjeevini, E. Dogan Sumer, Dennis S. Bernstein

Abstract: Real-time numerical differentiation plays a crucial role in many digital control algorithms, such as PID control, which requires numerical differentiation to implement derivative action. This paper addresses the problem of numerical differentiation for real-time implementation with minimal prior information about the signal and noise using adaptive input and state estimation. Adaptive input estima… ▽ More Real-time numerical differentiation plays a crucial role in many digital control algorithms, such as PID control, which requires numerical differentiation to implement derivative action. This paper addresses the problem of numerical differentiation for real-time implementation with minimal prior information about the signal and noise using adaptive input and state estimation. Adaptive input estimation with adaptive state estimation (AIE/ASE) is based on retrospective cost input estimation, while adaptive state estimation is based on an adaptive Kalman filter in which the input-estimation error covariance and the measurement-noise covariance are updated online. The accuracy of AIE/ASE is compared numerically to several conventional numerical differentiation methods. Finally, AIE/ASE is applied to simulated vehicle position data generated from CarSim. △ Less

Submitted 15 August, 2023; originally announced August 2023.

Comments: This paper is under review at the International Journal of Control

Generalized Forgetting Recursive Least Squares: Stability and Robustness Guarantees

Authors: Brian Lai, Dennis S. Bernstein

Abstract: This work presents generalized forgetting recursive least squares (GF-RLS), a generalization of recursive least squares (RLS) that encompasses many extensions of RLS as special cases. First, sufficient conditions are presented for the 1) Lyapunov stability, 2) uniform Lyapunov stability, 3) global asymptotic stability, and 4) global uniform exponential stability of parameter estimation error in GF… ▽ More This work presents generalized forgetting recursive least squares (GF-RLS), a generalization of recursive least squares (RLS) that encompasses many extensions of RLS as special cases. First, sufficient conditions are presented for the 1) Lyapunov stability, 2) uniform Lyapunov stability, 3) global asymptotic stability, and 4) global uniform exponential stability of parameter estimation error in GF-RLS when estimating fixed parameters without noise. Second, robustness guarantees are derived for the estimation of time-varying parameters in the presence of measurement noise and regressor noise. These robustness guarantees are presented in terms of global uniform ultimate boundedness of the parameter estimation error. A specialization of this result gives a bound to the asymptotic bias of least squares estimators in the errors-in-variables problem. Lastly, a survey is presented to show how GF-RLS can be used to analyze various extensions of RLS from the literature. △ Less

Submitted 6 May, 2024; v1 submitted 8 August, 2023; originally announced August 2023.

Comments: Accepted to the IEEE Transactions on Automatic Control. Scheduled to appear in the 2024 November issue

Self-Excited Dynamics of Discrete-Time Lur'e Models with Affinely Constrained, Piecewise-C1 Feedback Nonlinearities

Authors: Juan Paredes, Omran Kouba, Dennis S. Bernstein

Abstract: Self-excited systems (SES) arise in numerous applications, such as fluid-structure interaction, combustion, and biochemical systems. In support of system identification and digital control of SES, this paper analyzes discrete-time Lur'e models with affinely constrained, piecewise-C1 feedback nonlinearities. The main result provides sufficient conditions under which a discrete-time Lur'e model is s… ▽ More Self-excited systems (SES) arise in numerous applications, such as fluid-structure interaction, combustion, and biochemical systems. In support of system identification and digital control of SES, this paper analyzes discrete-time Lur'e models with affinely constrained, piecewise-C1 feedback nonlinearities. The main result provides sufficient conditions under which a discrete-time Lur'e model is self-excited in the sense that its response is 1) bounded for all initial conditions, and 2) nonconvergent for almost all initial conditions. △ Less

Submitted 7 July, 2023; originally announced July 2023.

Comments: 12 pages, 12 figures, under review

Matroid lifts and representability

Authors: Daniel Irving Bernstein, Zach Walsh

Abstract: A 1965 result of Crapo shows that every elementary lift of a matroid $M$ can be constructed from a linear class of circuits of $M$. In a recent paper, Walsh generalized this construction by defining a rank-$k$ lift of a matroid $M$ given a rank-$k$ matroid $N$ on the set of circuits of $M$, and conjectured that all matroid lifts can be obtained in this way. In this sequel paper we simplify Walsh's… ▽ More A 1965 result of Crapo shows that every elementary lift of a matroid $M$ can be constructed from a linear class of circuits of $M$. In a recent paper, Walsh generalized this construction by defining a rank-$k$ lift of a matroid $M$ given a rank-$k$ matroid $N$ on the set of circuits of $M$, and conjectured that all matroid lifts can be obtained in this way. In this sequel paper we simplify Walsh's construction and show that this conjecture is true for representable matroids but is false in general. This gives a new way to certify that a particular matroid is non-representable, which we use to construct new classes of non-representable matroids. Walsh also applied the new matroid lift construction to gain graphs over the additive group of a non-prime finite field, generalizing a construction of Zaslavsky for these special groups. He conjectured that this construction is possible on three or more vertices only for the additive group of a non-prime finite field. We show that this conjecture holds for four or more vertices, but fails for exactly three. △ Less

Submitted 21 June, 2023; originally announced June 2023.

Maximum likelihood thresholds of generic linear concentration models

Authors: Daniel Irving Bernstein, Steven J. Gortler, Louis Theran

Abstract: The maximum likelihood threshold of a statistical model is the minimum number of datapoints required to fit the model via maximum likelihood estimation. In this paper we determine the maximum likelihood thresholds of generic linear concentration models. This turns out to be the number one would expect from a naive dimension count, which is surprising and nontrivial to prove given that the maximum… ▽ More The maximum likelihood threshold of a statistical model is the minimum number of datapoints required to fit the model via maximum likelihood estimation. In this paper we determine the maximum likelihood thresholds of generic linear concentration models. This turns out to be the number one would expect from a naive dimension count, which is surprising and nontrivial to prove given that the maximum likelihood threshold is a semi-algebraic concept. We also describe geometrically how a linear concentration model can fail to exhibit this generic behavior and briefly discuss connections to rigidity theory. △ Less

Submitted 10 May, 2023; originally announced May 2023.

Experimental Flight Testing of an Adaptive Autopilot with Parameter Drift Mitigation

Authors: Yin Yong Chee, Parham Oveissi, Siyuan Shao, Joonghyun Lee, Juan A. Paredes, Dennis S. Bernstein, Ankit Goel

Abstract: This paper modifies an adaptive multicopter autopilot to mitigate instabilities caused by adaptive parameter drift and presents simulation and experimental results to validate the modified autopilot. The modified adaptive controller is obtained by including a static nonlinearity in the adaptive loop, updated by the retrospective cost adaptive control algorithm. It is shown in simulation and physic… ▽ More This paper modifies an adaptive multicopter autopilot to mitigate instabilities caused by adaptive parameter drift and presents simulation and experimental results to validate the modified autopilot. The modified adaptive controller is obtained by including a static nonlinearity in the adaptive loop, updated by the retrospective cost adaptive control algorithm. It is shown in simulation and physical test experiments that the adaptive autopilot with proposed modifications can continually improve the fixed-gain autopilot as well as prevent the drift of the adaptive parameters, thus improving the robustness of the adaptive autopilot. △ Less

Submitted 20 April, 2023; originally announced April 2023.

Comments: 6 pages, 16 figures, submitted to IROS 2023

Euler's Equation via Lagrangian Dynamics with Generalized Coordinates

Authors: Dennis S. Bernstein, Ankit Goel, Omran Kouba

Abstract: Euler's equation relates the change in angular momentum of a rigid body to the applied torque. This paper fills a gap in the literature by using Lagrangian dynamics to derive Euler's equation in terms of generalized coordinates. This is done by parameterizing the angular velocity vector in terms of 3-2-1 and 3-1-3 Euler angles as well as Euler parameters, that is, unit quaternions. Euler's equation relates the change in angular momentum of a rigid body to the applied torque. This paper fills a gap in the literature by using Lagrangian dynamics to derive Euler's equation in terms of generalized coordinates. This is done by parameterizing the angular velocity vector in terms of 3-2-1 and 3-1-3 Euler angles as well as Euler parameters, that is, unit quaternions. △ Less

Submitted 22 December, 2022; originally announced December 2022.

Experimental Flight Testing of a Fault-Tolerant Adaptive Autopilot for Fixed-Wing Aircraft

Authors: Joonghyun Lee, John Spencer, Siyuan Shao, Juan Augusto Paredes, Dennis S. Bernstein, Ankit Goel

Abstract: This paper presents an adaptive autopilot for fixed-wing aircraft and compares its performance with a fixed-gain autopilot. The adaptive autopilot is constructed by augmenting the autopilot architecture with adaptive control laws that are updated using retrospective cost adaptive control. In order to investigate the performance of the adaptive autopilot, the default gains of the fixed-gain autopil… ▽ More This paper presents an adaptive autopilot for fixed-wing aircraft and compares its performance with a fixed-gain autopilot. The adaptive autopilot is constructed by augmenting the autopilot architecture with adaptive control laws that are updated using retrospective cost adaptive control. In order to investigate the performance of the adaptive autopilot, the default gains of the fixed-gain autopilot are scaled to degrade its performance. This scenario provides a venue for determining the ability of the adaptive autopilot to compensate for the degraded fixed-gain autopilot. Next, the performance of the adaptive autopilot is examined under failure conditions by simulating a scenario where one of the control surfaces is assumed to be stuck at an unknown angle. The adaptive autopilot is also tested in physical flight experiments under degraded-nominal conditions, and the resulting performance improvement is examined. △ Less

Submitted 24 October, 2022; originally announced October 2022.

Comments: 8 pages, submitted to 2023 American Control Conference (ACC). arXiv admin note: substantial text overlap with arXiv:2110.11390

Computing maximum likelihood thresholds using graph rigidity

Authors: Daniel Irving Bernstein, Sean Dewar, Steven J. Gortler, Anthony Nixon, Meera Sitharam, Louis Theran

Abstract: The maximum likelihood threshold (MLT) of a graph $G$ is the minimum number of samples to almost surely guarantee existence of the maximum likelihood estimate in the corresponding Gaussian graphical model. Recently a new characterization of the MLT in terms of rigidity-theoretic properties of $G$ was proved \cite{Betal}. This characterization was then used to give new combinatorial lower bounds on… ▽ More The maximum likelihood threshold (MLT) of a graph $G$ is the minimum number of samples to almost surely guarantee existence of the maximum likelihood estimate in the corresponding Gaussian graphical model. Recently a new characterization of the MLT in terms of rigidity-theoretic properties of $G$ was proved \cite{Betal}. This characterization was then used to give new combinatorial lower bounds on the MLT of any graph. We continue this line of research by exploiting combinatorial rigidity results to compute the MLT precisely for several families of graphs. These include graphs with at most $9$ vertices, graphs with at most 24 edges, every graph sufficiently close to a complete graph and graphs with bounded degrees. △ Less

Submitted 20 October, 2022; originally announced October 2022.

Comments: 15 pages, 5 figures. arXiv admin note: substantial text overlap with arXiv:2108.02185

MSC Class: 62H12 (Primary); 52C25 (Secondary)

Journal ref: Alg. Stat. 14 (2023) 287-305

On the Accuracy of the One-step UKF and the Two-step UKF

Authors: Ankit Goel, Dennis S. Bernstein

Abstract: The most accurate version of the unscented Kalman filter (UKF) involves the construction of two ensembles. To reduce computational cost, however, UKF is often implemented without the second ensemble. This simplification comes at a price, however, since, for linear systems, the one-step variation of the two-step UKF does not specialize to the classical Kalman filter, with an associated loss of accu… ▽ More The most accurate version of the unscented Kalman filter (UKF) involves the construction of two ensembles. To reduce computational cost, however, UKF is often implemented without the second ensemble. This simplification comes at a price, however, since, for linear systems, the one-step variation of the two-step UKF does not specialize to the classical Kalman filter, with an associated loss of accuracy. This paper remedies this drawback by developing a modified one-step UKF that recovers the classical Kalman filter for linear systems. Numerical examples show that the modified one-step UKF also recovers the accuracy of the two-step UKF in nonlinear systems with linear outputs. △ Less

Submitted 18 August, 2022; originally announced August 2022.

Comments: To appear in the proceedings of CDC2022. arXiv admin note: text overlap with arXiv:2104.00736

Retrospective Cost Parameter Estimation with Application to Space Weather Modeling

Authors: Ankit Goel, Dennis S. Bernstein

Abstract: This chapter reviews standard parameter-estimation techniques and presents a novel gradient-, ensemble-, adjoint-free data-driven parameter estimation technique in the DDDAS framework. This technique, called retrospective cost parameter estimation (RCPE), is motivated by large-scale complex estimation models characterized by high-dimensional nonlinear dynamics, nonlinear parameterizations, and rep… ▽ More This chapter reviews standard parameter-estimation techniques and presents a novel gradient-, ensemble-, adjoint-free data-driven parameter estimation technique in the DDDAS framework. This technique, called retrospective cost parameter estimation (RCPE), is motivated by large-scale complex estimation models characterized by high-dimensional nonlinear dynamics, nonlinear parameterizations, and representational models. RCPE is illustrated by estimating unknown parameters in three examples. In the first example, salient features of RCPE are investigated by considering parameter estimation problem in a low-order nonlinear system. In the second example, RCPE is used to estimate the convective coefficient and the viscosity in the generalized Burgers equation by using a scalar measurement. In the final example, RCPE is used to estimate thermal conductivity coefficients that relate temporal temperature variation with the vertical gradient of the temperature in the atmosphere. △ Less

Submitted 15 August, 2022; originally announced August 2022.

An Adaptive Digital Autopilot for Fixed-Wing Aircraft with Actuator Faults

Authors: Joonghyun Lee, John Spencer, Juan Augusto Paredes, Sai Ravela, Dennis S. Bernstein, Ankit Goel

Abstract: This paper develops an adaptive digital autopilot for a fixed-wing aircraft and compares its performance with a fixed-gain autopilot. The adaptive digital autopilot is constructed by augmenting the autopilot architecture implemented in PX4 flight stack with adaptive digital control laws that are updated using the retrospective cost adaptive control algorithm. In order to investigate the performanc… ▽ More This paper develops an adaptive digital autopilot for a fixed-wing aircraft and compares its performance with a fixed-gain autopilot. The adaptive digital autopilot is constructed by augmenting the autopilot architecture implemented in PX4 flight stack with adaptive digital control laws that are updated using the retrospective cost adaptive control algorithm. In order to investigate the performance of the adaptive digital autopilot, the default gains of the fixed-gain autopilot are scaled down to degrade its performance. This scenario provides a venue for determining the ability of the adaptive digital autopilot to compensate for the detuned fixed-gain autopilot. Next, the performance of the adaptive autopilot is examined under failure conditions by simulating a scenario where one of the control surfaces is assumed to be stuck at an unknown angular position. The adaptive digital autopilot is tested in simulation, and the resulting performance improvements are examined. △ Less

Submitted 21 October, 2021; originally announced October 2021.

Comments: 6 pages, 11 figures, submitted to ACC 2022

$K_{5,5}$ is fully reconstructible in $\mathbb{C}^3$

Authors: Daniel Irving Bernstein, Steven J. Gortler

Abstract: A graph $G$ is fully reconstructible in $\mathbb{C}^d$ if the graph is determined from its $d$-dimensional measurement variety. The full reconstructibility problem has been solved for $d=1$ and $d=2$. For $d=3$, some necessary and some sufficient conditions are known and $K_{5,5}$ falls squarely within the gap in the theory. In this paper, we show that $K_{5,5}$ is fully reconstructible in… ▽ More A graph $G$ is fully reconstructible in $\mathbb{C}^d$ if the graph is determined from its $d$-dimensional measurement variety. The full reconstructibility problem has been solved for $d=1$ and $d=2$. For $d=3$, some necessary and some sufficient conditions are known and $K_{5,5}$ falls squarely within the gap in the theory. In this paper, we show that $K_{5,5}$ is fully reconstructible in $\mathbb{C}^3$. △ Less

Submitted 17 November, 2022; v1 submitted 19 October, 2021; originally announced October 2021.

Journal ref: Discrete Applied Mathematics 322 (2022): 442-447

An Adaptive PID Autotuner for Multicopters with Experimental Results

Authors: John Spencer, Joonghyun Lee, Juan Augusto Paredes, Ankit Goel, Dennis Bernstein

Abstract: This paper develops an adaptive PID autotuner for multicopters, and presents simulation and experimental results. The autotuner consists of adaptive digital control laws based on retrospective cost adaptive control implemented in the PX4 flight stack. A learning trajectory is used to optimize the autopilot during a single flight. The autotuned autopilot is then compared with the default PX4 autopi… ▽ More This paper develops an adaptive PID autotuner for multicopters, and presents simulation and experimental results. The autotuner consists of adaptive digital control laws based on retrospective cost adaptive control implemented in the PX4 flight stack. A learning trajectory is used to optimize the autopilot during a single flight. The autotuned autopilot is then compared with the default PX4 autopilot by flying a test trajectory constructed using the second-order Hilbert curve. In order to investigate the sensitivity of the autotuner to the quadcopter dynamics, the mass of the quadcopter is varied, and the performance of the autotuned and default autopilot is compared. It is observed that the autotuned autopilot outperforms the default autopilot. △ Less

Submitted 27 September, 2021; originally announced September 2021.

Comments: Submitted to ICRA 2022, 7 pages, 10 figures

Self-Excited Dynamics of Discrete-Time Lur'e Systems

Authors: Juan A. Paredes, Syed Aseem Ul Islam, Omran Kouba, Dennis S. Bernstein

Abstract: Self-excited systems arise in numerous applications, such as biochemical systems, fluid-structure interaction, and combustion. This paper analyzes a discrete-time Lur'e system with a piecewise-linear saturation feedback nonlinearity. The main result provides sufficient conditions under which the Lur'e system is self-excited in the sense that its response is bounded and nonconvergent. Self-excited systems arise in numerous applications, such as biochemical systems, fluid-structure interaction, and combustion. This paper analyzes a discrete-time Lur'e system with a piecewise-linear saturation feedback nonlinearity. The main result provides sufficient conditions under which the Lur'e system is self-excited in the sense that its response is bounded and nonconvergent. △ Less

Submitted 26 September, 2021; originally announced September 2021.

Comments: Submitted to Systems & Control Letters, 12 pages, 7 figures

Maximum likelihood thresholds via graph rigidity

Authors: Daniel Irving Bernstein, Sean Dewar, Steven J. Gortler, Anthony Nixon, Meera Sitharam, Louis Theran

Abstract: The maximum likelihood threshold (MLT) of a graph $G$ is the minimum number of samples to almost surely guarantee existence of the maximum likelihood estimate in the corresponding Gaussian graphical model. We give a new characterization of the MLT in terms of rigidity-theoretic properties of $G$ and use this characterization to give new combinatorial lower bounds on the MLT of any graph. We use… ▽ More The maximum likelihood threshold (MLT) of a graph $G$ is the minimum number of samples to almost surely guarantee existence of the maximum likelihood estimate in the corresponding Gaussian graphical model. We give a new characterization of the MLT in terms of rigidity-theoretic properties of $G$ and use this characterization to give new combinatorial lower bounds on the MLT of any graph. We use the new lower bounds to give high-probability guarantees on the maximum likelihood thresholds of sparse Erd{ö}s-Rényi random graphs in terms of their average density. These examples show that the new lower bounds are within a polylog factor of tight, where, on the same graph families, all known lower bounds are trivial. Based on computational experiments made possible by our methods, we conjecture that the MLT of an Erd{ö}s-Rényi random graph is equal to its generic completion rank with high probability. Using structural results on rigid graphs in low dimension, we can prove the conjecture for graphs with MLT at most $4$ and describe the threshold probability for the MLT to switch from $3$ to $4$. We also give a geometric characterization of the MLT of a graph in terms of a new "lifting" problem for frameworks that is interesting in its own right. The lifting perspective yields a new connection between the weak MLT (where the maximum likelihood estimate exists only with positive probability) and the classical Hadwiger-Nelson problem. △ Less

Submitted 5 December, 2023; v1 submitted 4 August, 2021; originally announced August 2021.

Comments: In press at Annals of Applied Probability

MSC Class: 05C75; 62A99; 52C25; 62R01; 90C25

Learning from Partially Overlapping Labels: Image Segmentation under Annotation Shift

Authors: Gregory Filbrandt, Konstantinos Kamnitsas, David Bernstein, Alexandra Taylor, Ben Glocker

Abstract: Scarcity of high quality annotated images remains a limiting factor for training accurate image segmentation models. While more and more annotated datasets become publicly available, the number of samples in each individual database is often small. Combining different databases to create larger amounts of training data is appealing yet challenging due to the heterogeneity as a result of difference… ▽ More Scarcity of high quality annotated images remains a limiting factor for training accurate image segmentation models. While more and more annotated datasets become publicly available, the number of samples in each individual database is often small. Combining different databases to create larger amounts of training data is appealing yet challenging due to the heterogeneity as a result of differences in data acquisition and annotation processes, often yielding incompatible or even conflicting information. In this paper, we investigate and propose several strategies for learning from partially overlapping labels in the context of abdominal organ segmentation. We find that combining a semi-supervised approach with an adaptive cross entropy loss can successfully exploit heterogeneously annotated data and substantially improve segmentation accuracy compared to baseline and alternative approaches. △ Less

Submitted 13 July, 2021; originally announced July 2021.

Regularization-Induced Bias and Consistency in Recursive Least Squares

Authors: Brian Lai, Syed Aseem Ul Islam, Dennis S. Bernstein

Abstract: Within the context of recursive least squares (RLS) parameter estimation, the goal of the present paper is to study the effect of regularization-induced bias on the transient and asymptotic accuracy of the parameter estimates. We consider this question in three stages. First, we consider regression with random data, in which case persistency is guaranteed. Next, we apply RLS to finite-impulse-resp… ▽ More Within the context of recursive least squares (RLS) parameter estimation, the goal of the present paper is to study the effect of regularization-induced bias on the transient and asymptotic accuracy of the parameter estimates. We consider this question in three stages. First, we consider regression with random data, in which case persistency is guaranteed. Next, we apply RLS to finite-impulse-response (FIR) system identification and, finally, to infinite-impulse-response (IIR) system identification. For each case, we relate the condition number of the regressor matrix to the transient response and rate of convergence of the parameter estimates. △ Less

Submitted 10 August, 2021; v1 submitted 16 June, 2021; originally announced June 2021.

Journal ref: 2021 American Control Conference (ACC), 2021, pp. 3987-3992

OpenSSLNTRU: Faster post-quantum TLS key exchange

Authors: Daniel J. Bernstein, Billy Bob Brumley, Ming-Shing Chen, Nicola Tuveri

Abstract: Google's CECPQ1 experiment in 2016 integrated a post-quantum key-exchange algorithm, newhope1024, into TLS 1.2. The Google-Cloudflare CECPQ2 experiment in 2019 integrated a more efficient key-exchange algorithm, ntruhrss701, into TLS 1.3. This paper revisits the choices made in CECPQ2, and shows how to achieve higher performance for post-quantum key exchange in TLS 1.3 using a higher-security al… ▽ More Google's CECPQ1 experiment in 2016 integrated a post-quantum key-exchange algorithm, newhope1024, into TLS 1.2. The Google-Cloudflare CECPQ2 experiment in 2019 integrated a more efficient key-exchange algorithm, ntruhrss701, into TLS 1.3. This paper revisits the choices made in CECPQ2, and shows how to achieve higher performance for post-quantum key exchange in TLS 1.3 using a higher-security algorithm, sntrup761. Previous work had indicated that ntruhrss701 key generation was much faster than sntrup761 key generation, but this paper makes sntrup761 key generation much faster by generating a batch of keys at once. Batch key generation is invisible at the TLS protocol layer, but raises software-engineering questions regarding the difficulty of integrating batch key exchange into existing TLS libraries and applications. This paper shows that careful choices of software layers make it easy to integrate fast post-quantum software, including batch key exchange, into TLS with minor changes to TLS libraries and no changes to applications. As a demonstration of feasibility, this paper reports successful integration of its fast sntrup761 library, via a lightly patched OpenSSL, into an unmodified web browser and an unmodified TLS terminator. This paper also reports TLS 1.3 handshake benchmarks, achieving more TLS 1.3 handshakes per second than any software included in OpenSSL. △ Less

Submitted 14 December, 2021; v1 submitted 16 June, 2021; originally announced June 2021.

Comments: 20 pages, 5 figures; accepted at USENIX Security 2022; added Artifact Evaluation badges and final Artifact Appendix

Two Modifications of the Unscented Kalman Filter that Specialize to the Kalman Filter for Linear Systems

Authors: Ankit Goel, Dennis S. Bernstein

Abstract: Although the unscented Kalman filter (UKF) is applicable to nonlinear systems, it turns out that, for linear systems, UKF does not specialize to the classical Kalman filter. This situation suggests that it may be advantageous to modify UKF in such a way that, for linear systems, the Kalman filter is recovered. The ultimate goal is thus to develop modifications of UKF that specialize to the Kalman… ▽ More Although the unscented Kalman filter (UKF) is applicable to nonlinear systems, it turns out that, for linear systems, UKF does not specialize to the classical Kalman filter. This situation suggests that it may be advantageous to modify UKF in such a way that, for linear systems, the Kalman filter is recovered. The ultimate goal is thus to develop modifications of UKF that specialize to the Kalman filter for linear systems and have improved accuracy for nonlinear systems. With this motivation, this paper presents two modifications of UKF that specialize to the Kalman filter for linear systems. The first modification (EUKF-A) requires the Jacobian of the dynamics map, whereas the second modification (EUKF-C) requires the Jacobian of the measurement map. For various nonlinear examples, the accuracy of EUKF-A and EUKF-C is compared to the accuracy of UKF. △ Less

Submitted 1 April, 2021; originally announced April 2021.

Comments: Submitted to CDC 2021

Effects of Coulomb Coupling On Friction In Strongly Magnetized Plasmas

Authors: David J. Bernstein, Scott D. Baalrud

Abstract: The friction force on a test particle traveling through a plasma that is both strongly coupled and strongly magnetized is studied using molecular dynamics simulations. In addition to the usual stopping power component aligned antiparallel to the velocity, a transverse component that is perpendicular to both the velocity and Lorentz force is observed. This component, which was recently discovered i… ▽ More The friction force on a test particle traveling through a plasma that is both strongly coupled and strongly magnetized is studied using molecular dynamics simulations. In addition to the usual stopping power component aligned antiparallel to the velocity, a transverse component that is perpendicular to both the velocity and Lorentz force is observed. This component, which was recently discovered in weakly coupled plasmas, is found to increase in both absolute and relative magnitude in the strongly coupled regime. Strong coupling is also observed to induce a third component of the friction force in the direction of the Lorentz force. These first-principles simulations reveal novel physics associated with collisions in strongly coupled, strongly magnetized, plasmas that are not predicted by existing kinetic theories. The effect is expected to influence macroscopic transport in a number of laboratory experiments and astrophysical plasmas. △ Less

Submitted 19 February, 2021; originally announced February 2021.

Data-Driven Retrospective Cost Adaptive Control for Flight Control Application

Authors: Syed Aseem Ul Islam, Tam W. Nguyen, Ilya V. Kolmanovsky, Dennis S. Bernstein

Abstract: Unlike fixed-gain robust control, which trades off performance with modeling uncertainty, direct adaptive control uses partial modeling information for online tuning. The present paper combines retrospective cost adaptive control (RCAC), a direct adaptive control technique for sampled-data systems, with online system identification based on recursive least squares (RLS) with variable-rate forgetti… ▽ More Unlike fixed-gain robust control, which trades off performance with modeling uncertainty, direct adaptive control uses partial modeling information for online tuning. The present paper combines retrospective cost adaptive control (RCAC), a direct adaptive control technique for sampled-data systems, with online system identification based on recursive least squares (RLS) with variable-rate forgetting (VRF). The combination of RCAC and RLS-VRF constitutes data-driven RCAC (DDRCAC), where the online system identification is used to construct the target model, which defines the retrospective performance variable. This paper investigates the ability of RLS-VRF to provide the modeling information needed for the target model, especially nonminimum-phase (NMP) zeros. DDRCAC is applied to single-input, single-output (SISO) and multiple-input, multiple-output (MIMO) numerical examples with unknown NMP zeros, as well as several flight control problems, namely, unknown transition from minimum-phase to NMP lateral dynamics, flexible modes, flutter, and nonlinear planar missile dynamics. △ Less

Submitted 8 April, 2021; v1 submitted 14 February, 2021; originally announced February 2021.

Comments: 60 pages, 28 figures, accepted by AIAA Journal of Guidance, Control, and Dynamics

Experimental Implementation of an Adaptive Digital Autopilot

Authors: Ankit Goel, Juan Augusto Paredes, Harshil Dadhaniya, Syed Aseem Ul Islam, Abdulazeez Mohammed Salim, Sai Ravela, Dennis Bernstein

Abstract: This paper develops an adaptive digital autopilot for quadcopters and presents experimental results. The adaptive digital autopilot is constructed by augmenting the PX4 autopilot control system architecture with adaptive digital control laws based on retrospective cost adaptive control (RCAC). In order to investigate the performance of the adaptive digital autopilot, the default gains of the fixed… ▽ More This paper develops an adaptive digital autopilot for quadcopters and presents experimental results. The adaptive digital autopilot is constructed by augmenting the PX4 autopilot control system architecture with adaptive digital control laws based on retrospective cost adaptive control (RCAC). In order to investigate the performance of the adaptive digital autopilot, the default gains of the fixed-gain autopilot are scaled by a small factor, which severely degrades its performance. This scenario thus provides a venue for determining the ability of the adaptive digital autopilot to compensate for the detuned fixed-gain autopilot. The adaptive digital autopilot is tested in simulation and physical flight tests, and the resulting performance improvements are examined. △ Less

Submitted 4 December, 2020; originally announced December 2020.

Comments: Submitted to ACC 2021

Output-Feedback Model Predictive Control with Online Identification

Authors: Tam W. Nguyen, Syed Aseem Ul Islam, Dennis S. Bernstein, Ilya V. Kolmanovsky

Abstract: Model predictive control (MPC) is a widely used modern control technique with numerous successful application in diverse areas. Much of this success is due to the ability of MPC to enforce state and control constraints, which are crucial in many applications of control. In order to avoid the need for an observer, output-feedback model predictive control with online identification (OFMPCOI) uses th… ▽ More Model predictive control (MPC) is a widely used modern control technique with numerous successful application in diverse areas. Much of this success is due to the ability of MPC to enforce state and control constraints, which are crucial in many applications of control. In order to avoid the need for an observer, output-feedback model predictive control with online identification (OFMPCOI) uses the block observable canonical form whose state consists of past values of the control inputs and measured outputs. Online identification is performed using recursive least squares (RLS) with variable-rate forgetting. The article describes the algorithmic details of OFMPCOI and numerically investigates its performance through a collection of numerical examples that highlight various control challenges, such as model order uncertainty, sensor noise, prediction horizon, stabilization, magnitude and move-size saturation, and stabilization. The numerical examples are used to probe the performance of OFMPCOI in terms of persistency, consistency, and exigency. Since OFMPCOI does not employ a separate control perturbation to enhance persistency, the focus is on self-generated persistency during transient operation. For closed-loop identification using RLS, sensor noise gives rise to bias in the identified model, and the goal is to determine the effect of the lack of consistency. Finally, the numerical examples reveal exigency, which is the extent to which the online identification emphasizes model characteristics that are most relevant to meeting performance objectives. △ Less

Submitted 22 September, 2020; originally announced September 2020.

Comments: 61 pages, 27 figures, "T. W. Nguyen, S. A. Ul Islam, A. L. Bruce, A. Goel, D. S. Bernstein and I. V. Kolmanovsky, "Output-Feedback RLS-Based Model Predictive Control*," 2020 American Control Conference (ACC), Denver, CO, USA, 2020, pp. 2395-2400, doi: 10.23919/ACC45564.2020.9148011."

Computation Of Microbial Ecosystems in Time and Space (COMETS): An open source collaborative platform for modeling ecosystems metabolism

Authors: Ilija Dukovski, Djordje Bajić, Jeremy M Chacón, Michael Quintin, Jean CC Vila, Snorre Sulheim, Alan R Pacheco, David B Bernstein, William J Rieh, Kirill S Korolev, Alvaro Sanchez, William R Harcombe, Daniel Segrè

Abstract: Genome-scale stoichiometric modeling of metabolism has become a standard systems biology tool for modeling cellular physiology and growth. Extensions of this approach are also emerging as a valuable avenue for predicting, understanding and designing microbial communities. COMETS (Computation Of Microbial Ecosystems in Time and Space) was initially developed as an extension of dynamic flux balance… ▽ More Genome-scale stoichiometric modeling of metabolism has become a standard systems biology tool for modeling cellular physiology and growth. Extensions of this approach are also emerging as a valuable avenue for predicting, understanding and designing microbial communities. COMETS (Computation Of Microbial Ecosystems in Time and Space) was initially developed as an extension of dynamic flux balance analysis, which incorporates cellular and molecular diffusion, enabling simulations of multiple microbial species in spatially structured environments. Here we describe how to best use and apply the most recent version of this platform, COMETS 2, which incorporates a more accurate biophysical model of microbial biomass expansion upon growth, as well as several new biological simulation modules, including evolutionary dynamics and extracellular enzyme activity. COMETS 2 provides user-friendly Python and MATLAB interfaces compatible with the well-established COBRA models and methods, and comprehensive documentation and tutorials, facilitating the use of COMETS for researchers at all levels of expertise with metabolic simulations. This protocol provides a detailed guideline for installing, testing and applying COMETS 2 to different scenarios, with broad applicability to microbial communities across biomes and scales. △ Less

Submitted 3 September, 2020; originally announced September 2020.

Comments: 146 pages, 12 figures, 2 supplementary figures, 3 supplementary videos. Nat Protoc (2021)

Journal ref: Dukovski, I., Bajić, D., Chacón, J.M. et al. A metabolic modeling platform for the computation of microbial ecosystems in time and space (COMETS). Nat Protoc (2021)

Friction Force in Strongly Magnetized Plasmas

Authors: David J. Bernstein, Trevor Lafleur, Jerome Daligault, Scott D. Baalrud

Abstract: A charged particle moving through a plasma experiences a friction force that commonly acts antiparallel to its velocity. It was recently predicted that in strongly magnetized plasmas, in which the plasma particle gyro-frequency exceeds the plasma frequency, the friction also includes a transverse component that is perpendicular to both the velocity and Lorentz force. Here, this prediction is confi… ▽ More A charged particle moving through a plasma experiences a friction force that commonly acts antiparallel to its velocity. It was recently predicted that in strongly magnetized plasmas, in which the plasma particle gyro-frequency exceeds the plasma frequency, the friction also includes a transverse component that is perpendicular to both the velocity and Lorentz force. Here, this prediction is confirmed using molecular dynamics simulations, and it is shown that the relative magnitude of the transverse component increases with plasma coupling strength. This result influences single particle motion and macroscopic transport in strongly magnetized plasmas found in a broad range of applications. △ Less

Submitted 25 August, 2020; originally announced August 2020.

Journal ref: Phys. Rev. E 102, 041201 (2020)

BasicBlocker: ISA Redesign to Make Spectre-Immune CPUs Faster

Authors: Jan Philipp Thoma, Jakob Feldtkeller, Markus Krausz, Tim Güneysu, Daniel J. Bernstein

Abstract: Recent research has revealed an ever-growing class of microarchitectural attacks that exploit speculative execution, a standard feature in modern processors. Proposed and deployed countermeasures involve a variety of compiler updates, firmware updates, and hardware updates. None of the deployed countermeasures have convincing security arguments, and many of them have already been broken. The obv… ▽ More Recent research has revealed an ever-growing class of microarchitectural attacks that exploit speculative execution, a standard feature in modern processors. Proposed and deployed countermeasures involve a variety of compiler updates, firmware updates, and hardware updates. None of the deployed countermeasures have convincing security arguments, and many of them have already been broken. The obvious way to simplify the analysis of speculative-execution attacks is to eliminate speculative execution. This is normally dismissed as being unacceptably expensive, but the underlying cost analyses consider only software written for current instruction-set architectures, so they do not rule out the possibility of a new instruction-set architecture providing acceptable performance without speculative execution. A new ISA requires compiler and hardware updates, but these are happening in any case. This paper introduces BasicBlocker, a generic ISA modification that works for all common ISAs and that allows non-speculative CPUs to obtain most of the performance benefit that would have been provided by speculative execution. To demonstrate the feasibility of BasicBlocker, this paper defines a variant of the RISC-V ISA called BBRISC-V and provides a thorough evaluation on both a 5-stage in-order soft core and a superscalar out-of-order processor using an associated compiler and a variety of benchmark programs. △ Less

Submitted 5 May, 2021; v1 submitted 31 July, 2020; originally announced July 2020.

Comments: Preprint

Adaptive Digital PID Control of a Quadcopter with Unknown Dynamics

Authors: Ankit Goel, Abdulazeez Mohammed Salim, Ahmad Ansari, Sai Ravela, Dennis Bernstein

Abstract: This paper develops an adaptive autopilot for quadcopters with unknown dynamics. To do this, the PX4 autopilot architecture is modified so that the feedback and feedforward controllers are replaced by adaptive control laws based on retrospective cost adaptive control (RCAC). The present paper provides a numerical investigation of the performance of the adaptive autopilot on a quadcopter with unkno… ▽ More This paper develops an adaptive autopilot for quadcopters with unknown dynamics. To do this, the PX4 autopilot architecture is modified so that the feedback and feedforward controllers are replaced by adaptive control laws based on retrospective cost adaptive control (RCAC). The present paper provides a numerical investigation of the performance of the adaptive autopilot on a quadcopter with unknown dynamics. In order to reflect the absence of prior modeling information, all of the adaptive digital controllers are initialized at zero gains. In addition, moment-of-inertia of the quadcopter is varied to test the robustness of the adaptive autopilot. In all test cases, the vehicle is commanded to follow a given trajectory, and the resulting performance is examined. △ Less

Submitted 30 May, 2020; originally announced June 2020.

Comments: Submitted to ACC2020

Identification of Self-Excited Systems Using Discrete-Time, Time-Delayed Lur'e Models

Authors: Juan A. Paredes, Dennis S. Bernstein

Abstract: This paper presents a system identification technique for systems whose output is asymptotically periodic under constant inputs. The model used for system identification is a discrete-time Lur'e model consisting of asymptotically stable linear dynamics, a time delay, a washout filter, and a static nonlinear feedback mapping. For all sufficiently large scalings of the loop transfer function, these… ▽ More This paper presents a system identification technique for systems whose output is asymptotically periodic under constant inputs. The model used for system identification is a discrete-time Lur'e model consisting of asymptotically stable linear dynamics, a time delay, a washout filter, and a static nonlinear feedback mapping. For all sufficiently large scalings of the loop transfer function, these components cause divergence under small signal levels and decay under large signal amplitudes, thus producing an asymptotically oscillatory output. A bias-generation mechanism is used to provide a bias in the oscillation. The contribution of the paper is a least-squares technique that estimates the coefficients of the linear model as well as the parameterization of the continuous, piecewise-linear feedback mapping. △ Less

Submitted 29 April, 2020; originally announced April 2020.

Generic symmetry-forced infinitesimal rigidity: translations and rotations

Authors: Daniel Irving Bernstein

Abstract: We characterize the combinatorial types of symmetric frameworks in the plane that are minimally generically symmetry-forced infinitesimally rigid when the symmetry group consists of rotations and translations. Along the way, we use tropical geometry to show how a construction of Edmonds that associates a matroid to a submodular function can be used to give a description of the algebraic matroid of… ▽ More We characterize the combinatorial types of symmetric frameworks in the plane that are minimally generically symmetry-forced infinitesimally rigid when the symmetry group consists of rotations and translations. Along the way, we use tropical geometry to show how a construction of Edmonds that associates a matroid to a submodular function can be used to give a description of the algebraic matroid of a Hadamard product of two linear spaces in terms of the matroids of each linear space. This leads to new, short, proofs of Laman's theorem, and a theorem of Jord{á}n, Kaszanitzky, and Tanigawa, and Malestein and Theran characterizing the minimally generically symmetry-forced rigid graphs in the plane when the symmetry group contains only rotations. △ Less

Submitted 8 December, 2021; v1 submitted 23 March, 2020; originally announced March 2020.

Comments: Introduction is now more accessible to a wider audience. This version will appear in SIAGA

MSC Class: 52C25; 05B35; 14T05

Faster computation of isogenies of large prime degree

Authors: Daniel Bernstein, Luca de Feo, Antonin Leroux, Benjamin Smith

Abstract: Let $\mathcal{E}/\mathbb{F}_q$ be an elliptic curve, and $P$ a point in $\mathcal{E}(\mathbb{F}_q)$ of prime order $\ell$. Vélu's formulae let us compute a quotient curve $\mathcal{E}' = \mathcal{E}/\langle{P}\rangle$ and rational maps defining a quotient isogeny $φふぁい: \mathcal{E} \to \mathcal{E}'$ in $\tilde{O}(\ell)$ $\mathbb{F}_q$-operations, where the $\tilde{O}$ is uniform in $q$.This article s… ▽ More Let $\mathcal{E}/\mathbb{F}_q$ be an elliptic curve, and $P$ a point in $\mathcal{E}(\mathbb{F}_q)$ of prime order $\ell$. Vélu's formulae let us compute a quotient curve $\mathcal{E}' = \mathcal{E}/\langle{P}\rangle$ and rational maps defining a quotient isogeny $φふぁい: \mathcal{E} \to \mathcal{E}'$ in $\tilde{O}(\ell)$ $\mathbb{F}_q$-operations, where the $\tilde{O}$ is uniform in $q$.This article shows how to compute $\mathcal{E}'$, and $φふぁい(Q)$ for $Q$ in $\mathcal{E}(\mathbb{F}_q)$, using only $\tilde{O}(\sqrt{\ell})$ $\mathbb{F}_q$-operations, where the $\tilde{O}$ is again uniform in $q$.As an application, this article speeds up some computations used in the isogeny-based cryptosystems CSIDH and CSURF. △ Less

Submitted 23 March, 2020; originally announced March 2020.

On Alignment in Deep Linear Neural Networks

Authors: Adityanarayanan Radhakrishnan, Eshaan Nichani, Daniel Bernstein, Caroline Uhler

Abstract: We study the properties of alignment, a form of implicit regularization, in linear neural networks under gradient descent. We define alignment for fully connected networks with multidimensional outputs and show that it is a natural extension of alignment in networks with 1-dimensional outputs as defined by Ji and Telgarsky, 2018. While in fully connected networks, there always exists a global mini… ▽ More We study the properties of alignment, a form of implicit regularization, in linear neural networks under gradient descent. We define alignment for fully connected networks with multidimensional outputs and show that it is a natural extension of alignment in networks with 1-dimensional outputs as defined by Ji and Telgarsky, 2018. While in fully connected networks, there always exists a global minimum corresponding to an aligned solution, we analyze alignment as it relates to the training process. Namely, we characterize when alignment is an invariant of training under gradient descent by providing necessary and sufficient conditions for this invariant to hold. In such settings, the dynamics of gradient descent simplify, thereby allowing us to provide an explicit learning rate under which the network converges linearly to a global minimum. We then analyze networks with layer constraints such as convolutional networks. In this setting, we prove that gradient descent is equivalent to projected gradient descent, and that alignment is impossible with sufficiently large datasets. △ Less

Submitted 16 June, 2020; v1 submitted 13 March, 2020; originally announced March 2020.

Recursive Least Squares with Variable-Direction Forgetting -- Compensating for the loss of persistency

Authors: Ankit Goel, Adam L. Bruce, Dennis S. Bernstein

Abstract: Learning depends on the ability to acquire and assimilate new information. This ability depends---somewhat counterintuitively---on the ability to forget. In particular, effective forgetting requires the ability to recognize and utilize new information to order to update a system model. This article is a tutorial on forgetting within the context of recursive least squares (RLS). To do this, RLS is… ▽ More Learning depends on the ability to acquire and assimilate new information. This ability depends---somewhat counterintuitively---on the ability to forget. In particular, effective forgetting requires the ability to recognize and utilize new information to order to update a system model. This article is a tutorial on forgetting within the context of recursive least squares (RLS). To do this, RLS is first presented in its classical form, which employs uniform-direction forgetting. Next, examples are given to motivate the need for variable-direction forgetting, especially in cases where the excitation is not persistent. Some of these results are well known, whereas others complement the prior literature. The goal is to provide a self-contained tutorial of the main ideas and techniques for students and researchers whose research may benefit from variable-direction forgetting. △ Less

Submitted 7 March, 2020; originally announced March 2020.

Comments: To appear in August 2020 issue of IEEE Control Systems Magazine

Journal ref: IEEE Control Systems Magazine ( Volume: 40, Issue: 4, Aug. 2020) 80-102

A Discrete-Time, Time-Delayed Lur'e Model with Biased Self-Excited Oscillations

Authors: Juan Paredes, Syed Aseem Ul Islam, Omran Kouba, Dennis S. Bernstein

Abstract: Self-excited systems arise in many applications, such as biochemical systems, mechanical systems with fluid-structure interaction, and fuel-driven systems with combustion dynamics. This paper presents a Lur'e model that exhibits biased self-excited oscillations under constant inputs. The model involves asymptotically stable linear dynamics, time delay, a washout filter, and a saturation nonlineari… ▽ More Self-excited systems arise in many applications, such as biochemical systems, mechanical systems with fluid-structure interaction, and fuel-driven systems with combustion dynamics. This paper presents a Lur'e model that exhibits biased self-excited oscillations under constant inputs. The model involves asymptotically stable linear dynamics, time delay, a washout filter, and a saturation nonlinearity. For all sufficiently large scalings of the loop transfer function, these components cause divergence under small signal levels and decay under large signal amplitudes, thus producing an oscillatory response. A bias-generation mechanism is used to specify the mean of the oscillation. The main contribution of the paper is a detailed analysis of a discrete-time version of this model. △ Less

Submitted 6 March, 2020; originally announced March 2020.

Comments: Submitted to IEEE Transactions on Automatic Control, 13 pages, 20 figures

Convergence and Consistency of Recursive Least Squares with Variable-Rate Forgetting

Authors: Adam L. Bruce, Ankit Goel, Dennis S. Bernstein

Abstract: A recursive least squares algorithm with variable rate forgetting (VRF) is derived by minimizing a quadratic cost function.Under persistent excitation and boundedness of the forgetting factor, the minimizer given by VRF is shown to converge to the true parameters. In addition, under persistent excitation and with noisy measurements, where the noise is uncorrelated with the regressor, conditions ar… ▽ More A recursive least squares algorithm with variable rate forgetting (VRF) is derived by minimizing a quadratic cost function.Under persistent excitation and boundedness of the forgetting factor, the minimizer given by VRF is shown to converge to the true parameters. In addition, under persistent excitation and with noisy measurements, where the noise is uncorrelated with the regressor, conditions are given under which the minimizer given by VRF is a consistent estimator of the true parameters.The results are illustrated by a numerical example involving abruptly changing parameters. △ Less

Submitted 5 March, 2020; originally announced March 2020.

Comments: Submitted to Automatica

Ordering-Based Causal Structure Learning in the Presence of Latent Variables

Authors: Daniel Irving Bernstein, Basil Saeed, Chandler Squires, Caroline Uhler

Abstract: We consider the task of learning a causal graph in the presence of latent confounders given i.i.d.~samples from the model. While current algorithms for causal structure discovery in the presence of latent confounders are constraint-based, we here propose a score-based approach. We prove that under assumptions weaker than faithfulness, any sparsest independence map (IMAP) of the distribution belong… ▽ More We consider the task of learning a causal graph in the presence of latent confounders given i.i.d.~samples from the model. While current algorithms for causal structure discovery in the presence of latent confounders are constraint-based, we here propose a score-based approach. We prove that under assumptions weaker than faithfulness, any sparsest independence map (IMAP) of the distribution belongs to the Markov equivalence class of the true model. This motivates the \emph{Sparsest Poset} formulation - that posets can be mapped to minimal IMAPs of the true model such that the sparsest of these IMAPs is Markov equivalent to the true model. Motivated by this result, we propose a greedy algorithm over the space of posets for causal structure discovery in the presence of latent confounders and compare its performance to the current state-of-the-art algorithms FCI and FCI+ on synthetic data. △ Less

Submitted 24 March, 2020; v1 submitted 20 October, 2019; originally announced October 2019.

Comments: To appear in AISTATS 2020