-
Activation thresholds and expressiveness of polynomial neural networks
Authors:
Bella Finkel,
Jose Israel Rodriguez,
Chenxi Wu,
Thomas Yahl
Abstract:
Polynomial neural networks have been implemented in a range of applications and present an advantageous framework for theoretical machine learning. A polynomial neural network of fixed architecture and activation degree gives an algebraic map from the network's weights to a set of polynomials. The image of this map is the space of functions representable by the network. Its Zariski closure is an a…
▽ More
Polynomial neural networks have been implemented in a range of applications and present an advantageous framework for theoretical machine learning. A polynomial neural network of fixed architecture and activation degree gives an algebraic map from the network's weights to a set of polynomials. The image of this map is the space of functions representable by the network. Its Zariski closure is an affine variety known as a neurovariety. The dimension of a polynomial neural network's neurovariety provides a measure of its expressivity. In this work, we introduce the notion of the activation threshold of a network architecture which expresses when the dimension of a neurovariety achieves its theoretical maximum. In addition, we prove expressiveness results for polynomial neural networks with equi-width~architectures.
△ Less
Submitted 8 August, 2024;
originally announced August 2024.
-
Decomposable sparse polynomial systems
Authors:
Taylor Brysiewicz,
Jose Israel Rodriguez,
Frank Sottile,
Thomas Yahl
Abstract:
The Macaulay2 package DecomposableSparseSystems implements methods for studying and numerically solving decomposable sparse polynomial systems. We describe the structure of decomposable sparse systems and explain how the methods in this package may be used to exploit this structure, with examples.
The Macaulay2 package DecomposableSparseSystems implements methods for studying and numerically solving decomposable sparse polynomial systems. We describe the structure of decomposable sparse systems and explain how the methods in this package may be used to exploit this structure, with examples.
△ Less
Submitted 4 June, 2020;
originally announced June 2020.
-
A Probabilistic Algorithm for Computing Data-Discriminants of Likelihood Equations
Authors:
Jose Israel Rodriguez,
Xiaoxian Tang
Abstract:
An algebraic approach to the maximum likelihood estimation problem is to solve a very structured parameterized polynomial system called likelihood equations that have finitely many complex (real or non-real) solutions. The only solutions that are statistically meaningful are the real solutions with positive coordinates. In order to classify the parameters (data) according to the number of real/pos…
▽ More
An algebraic approach to the maximum likelihood estimation problem is to solve a very structured parameterized polynomial system called likelihood equations that have finitely many complex (real or non-real) solutions. The only solutions that are statistically meaningful are the real solutions with positive coordinates. In order to classify the parameters (data) according to the number of real/positive solutions, we study how to efficiently compute the discriminants, say data-discriminants (DD), of the likelihood equations. We develop a probabilistic algorithm with three different strategies for computing DDs. Our implemented probabilistic algorithm based on Maple and FGb is more efficient than our previous version presented in ISSAC2015, and is also more efficient than the standard elimination for larger benchmarks. By applying RAGlib to a DD we compute, we give the real root classification of 3 by 3 symmetric matrix model.
△ Less
Submitted 10 June, 2016; v1 submitted 12 December, 2015;
originally announced December 2015.
-
Data-Discriminants of Likelihood Equations
Authors:
Jose Israel Rodriguez,
Xiaoxian Tang
Abstract:
Maximum likelihood estimation (MLE) is a fundamental computational problem in statistics. The problem is to maximize the likelihood function with respect to given data on a statistical model. An algebraic approach to this problem is to solve a very structured parameterized polynomial system called likelihood equations. For general choices of data, the number of complex solutions to the likelihood…
▽ More
Maximum likelihood estimation (MLE) is a fundamental computational problem in statistics. The problem is to maximize the likelihood function with respect to given data on a statistical model. An algebraic approach to this problem is to solve a very structured parameterized polynomial system called likelihood equations. For general choices of data, the number of complex solutions to the likelihood equations is finite and called the ML-degree of the model. The only solutions to the likelihood equations that are statistically meaningful are the real/positive solutions. However, the number of real/positive solutions is not characterized by the ML-degree. We use discriminants to classify data according to the number of real/positive solutions of the likelihood equations. We call these discriminants data-discriminants (DD). We develop a probabilistic algorithm for computing DDs. Experimental results show that, for the benchmarks we have tried, the probabilistic algorithm is more efficient than the standard elimination algorithm. Based on the computational results, we discuss the real root classification problem for the 3 by 3 symmetric matrix~model.
△ Less
Submitted 5 May, 2015; v1 submitted 1 January, 2015;
originally announced January 2015.
-
Bertini for Macaulay2
Authors:
Daniel J. Bates,
Elizabeth Gross,
Anton Leykin,
Jose Israel Rodriguez
Abstract:
Numerical algebraic geometry is the field of computational mathematics concerning the numerical solution of polynomial systems of equations. Bertini, a popular software package for computational applications of this field, includes implementations of a variety of algorithms based on polynomial homotopy continuation. The Macaulay2 package Bertini.m2 provides an interface to Bertini, making it possi…
▽ More
Numerical algebraic geometry is the field of computational mathematics concerning the numerical solution of polynomial systems of equations. Bertini, a popular software package for computational applications of this field, includes implementations of a variety of algorithms based on polynomial homotopy continuation. The Macaulay2 package Bertini.m2 provides an interface to Bertini, making it possible to access the core run modes of Bertini in Macaulay2. With these run modes, users can find approximate solutions to zero-dimensional systems and positive-dimensional systems, test numerically whether a point lies on a variety, sample numerically from a variety, and perform parameter homotopy runs.
△ Less
Submitted 11 October, 2013;
originally announced October 2013.
