Showing 1–50 of 114 results for author: Kanter, I

Towards a universal mechanism for successful deep learning

Authors: Yuval Meir, Yarden Tzach, Shiri Hodassman, Ofek Tevet, Ido Kanter

Abstract: Recently, the underlying mechanism for successful deep learning (DL) was presented based on a quantitative method that measures the quality of a single filter in each layer of a DL model, particularly VGG-16 trained on CIFAR-10. This method exemplifies that each filter identifies small clusters of possible output labels, with additional noise selected as labels outside the clusters. This feature i… ▽ More Recently, the underlying mechanism for successful deep learning (DL) was presented based on a quantitative method that measures the quality of a single filter in each layer of a DL model, particularly VGG-16 trained on CIFAR-10. This method exemplifies that each filter identifies small clusters of possible output labels, with additional noise selected as labels outside the clusters. This feature is progressively sharpened with each layer, resulting in an enhanced signal-to-noise ratio (SNR), which leads to an increase in the accuracy of the DL network. In this study, this mechanism is verified for VGG-16 and EfficientNet-B0 trained on the CIFAR-100 and ImageNet datasets, and the main results are as follows. First, the accuracy and SNR progressively increase with the layers. Second, for a given deep architecture, the maximal error rate increases approximately linearly with the number of output labels. Third, similar trends were obtained for dataset labels in the range [3, 1,000], thus supporting the universality of this mechanism. Understanding the performance of a single filter and its dominating features paves the way to highly dilute the deep architecture without affecting its overall accuracy, and this can be achieved by applying the filter's cluster connections (AFCC). △ Less

Submitted 12 March, 2024; v1 submitted 14 September, 2023; originally announced September 2023.

Comments: 31 pages,7 figures, 9 tables. arXiv admin note: text overlap with arXiv:2305.18078

Journal ref: Scientific Reports volume 14, Article number: 5881 (2024)

Statistical Mechanics of Learning via Reverberation in Bidirectional Associative Memories

Authors: Martino Salomone Centonze, Ido Kanter, Adriano Barra

Abstract: We study bi-directional associative neural networks that, exposed to noisy examples of an extensive number of random archetypes, learn the latter (with or without the presence of a teacher) when the supplied information is enough: in this setting, learning is heteroassociative -- involving couples of patterns -- and it is achieved by reverberating the information depicted from the examples through… ▽ More We study bi-directional associative neural networks that, exposed to noisy examples of an extensive number of random archetypes, learn the latter (with or without the presence of a teacher) when the supplied information is enough: in this setting, learning is heteroassociative -- involving couples of patterns -- and it is achieved by reverberating the information depicted from the examples through the layers of the network. By adapting Guerra's interpolation technique, we provide a full statistical mechanical picture of supervised and unsupervised learning processes (at the replica symmetric level of description) obtaining analytically phase diagrams, thresholds for learning, a picture of the ground-state in plain agreement with Monte Carlo simulations and signal-to-noise outcomes. In the large dataset limit, the Kosko storage prescription as well as its statistical mechanical picture provided by Kurchan, Peliti, and Saber in the eighties is fully recovered. Computational advantages in dealing with information reverberation, rather than storage, are discussed for natural test cases. In particular, we show how this network admits an integral representation in terms of two coupled restricted Boltzmann machines, whose hidden layers are entirely built of by grand-mother neurons, to prove that by coupling solely these grand-mother neurons we can correlate the patterns they are related to: it is thus possible to recover Pavlov's Classical Conditioning by adding just one synapse among the correct grand-mother neurons (hence saving an extensive number of these links for further information storage w.r.t. the classical autoassociative setting). △ Less

Submitted 17 July, 2023; originally announced July 2023.

The mechanism underlying successful deep learning

Authors: Yarden Tzach, Yuval Meir, Ofek Tevet, Ronit D. Gross, Shiri Hodassman, Roni Vardi, Ido Kanter

Abstract: Deep architectures consist of tens or hundreds of convolutional layers (CLs) that terminate with a few fully connected (FC) layers and an output layer representing the possible labels of a complex classification task. According to the existing deep learning (DL) rationale, the first CL reveals localized features from the raw data, whereas the subsequent layers progressively extract higher-level fe… ▽ More Deep architectures consist of tens or hundreds of convolutional layers (CLs) that terminate with a few fully connected (FC) layers and an output layer representing the possible labels of a complex classification task. According to the existing deep learning (DL) rationale, the first CL reveals localized features from the raw data, whereas the subsequent layers progressively extract higher-level features required for refined classification. This article presents an efficient three-phase procedure for quantifying the mechanism underlying successful DL. First, a deep architecture is trained to maximize the success rate (SR). Next, the weights of the first several CLs are fixed and only the concatenated new FC layer connected to the output is trained, resulting in SRs that progress with the layers. Finally, the trained FC weights are silenced, except for those emerging from a single filter, enabling the quantification of the functionality of this filter using a correlation matrix between input labels and averaged output fields, hence a well-defined set of quantifiable features is obtained. Each filter essentially selects a single output label independent of the input label, which seems to prevent high SRs; however, it counterintuitively identifies a small subset of possible output labels. This feature is an essential part of the underlying DL mechanism and is progressively sharpened with layers, resulting in enhanced signal-to-noise ratios and SRs. Quantitatively, this mechanism is exemplified by the VGG-16, VGG-6, and AVGG-16. The proposed mechanism underlying DL provides an accurate tool for identifying each filter's quality and is expected to direct additional procedures to improve the SR, computational complexity, and latency of DL. △ Less

Submitted 29 May, 2023; originally announced May 2023.

Comments: 33 pages, 8 figures

Enhancing the accuracies by performing pooling decisions adjacent to the output layer

Authors: Yuval Meir, Yarden Tzach, Ronit D. Gross, Ofek Tevet, Roni Vardi, Ido Kanter

Abstract: Learning classification tasks of (2^nx2^n) inputs typically consist of \le n (2x2) max-pooling (MP) operators along the entire feedforward deep architecture. Here we show, using the CIFAR-10 database, that pooling decisions adjacent to the last convolutional layer significantly enhance accuracies. In particular, average accuracies of the advanced-VGG with m layers (A-VGGm) architectures are 0.936,… ▽ More Learning classification tasks of (2^nx2^n) inputs typically consist of \le n (2x2) max-pooling (MP) operators along the entire feedforward deep architecture. Here we show, using the CIFAR-10 database, that pooling decisions adjacent to the last convolutional layer significantly enhance accuracies. In particular, average accuracies of the advanced-VGG with m layers (A-VGGm) architectures are 0.936, 0.940, 0.954, 0.955, and 0.955 for m=6, 8, 14, 13, and 16, respectively. The results indicate A-VGG8s' accuracy is superior to VGG16s', and that the accuracies of A-VGG13 and A-VGG16 are equal, and comparable to that of Wide-ResNet16. In addition, replacing the three fully connected (FC) layers with one FC layer, A-VGG6 and A-VGG14, or with several linear activation FC layers, yielded similar accuracies. These significantly enhanced accuracies stem from training the most influential input-output routes, in comparison to the inferior routes selected following multiple MP decisions along the deep architecture. In addition, accuracies are sensitive to the order of the non-commutative MP and average pooling operators adjacent to the output layer, varying the number and location of training routes. The results call for the reexamination of previously proposed deep architectures and their accuracies by utilizing the proposed pooling strategy adjacent to the output layer. △ Less

Submitted 31 August, 2023; v1 submitted 10 March, 2023; originally announced March 2023.

Comments: 29 pages, 3 figures, 1 table, and Supplementary Information

Journal ref: Sci Rep 13, 13385 (2023)

Learning on tree architectures outperforms a convolutional feedforward network

Authors: Yuval Meir, Itamar Ben-Noam, Yarden Tzach, Shiri Hodassman, Ido Kanter

Abstract: Advanced deep learning architectures consist of tens of fully connected and convolutional hidden layers, currently extended to hundreds, are far from their biological realization. Their implausible biological dynamics relies on changing a weight in a non-local manner, as the number of routes between an output unit and a weight is typically large, using the backpropagation technique. Here, a 3-laye… ▽ More Advanced deep learning architectures consist of tens of fully connected and convolutional hidden layers, currently extended to hundreds, are far from their biological realization. Their implausible biological dynamics relies on changing a weight in a non-local manner, as the number of routes between an output unit and a weight is typically large, using the backpropagation technique. Here, a 3-layer tree architecture inspired by experimental-based dendritic tree adaptations is developed and applied to the offline and online learning of the CIFAR-10 database. The proposed architecture outperforms the achievable success rates of the 5-layer convolutional LeNet. Moreover, the highly pruned tree backpropagation approach of the proposed architecture, where a single route connects an output unit and a weight, represents an efficient dendritic deep learning. △ Less

Submitted 30 January, 2023; v1 submitted 21 November, 2022; originally announced November 2022.

Comments: 21 pages, 4 figures, 2 table

Journal ref: Sci Rep 13, 962 (2023)

Efficient shallow learning as an alternative to deep learning

Authors: Yuval Meir, Ofek Tevet, Yarden Tzach, Shiri Hodassman, Ronit D. Gross, Ido Kanter

Abstract: The realization of complex classification tasks requires training of deep learning (DL) architectures consisting of tens or even hundreds of convolutional and fully connected hidden layers, which is far from the reality of the human brain. According to the DL rationale, the first convolutional layer reveals localized patterns in the input and large-scale patterns in the following layers, until it… ▽ More The realization of complex classification tasks requires training of deep learning (DL) architectures consisting of tens or even hundreds of convolutional and fully connected hidden layers, which is far from the reality of the human brain. According to the DL rationale, the first convolutional layer reveals localized patterns in the input and large-scale patterns in the following layers, until it reliably characterizes a class of inputs. Here, we demonstrate that with a fixed ratio between the depths of the first and second convolutional layers, the error rates of the generalized shallow LeNet architecture, consisting of only five layers, decay as a power law with the number of filters in the first convolutional layer. The extrapolation of this power law indicates that the generalized LeNet can achieve small error rates that were previously obtained for the CIFAR-10 database using DL architectures. A power law with a similar exponent also characterizes the generalized VGG-16 architecture. However, this results in a significantly increased number of operations required to achieve a given error rate with respect to LeNet. This power law phenomenon governs various generalized LeNet and VGG-16 architectures, hinting at its universal behavior and suggesting a quantitative hierarchical time-space complexity among machine learning architectures. Additionally, the conservation law along the convolutional layers, which is the square-root of their size times their depth, is found to asymptotically minimize error rates. The efficient shallow learning that is demonstrated in this study calls for further quantitative examination using various databases and architectures and its accelerated implementation using future dedicated hardware developments. △ Less

Submitted 23 November, 2022; v1 submitted 15 November, 2022; originally announced November 2022.

Comments: 26 pages, 4 figures (improved figures resolution)

Report number: https://www.nature.com/articles/s41598-023-32559-8

Journal ref: Sci Rep 13, 5423 (2023)

Power-law Scaling to Assist with Key Challenges in Artificial Intelligence

Authors: Yuval Meir, Shira Sardi, Shiri Hodassman, Karin Kisos, Itamar Ben-Noam, Amir Goldental, Ido Kanter

Abstract: Power-law scaling, a central concept in critical phenomena, is found to be useful in deep learning, where optimized test errors on handwritten digit examples converge as a power-law to zero with database size. For rapid decision making with one training epoch, each example is presented only once to the trained network, the power-law exponent increased with the number of hidden layers. For the larg… ▽ More Power-law scaling, a central concept in critical phenomena, is found to be useful in deep learning, where optimized test errors on handwritten digit examples converge as a power-law to zero with database size. For rapid decision making with one training epoch, each example is presented only once to the trained network, the power-law exponent increased with the number of hidden layers. For the largest dataset, the obtained test error was estimated to be in the proximity of state-of-the-art algorithms for large epoch numbers. Power-law scaling assists with key challenges found in current artificial intelligence applications and facilitates an a priori dataset size estimation to achieve a desired test accuracy. It establishes a benchmark for measuring training complexity and a quantitative hierarchy of machine learning tasks and algorithms. △ Less

Submitted 15 November, 2022; originally announced November 2022.

Comments: 30 pages, 5 figures

Journal ref: Sci Rep 10, 19628 (2020)

Recurrent neural networks that generalize from examples and optimize by dreaming

Authors: Miriam Aquaro, Francesco Alemanno, Ido Kanter, Fabrizio Durante, Elena Agliari, Adriano Barra

Abstract: The gap between the huge volumes of data needed to train artificial neural networks and the relatively small amount of data needed by their biological counterparts is a central puzzle in machine learning. Here, inspired by biological information-processing, we introduce a generalized Hopfield network where pairwise couplings between neurons are built according to Hebb's prescription for on-line le… ▽ More The gap between the huge volumes of data needed to train artificial neural networks and the relatively small amount of data needed by their biological counterparts is a central puzzle in machine learning. Here, inspired by biological information-processing, we introduce a generalized Hopfield network where pairwise couplings between neurons are built according to Hebb's prescription for on-line learning and allow also for (suitably stylized) off-line sleeping mechanisms. Moreover, in order to retain a learning framework, here the patterns are not assumed to be available, instead, we let the network experience solely a dataset made of a sample of noisy examples for each pattern. We analyze the model by statistical-mechanics tools and we obtain a quantitative picture of its capabilities as functions of its control parameters: the resulting network is an associative memory for pattern recognition that learns from examples on-line, generalizes and optimizes its storage capacity by off-line sleeping. Remarkably, the sleeping mechanisms always significantly reduce (up to $\approx 90\%$) the dataset size required to correctly generalize, further, there are memory loads that are prohibitive to Hebbian networks without sleeping (no matter the size and quality of the provided examples), but that are easily handled by the present "rested" neural networks. △ Less

Submitted 17 April, 2022; originally announced April 2022.

Brain inspired neuronal silencing mechanism to enable reliable sequence identification

Authors: Shiri Hodassman, Yuval Meir, Karin Kisos, Itamar Ben-Noam, Yael Tugendhaft, Amir Goldental, Roni Vardi, Ido Kanter

Abstract: Real-time sequence identification is a core use-case of artificial neural networks (ANNs), ranging from recognizing temporal events to identifying verification codes. Existing methods apply recurrent neural networks, which suffer from training difficulties; however, performing this function without feedback loops remains a challenge. Here, we present an experimental neuronal long-term plasticity m… ▽ More Real-time sequence identification is a core use-case of artificial neural networks (ANNs), ranging from recognizing temporal events to identifying verification codes. Existing methods apply recurrent neural networks, which suffer from training difficulties; however, performing this function without feedback loops remains a challenge. Here, we present an experimental neuronal long-term plasticity mechanism for high-precision feedforward sequence identification networks (ID-nets) without feedback loops, wherein input objects have a given order and timing. This mechanism temporarily silences neurons following their recent spiking activity. Therefore, transitory objects act on different dynamically created feedforward sub-networks. ID-nets are demonstrated to reliably identify 10 handwritten digit sequences, and are generalized to deep convolutional ANNs with continuous activation nodes trained on image sequences. Counterintuitively, their classification performance, even with a limited number of training examples, is high for sequences but low for individual objects. ID-nets are also implemented for writer-dependent recognition, and suggested as a cryptographic tool for encrypted authentication. The presented mechanism opens new horizons for advanced ANN algorithms. △ Less

Submitted 2 October, 2022; v1 submitted 24 March, 2022; originally announced March 2022.

Comments: 38 pages, 11 figures

Journal ref: Sci Rep 12, 16003 (2022)

Supervised Hebbian Learning

Authors: Francesco Alemanno, Miriam Aquaro, Ido Kanter, Adriano Barra, Elena Agliari

Abstract: In neural network's Literature, Hebbian learning traditionally refers to the procedure by which the Hopfield model and its generalizations store archetypes (i.e., definite patterns that are experienced just once to form the synaptic matrix). However, the term "Learning" in Machine Learning refers to the ability of the machine to extract features from the supplied dataset (e.g., made of blurred exa… ▽ More In neural network's Literature, Hebbian learning traditionally refers to the procedure by which the Hopfield model and its generalizations store archetypes (i.e., definite patterns that are experienced just once to form the synaptic matrix). However, the term "Learning" in Machine Learning refers to the ability of the machine to extract features from the supplied dataset (e.g., made of blurred examples of these archetypes), in order to make its own representation of the unavailable archetypes. Here, given a sample of examples, we define a supervised learning protocol by which the Hopfield network can infer the archetypes, and we detect the correct control parameters (including size and quality of the dataset) to depict a phase diagram for the system performance. We also prove that, for structureless datasets, the Hopfield model equipped with this supervised learning rule is equivalent to a restricted Boltzmann machine and this suggests an optimal and interpretable training routine. Finally, this approach is generalized to structured datasets: we highlight a quasi-ultrametric organization (reminiscent of replica-symmetry-breaking) in the analyzed datasets and, consequently, we introduce an additional "replica hidden layer" for its (partial) disentanglement, which is shown to improve MNIST classification from 75% to 95%, and to offer a new perspective on deep architectures. △ Less

Submitted 7 September, 2022; v1 submitted 2 March, 2022; originally announced March 2022.

Report number: Roma01.Math.MP

Journal ref: Europhysics Letters, Volume 141, Number 1 (2034)

Long anisotropic absolute refractory periods with rapid rise-times to reliable responsiveness

Authors: Shira Sardi, Roni Vardi, Yael Tugendhaft, Anton Sheinin, Amir Goldental, Ido Kanter

Abstract: Refractoriness is a fundamental property of excitable elements, such as neurons, indicating the probability for re-excitation in a given time-lag, and is typically linked to the neuronal hyperpolarization following an evoked spike. Here we measured the refractory periods (RPs) in neuronal cultures and observed that an average anisotropic absolute RP could exceed 10 milliseconds and its tail 20 mil… ▽ More Refractoriness is a fundamental property of excitable elements, such as neurons, indicating the probability for re-excitation in a given time-lag, and is typically linked to the neuronal hyperpolarization following an evoked spike. Here we measured the refractory periods (RPs) in neuronal cultures and observed that an average anisotropic absolute RP could exceed 10 milliseconds and its tail 20 milliseconds, independent of a large stimulation frequency range. It is an order of magnitude longer than anticipated and comparable with the decaying membrane potential timescale. It is followed by a sharp rise-time (relative RP) of merely ~1 millisecond to complete responsiveness. Extracellular stimulations result in longer absolute RPs than solely intracellular ones, and a pair of extracellular stimulations from two different routes exhibits distinct absolute RPs, depending on their order. Our results indicate that a neuron is an accurate excitable element, where the diverse RPs cannot be attributed solely to the soma and imply fast mutual interactions between different stimulation routes and dendrites. Further elucidation of neuronal computational capabilities and their interplay with adaptation mechanisms is warranted. △ Less

Submitted 3 January, 2022; v1 submitted 4 November, 2021; originally announced November 2021.

Comments: 28 pages, 6 figures

Journal ref: Phys. Rev. E 105, 014401 (2022)

Significant anisotropic neuronal refractory period plasticity

Authors: Roni Vardi, Yael Tugendhaft, Shira Sardi, Ido Kanter

Abstract: Refractory periods are an unavoidable feature of excitable elements, resulting in necessary time-lags for re-excitation. Herein, we measure neuronal absolute refractory periods (ARPs) in synaptic blocked neuronal cultures. In so doing, we show that their duration can be significantly extended by dozens of milliseconds using preceding evoked spikes generated by extracellular stimulations. The ARP i… ▽ More Refractory periods are an unavoidable feature of excitable elements, resulting in necessary time-lags for re-excitation. Herein, we measure neuronal absolute refractory periods (ARPs) in synaptic blocked neuronal cultures. In so doing, we show that their duration can be significantly extended by dozens of milliseconds using preceding evoked spikes generated by extracellular stimulations. The ARP increases with the frequency of preceding stimulations, and saturates at the intermittent phase of the neuronal response latency, where a short relative refractory period might appear. Nevertheless, preceding stimulations via a different extracellular route does not affect the ARP. It is also found to be independent of preceding intracellular stimulations. All these features strongly suggest that the anisotropic ARPs originate in neuronal dendrites. The results demonstrate the fast and significant plasticity of the neuronal ARP, depending on the firing activity of its connecting neurons, which is expected to affect network dynamics. △ Less

Submitted 5 September, 2021; originally announced September 2021.

Comments: 16 pages, 11 figures

Report number: https://iopscience.iop.org/article/10.1209/0295-5075/ac177a/meta

Journal ref: 2021 EPL 134 60007

Brain experiments imply adaptation mechanisms which outperform common AI learning algorithms

Authors: Shira Sardi, Roni Vardi, Yuval Meir, Yael Tugendhaft, Shiri Hodassman, Amir Goldental, Ido Kanter

Abstract: Attempting to imitate the brain functionalities, researchers have bridged between neuroscience and artificial intelligence for decades; however, experimental neuroscience has not directly advanced the field of machine learning. Here, using neuronal cultures, we demonstrate that increased training frequency accelerates the neuronal adaptation processes. This mechanism was implemented on artificial… ▽ More Attempting to imitate the brain functionalities, researchers have bridged between neuroscience and artificial intelligence for decades; however, experimental neuroscience has not directly advanced the field of machine learning. Here, using neuronal cultures, we demonstrate that increased training frequency accelerates the neuronal adaptation processes. This mechanism was implemented on artificial neural networks, where a local learning step-size increases for coherent consecutive learning steps and tested on a simple dataset of handwritten digits, MNIST. Based on our online learning results with a few handwriting examples, success rates for brain-inspired algorithms substantially outperform the commonly used machine learning algorithms. We speculate this emerging bridge from slow brain function to machine learning will promote ultrafast decision making under limited examples, which is the reality in many aspects of human activity, robotic control, and network optimization. △ Less

Submitted 23 April, 2020; originally announced May 2020.

Comments: 30 pages, 7 figures

Journal ref: Scientific Reports 10, Article number: 6923 (2020) https://www.nature.com/articles/s41598-020-63755-5

Embedding information in physically generated random bit sequences while maintaining certified randomness

Authors: Shira Sardi, Herut Uzan, Shiri Otmazgin, Yaara Aviad, Michael Rosenbluh, Ido Kanter

Abstract: Ultrafast physical random bit generation at hundreds of Gb/s rates, with verified randomness, is a crucial ingredient in secure communication and have recently emerged using optics based physical systems. Here we examine the inverse problem and measure the ratio of information bits that can be systematically embedded in a random bit sequence without degrading its certified randomness. These ratios… ▽ More Ultrafast physical random bit generation at hundreds of Gb/s rates, with verified randomness, is a crucial ingredient in secure communication and have recently emerged using optics based physical systems. Here we examine the inverse problem and measure the ratio of information bits that can be systematically embedded in a random bit sequence without degrading its certified randomness. These ratios exceed 0.01 in experimentally obtained long random bit sequences. Based on these findings we propose a high-capacity private-key cryptosystem with a finite key length, where the existence as well as the content of the communication is concealed in the random sequence. Our results call for a rethinking of the current quantitative definition of practical classical randomness as well as the measure of randomness generated by quantum methods, which have to include bounds using the proposed inverse information embedding method. △ Less

Submitted 31 October, 2019; originally announced November 2019.

Comments: EPL in press

Journal ref: EPL, vol. 127 (2019) 60003

Simultaneous multi-patch-clamp and extracellular-array recordings: Single neuron reflects network activity

Authors: Roni Vardi, Amir Goldental, Shira Sardi, Anton Sheinin, Ido Kanter

Abstract: The increasing number of recording electrodes enhances the capability of capturing the network's cooperative activity, however, using too many monitors might alter the properties of the measured neural network and induce noise. Using a technique that merges simultaneous multi-patch-clamp and multi-electrode array recordings of neural networks in-vitro, we show that the membrane potential of a sing… ▽ More The increasing number of recording electrodes enhances the capability of capturing the network's cooperative activity, however, using too many monitors might alter the properties of the measured neural network and induce noise. Using a technique that merges simultaneous multi-patch-clamp and multi-electrode array recordings of neural networks in-vitro, we show that the membrane potential of a single neuron is a reliable and super-sensitive probe for monitoring such cooperative activities and their detailed rhythms. Specifically, the membrane potential and the spiking activity of a single neuron are either highly correlated or highly anti-correlated with the time-dependent macroscopic activity of the entire network. This surprising observation also sheds light on the cooperative origin of neuronal burst in cultured networks. Our findings present an alternative flexible approach to the technique based on a massive tiling of networks by large-scale arrays of electrodes to monitor their activity. △ Less

Submitted 20 July, 2017; originally announced July 2017.

Comments: 36 pages, 9 figures

Journal ref: Scientific Reports 6, Article number: 36228 (2016)

Vitality of Neural Networks under Reoccurring Catastrophic Failures

Authors: Shira Sardi, Amir Goldental, Hamutal Amir, Roni Vardi, Ido Kanter

Abstract: Catastrophic failures are complete and sudden collapses in the activity of large networks such as economics, electrical power grids and computer networks, which typically require a manual recovery process. Here we experimentally show that excitatory neural networks are governed by a non-Poissonian reoccurrence of catastrophic failures, where their repetition time follows a multimodal distribution… ▽ More Catastrophic failures are complete and sudden collapses in the activity of large networks such as economics, electrical power grids and computer networks, which typically require a manual recovery process. Here we experimentally show that excitatory neural networks are governed by a non-Poissonian reoccurrence of catastrophic failures, where their repetition time follows a multimodal distribution characterized by a few tenths of a second and tens of seconds timescales. The mechanism underlying the termination and reappearance of network activity is quantitatively shown here to be associated with nodal time-dependent features, neuronal plasticity, where hyperactive nodes damage the response capability of their neighbors. It presents a complementary mechanism for the emergence of Poissonian catastrophic failures from damage conductivity. The effect that hyperactive nodes degenerate their neighbors represents a type of local competition which is a common feature in the dynamics of real-world complex networks, whereas their spontaneous recoveries represent a vitality which enhances reliable functionality. △ Less

Submitted 20 July, 2017; originally announced July 2017.

Comments: 22 pages, 7 figures

Journal ref: Scientific Reports 6, Article number: 31674 (2016)

Fast Reversible Learning based on Neurons functioning as Anisotropic Multiplex Hubs

Authors: Roni Vardi, Amir Goldental, Anton Sheinin, Shira Sardi, Ido Kanter

Abstract: Neural networks are composed of neurons and synapses, which are responsible for learning in a slow adaptive dynamical process. Here we experimentally show that neurons act like independent anisotropic multiplex hubs, which relay and mute incoming signals following their input directions. Theoretically, the observed information routing enriches the computational capabilities of neurons by allowing,… ▽ More Neural networks are composed of neurons and synapses, which are responsible for learning in a slow adaptive dynamical process. Here we experimentally show that neurons act like independent anisotropic multiplex hubs, which relay and mute incoming signals following their input directions. Theoretically, the observed information routing enriches the computational capabilities of neurons by allowing, for instance, equalization among different information routes in the network, as well as high-frequency transmission of complex time-dependent signals constructed via several parallel routes. In addition, this kind of hubs adaptively eliminate very noisy neurons from the dynamics of the network, preventing masking of information transmission. The timescales for these features are several seconds at most, as opposed to the imprint of information by the synaptic plasticity, a process which exceeds minutes. Results open the horizon to the understanding of fast and adaptive learning realities in higher cognitive functionalities of the brain. △ Less

Submitted 20 July, 2017; originally announced July 2017.

Comments: 6 pages, 4 figures

Journal ref: EPL 118 (2017) 46002

Oscillations in networks of networks stem from adaptive nodes with memory

Authors: Amir Goldental, Herut Uzan, Shira Sardi, Ido Kanter

Abstract: We present an analytical framework that allows the quantitative study of statistical dynamic properties of networks with adaptive nodes that have memory and is used to examine the emergence of oscillations in networks with response failures. The frequency of the oscillations was quantitatively found to increase with the excitability of the nodes and with the average degree of the network and to de… ▽ More We present an analytical framework that allows the quantitative study of statistical dynamic properties of networks with adaptive nodes that have memory and is used to examine the emergence of oscillations in networks with response failures. The frequency of the oscillations was quantitatively found to increase with the excitability of the nodes and with the average degree of the network and to decrease with delays between nodes. For networks of networks, diverse cluster oscillation modes were found as a function of the topology. Analytical results are in agreement with large-scale simulations and open the horizon for understanding network dynamics composed of finite memory nodes as well as their different phases of activity. △ Less

Submitted 17 July, 2017; originally announced July 2017.

Comments: 17 pages, 4 figures

Journal ref: Scientific Reports 7, Article number: 2700 (2017)

Mimicking Collective Firing Patterns of Hundreds of Connected Neurons using a Single-Neuron Experiment

Authors: Amir Goldental, Pinhas Sabo, Shira Sardi, Roni Vardi, Ido Kanter

Abstract: The experimental study of neural networks requires simultaneous measurements of a massive number of neurons, while monitoring properties of the connectivity, synaptic strengths and delays. Current technological barriers make such a mission unachievable. In addition, as a result of the enormous number of required measurements, the estimated network parameters would differ from the original ones. He… ▽ More The experimental study of neural networks requires simultaneous measurements of a massive number of neurons, while monitoring properties of the connectivity, synaptic strengths and delays. Current technological barriers make such a mission unachievable. In addition, as a result of the enormous number of required measurements, the estimated network parameters would differ from the original ones. Here we present a versatile experimental technique, which enables the study of recurrent neural networks activity while being capable of dictating the network connectivity and synaptic strengths. This method is based on the observation that the response of neurons depends solely on their recent stimulations, a short-term memory. It allows a long-term scheme of stimulation and recording of a single neuron, to mimic simultaneous activity measurements of neurons in a recurrent network. Utilization of this technique demonstrates the spontaneous emergence of cooperative synchronous oscillations, in particular the coexistence of fast Gamma and slow Delta oscillations, and opens the horizon for the experimental study of other cooperative phenomena within large-scale neural networks. △ Less

Submitted 10 January, 2016; originally announced January 2016.

Comments: 26 pages and 6 figures, http://journal.frontiersin.org/article/10.3389/fnins.2015.00508/

Journal ref: Front. Neurosci. 9:508 (2015)

Broadband Macroscopic Cortical Oscillations Emerge from Intrinsic Neuronal Response Failures

Authors: Amir Goldental, Roni Vardi, Shira Sardi, Pinhas Sabo, Ido Kanter

Abstract: Broadband spontaneous macroscopic neural oscillations are rhythmic cortical firing which were extensively examined during the last century, however, their possible origination is still controversial. In this work we show how macroscopic oscillations emerge in solely excitatory random networks and without topological constraints. We experimentally and theoretically show that these oscillations stem… ▽ More Broadband spontaneous macroscopic neural oscillations are rhythmic cortical firing which were extensively examined during the last century, however, their possible origination is still controversial. In this work we show how macroscopic oscillations emerge in solely excitatory random networks and without topological constraints. We experimentally and theoretically show that these oscillations stem from the counterintuitive underlying mechanism - the intrinsic stochastic neuronal response failures. These neuronal response failures, which are characterized by short-term memory, lead to cooperation among neurons, resulting in sub- or several- Hertz macroscopic oscillations which coexist with high frequency gamma oscillations. A quantitative interplay between the statistical network properties and the emerging oscillations is supported by simulations of large networks based on single-neuron in-vitro experiments and a Langevin equation describing the network dynamics. Results call for the examination of these oscillations in the presence of inhibition and external drives. △ Less

Submitted 1 November, 2015; originally announced November 2015.

Comments: 21 pages, 5 figures

Journal ref: Front. Neural Circuits 9:65 (2015)

Neuronal Response Impedance Mechanism Implementing Cooperative Networks with Low Firing Rates and Microseconds Precision

Authors: Roni Vardi, Amir Goldental, Hagar Marmari, Haya Brama, Edward Stern, Shira Sardi, Pinhas Sabo, Ido Kanter

Abstract: Realizations of low firing rates in neural networks usually require globally balanced distributions among excitatory and inhibitory links, while feasibility of temporal coding is limited by neuronal millisecond precision. We show that cooperation, governing global network features, emerges through nodal properties, as opposed to link distributions. Using in vitro and in vivo experiments we demonst… ▽ More Realizations of low firing rates in neural networks usually require globally balanced distributions among excitatory and inhibitory links, while feasibility of temporal coding is limited by neuronal millisecond precision. We show that cooperation, governing global network features, emerges through nodal properties, as opposed to link distributions. Using in vitro and in vivo experiments we demonstrate microsecond precision of neuronal response timings under low stimulation frequencies, whereas moderate frequencies result in a chaotic neuronal phase characterized by degraded precision. Above a critical stimulation frequency, which varies among neurons, response failures were found to emerge stochastically such that the neuron functions as a low pass filter, saturating the average inter-spike-interval. This intrinsic neuronal response impedance mechanism leads to cooperation on a network level, such that firing rates are suppressed towards the lowest neuronal critical frequency simultaneously with neuronal microsecond precision. Our findings open up opportunities of controlling global features of network dynamics through few nodes with extreme properties. △ Less

Submitted 1 June, 2015; v1 submitted 19 November, 2014; originally announced November 2014.

Comments: 35 pages, 13 figures

Journal ref: Front. Neural Circuits 9:29 (2015)

Chaotic and non-chaotic phases in experimental responses of a single neuron

Authors: Hagar Marmari, Roni Vardi, Ido Kanter

Abstract: Consistency and predictability of brain functionalities depend on reproducible activity of a single neuron. We identify a reproducible non-chaotic neuronal phase where deviations between concave response latency profiles of a single neuron do not increase with the number of stimulations. A chaotic neuronal phase emerges at a transition to convex latency profiles which diverge exponentially, indica… ▽ More Consistency and predictability of brain functionalities depend on reproducible activity of a single neuron. We identify a reproducible non-chaotic neuronal phase where deviations between concave response latency profiles of a single neuron do not increase with the number of stimulations. A chaotic neuronal phase emerges at a transition to convex latency profiles which diverge exponentially, indicating irreproducible response timings. Our findings are supported by a quantitative mathematical framework and found robust to periodic and random stimulation patterns. In addition, these results put a bound on the neuronal temporal resolution which can be enhanced below a millisecond using neuronal chains. △ Less

Submitted 27 April, 2014; v1 submitted 22 April, 2014; originally announced April 2014.

Comments: 6 pages, 4 figures

Journal ref: EPL 106 (2014) 46002

Computational paradigm for dynamic logic-gates in neuronal activity

Authors: Amir Goldental, Shoshana Guberman, Roni Vardi, Ido Kanter

Abstract: In 1943 McCulloch and Pitts suggested that the brain is composed of reliable logic-gates similar to the logic at the core of today's computers. This framework had a limited impact on neuroscience, since neurons exhibit far richer dynamics. Here we propose a new experimentally corroborated paradigm in which the truth tables of the brain's logic-gates are time dependent, i.e. dynamic logicgates (DLG… ▽ More In 1943 McCulloch and Pitts suggested that the brain is composed of reliable logic-gates similar to the logic at the core of today's computers. This framework had a limited impact on neuroscience, since neurons exhibit far richer dynamics. Here we propose a new experimentally corroborated paradigm in which the truth tables of the brain's logic-gates are time dependent, i.e. dynamic logicgates (DLGs). The truth tables of the DLGs depend on the history of their activity and the stimulation frequencies of their input neurons. Our experimental results are based on a procedure where conditioned stimulations were enforced on circuits of neurons embedded within a large-scale network of cortical cells in-vitro. We demonstrate that the underlying biological mechanism is the unavoidable increase of neuronal response latencies to ongoing stimulations, which imposes a nonuniform gradual stretching of network delays. The limited experimental results are confirmed and extended by simulations and theoretical arguments based on identical neurons with a fixed increase of the neuronal response latency per evoked spike. We anticipate our results to lead to better understanding of the suitability of this computational paradigm to account for the brain's functionalities and will require the development of new systematic mathematical methods beyond the methods developed for traditional Boolean algebra. △ Less

Submitted 15 April, 2014; originally announced April 2014.

Comments: 32 pages, 14 figures, 1 table

Journal ref: Front. Comput. Neurosci. 8:52. (2014)

Error correction and fast detectors implemented by ultra-fast neuronal plasticity

Authors: Roni Vardi, Hagar Marmari, Ido Kanter

Abstract: We experimentally show that the neuron functions as a precise time-integrator, where the accumulated changes in neuronal response latencies, under complex and random stimulation patterns, are solely a function of a global quantity, the average time-lag between stimulations. In contrast, momentary leaps in the neuronal response latency follow trends of consecutive stimulations, indicating ultra-fas… ▽ More We experimentally show that the neuron functions as a precise time-integrator, where the accumulated changes in neuronal response latencies, under complex and random stimulation patterns, are solely a function of a global quantity, the average time-lag between stimulations. In contrast, momentary leaps in the neuronal response latency follow trends of consecutive stimulations, indicating ultra-fast neuronal plasticity. On a circuit level, this ultra-fast neuronal plasticity phenomenon implements error-correction mechanisms and fast detectors for misplaced stimulations. Additionally, at moderate/high stimulation rates this phenomenon destabilizes/stabilizes a periodic neuronal activity disrupted by misplaced stimulations. △ Less

Submitted 9 April, 2014; originally announced April 2014.

Comments: 7 pages, 4 figures, 1 table, to appear in Physical Review E

Journal ref: Phys. Rev. E 89, 042712 (2014)

Sudden synchrony leaps accompanied by frequency multiplications in neuronal activity

Authors: Roni Vardi, Amir Goldental, Shoshana Guberman, Alexander Kalmanovich, Hagar Marmari, Ido Kanter

Abstract: A classical view of neural coding relies on temporal firing synchrony among functional groups of neurons; however the underlying mechanism remains an enigma. Here we experimentally demonstrate a mechanism where time-lags among neuronal spiking leap from several tens of milliseconds to nearly zero-lag synchrony. It also allows sudden leaps out of synchrony, hence forming short epochs of synchrony.… ▽ More A classical view of neural coding relies on temporal firing synchrony among functional groups of neurons; however the underlying mechanism remains an enigma. Here we experimentally demonstrate a mechanism where time-lags among neuronal spiking leap from several tens of milliseconds to nearly zero-lag synchrony. It also allows sudden leaps out of synchrony, hence forming short epochs of synchrony. Our results are based on an experimental procedure where conditioned stimulations were enforced on circuits of neurons embedded within a large-scale network of cortical cells in vitro and are corroborated by simulations of neuronal populations. The underlying biological mechanisms are the unavoidable increase of the neuronal response latency to ongoing stimulations and temporal or spatial summation required to generate evoked spikes. These sudden leaps in and out of synchrony may be accompanied by multiplications of the neuronal firing frequency, hence offering reliable information-bearing indicators which may bridge between the two principal neuronal coding paradigms. △ Less

Submitted 30 October, 2013; originally announced October 2013.

Comments: 23 pages, 3 figures

Journal ref: Front. Neural Circuits, 30 October 2013

An experimental evidence-based computational paradigm for new logic-gates in neuronal activity

Authors: Roni Vardi, Shoshana Guberman, Amir Goldental, Ido Kanter

Abstract: We propose a new experimentally corroborated paradigm in which the functionality of the brain's logic-gates depends on the history of their activity, e.g. an OR-gate that turns into a XOR-gate over time. Our results are based on an experimental procedure where conditioned stimulations were enforced on circuits of neurons embedded within a large-scale network of cortical cells in-vitro. The underly… ▽ More We propose a new experimentally corroborated paradigm in which the functionality of the brain's logic-gates depends on the history of their activity, e.g. an OR-gate that turns into a XOR-gate over time. Our results are based on an experimental procedure where conditioned stimulations were enforced on circuits of neurons embedded within a large-scale network of cortical cells in-vitro. The underlying biological mechanism is the unavoidable increase of neuronal response latency to ongoing stimulations, which imposes a non-uniform gradual stretching of network delays. △ Less

Submitted 2 May, 2013; originally announced May 2013.

Comments: 10 pages, 4 figures, 1 table

Journal ref: EPL 103, 66001 (2013)

Time delayed laser networks: phase versus chaos synchronization

Authors: I. Reidler, M. Nixon, Y. Aviad, S. Guberman, A. A. Friesem, M. Rosenbluh, N. Davidson, I. Kanter

Abstract: The synchronization of chaotic lasers and the optical phase synchronization of light originating in multiple coupled lasers have both been extensively studied, however, the interplay between these two phenomena, especially at the network level is unexplored. Here we experimentally compare chaos synchronization of laser networks with heterogeneous coupling delay times to phase synchronization of si… ▽ More The synchronization of chaotic lasers and the optical phase synchronization of light originating in multiple coupled lasers have both been extensively studied, however, the interplay between these two phenomena, especially at the network level is unexplored. Here we experimentally compare chaos synchronization of laser networks with heterogeneous coupling delay times to phase synchronization of similar networks. While chaotic lasers exhibit deterioration in synchronization as the network time delay heterogeneity increases, phase synchronization is found to be independent of heterogeneity. The experimental results are found to be in agreement with numerical simulations for semiconductor lasers. △ Less

Submitted 3 April, 2013; originally announced April 2013.

Chaos Pass Filter: Linear Response of Synchronized Chaotic Systems

Authors: Steffen Zeeb, Johannes Kestler, Ido Kanter, Wolfgang Kinzel

Abstract: The linear response of synchronized time-delayed chaotic systems to small external perturbations, i.e., the phenomenon of chaos pass filter, is investigated for iterated maps. The distribution of distances, i.e., the deviations between two synchronized chaotic units due to external perturbations on the transfered signal, is used as a measure of the linear response. It is calculated numerically and… ▽ More The linear response of synchronized time-delayed chaotic systems to small external perturbations, i.e., the phenomenon of chaos pass filter, is investigated for iterated maps. The distribution of distances, i.e., the deviations between two synchronized chaotic units due to external perturbations on the transfered signal, is used as a measure of the linear response. It is calculated numerically and, for some special cases, analytically. Depending on the model parameters this distribution has power law tails in the region of synchronization leading to diverging moments of distances. This is a consequence of multiplicative and additive noise in the corresponding linear equations due to chaos and external perturbations. The linear response can also be quantified by the bit error rate of a transmitted binary message which perturbs the synchronized system. The bit error rate is given by an integral over the distribution of distances and is calculated analytically and numerically. It displays a complex nonmonotonic behavior in the region of synchronization. For special cases the distribution of distances has a fractal structure leading to a devil's staircase for the bit error rate as a function of coupling strength. The response to small harmonic perturbations shows resonances related to coupling and feedback delay times. A bi-directionally coupled chain of three units can completely filtered out the perturbation. Thus the second moment and the bit error rate become zero. △ Less

Submitted 29 January, 2013; originally announced January 2013.

Transient to Zero-Lag Synchronization in Excitable Networks

Authors: H. Brama, Y. Peleg, W. Kinzel, I. Kanter

Abstract: The scaling of transient times to zero-lag synchronization in networks composed of excitable units is shown to be governed by three features of the graph representing the network: the longest path between pairs of neurons (diameter), the largest loop (circumference) and the loop with the maximal average out degree. The upper bound of transient times can vary between O(1) and O(N2), where N is the… ▽ More The scaling of transient times to zero-lag synchronization in networks composed of excitable units is shown to be governed by three features of the graph representing the network: the longest path between pairs of neurons (diameter), the largest loop (circumference) and the loop with the maximal average out degree. The upper bound of transient times can vary between O(1) and O(N2), where N is the size of the network, and its scaling can be predicted in many scenarios from finite time accumulated information of the transient. Results challenge the assumption that functionality of neural networks might depend solely upon the synchronized repeated activation such as zero-lag synchronization. △ Less

Submitted 10 October, 2012; originally announced October 2012.

Comments: 5 pages, 3 figures, 1 table

Discontinuous Attractor Dimension at the Synchronization Transition of Time-Delayed Chaotic Systems

Authors: Steffen Zeeb, Thomas Dahms, Valentin Flunkert, Eckehard Schöll, Ido Kanter, Wolfgang Kinzel

Abstract: The attractor dimension at the transition to complete synchronization in a network of chaotic units with time-delayed couplings is investigated. In particular, we determine the Kaplan-Yorke dimension from the spectrum of Lyapunov exponents for iterated maps and for two coupled semiconductor lasers. We argue that the Kaplan-Yorke dimension must be discontinuous at the transition and compare it to t… ▽ More The attractor dimension at the transition to complete synchronization in a network of chaotic units with time-delayed couplings is investigated. In particular, we determine the Kaplan-Yorke dimension from the spectrum of Lyapunov exponents for iterated maps and for two coupled semiconductor lasers. We argue that the Kaplan-Yorke dimension must be discontinuous at the transition and compare it to the correlation dimension. For a system of Bernoulli maps we indeed find a jump in the correlation dimension. The magnitude of the discontinuity in the Kaplan-Yorke dimension is calculated for networks of Bernoulli units as a function of the network size. Furthermore the scaling of the Kaplan-Yorke dimension as well as of the Kolmogorov entropy with system size and time delay is investigated. △ Less

Submitted 10 October, 2012; originally announced October 2012.

Strong and Weak Chaos in Networks of Semiconductor Lasers with Time-delayed Couplings

Authors: Sven Heiligenthal, Thomas Jüngling, Otti D'Huys, Diana A. Arroyo-Almanza, Miguel C. Soriano, Ingo Fischer, Ido Kanter, Wolfgang Kinzel

Abstract: Nonlinear networks with time-delayed couplings may show strong and weak chaos, depending on the scaling of their Lyapunov exponent with the delay time. We study strong and weak chaos for semiconductor lasers, either with time-delayed self-feedback or for small networks. We examine the dependence on the pump current and consider the question whether strong and weak chaos can be identified from the… ▽ More Nonlinear networks with time-delayed couplings may show strong and weak chaos, depending on the scaling of their Lyapunov exponent with the delay time. We study strong and weak chaos for semiconductor lasers, either with time-delayed self-feedback or for small networks. We examine the dependence on the pump current and consider the question whether strong and weak chaos can be identified from the shape of the intensity trace, the auto-correlations and the external cavity modes. The concept of the sub-Lyapunov exponent $λらむだ_0$ is generalized to the case of two time-scale separated delays in the system. We give the first experimental evidence of strong and weak chaos in a network of lasers which supports the sequence 'weak to strong to weak chaos' upon monotonically increasing the coupling strength. Finally, we discuss strong and weak chaos for networks with several distinct sub-Lyapunov exponents and comment on the dependence of the sub-Lyapunov exponent on the number of a laser's inputs in a network. △ Less

Submitted 5 October, 2012; originally announced October 2012.

Synchronization with mismatched synaptic delays: A unique role of elastic neuronal latency

Authors: Roni Vardi, Reut Timor, Shimon Marom, Moshe Abeles, Ido Kanter

Abstract: We show that the unavoidable increase in neuronal response latency to ongoing stimulation serves as a nonuniform gradual stretching of neuronal circuit delay loops and emerges as an essential mechanism in the formation of various types of neuronal timers. Synchronization emerges as a transient phenomenon without predefined precise matched synaptic delays. These findings are described in an experim… ▽ More We show that the unavoidable increase in neuronal response latency to ongoing stimulation serves as a nonuniform gradual stretching of neuronal circuit delay loops and emerges as an essential mechanism in the formation of various types of neuronal timers. Synchronization emerges as a transient phenomenon without predefined precise matched synaptic delays. These findings are described in an experimental procedure where conditioned stimulations were enforced on a circuit of neurons embedded within a large-scale network of cortical cells in-vitro, and are corroborated by neuronal simulations. They evidence a new cortical timescale based on tens of microseconds stretching of neuronal circuit delay loops per spike, and with realistic delays of a few milliseconds, synchronization emerges for a finite fraction of neuronal circuit delays. △ Less

Submitted 12 September, 2012; originally announced September 2012.

Comments: 12 pages, 4 figures, 13 pages of Supplementary material

Journal ref: EPL, 100, 48003 (2012)

Phase transition in crowd synchrony of delay-coupled multilayer laser networks

Authors: Elad Cohen, Michael Rosenbluh, Ido Kanter

Abstract: An analogy between crowd synchrony and multi-layer neural network architectures is proposed. It indicates that many non-identical dynamical elements (oscillators) communicating indirectly via a few mediators (hubs) can synchronize when the number of delayed couplings to the hubs or the strength of the couplings is large enough. This phenomenon is modeled using a system of semiconductor lasers opti… ▽ More An analogy between crowd synchrony and multi-layer neural network architectures is proposed. It indicates that many non-identical dynamical elements (oscillators) communicating indirectly via a few mediators (hubs) can synchronize when the number of delayed couplings to the hubs or the strength of the couplings is large enough. This phenomenon is modeled using a system of semiconductor lasers optically delay-coupled in either a fully connected or a diluted manner to a fixed number of non-identical central hub lasers. A universal phase transition to crowd synchrony with hysteresis is observed, where the time to achieve synchronization diverges near the critical coupling independent of the number of hubs. △ Less

Submitted 17 April, 2012; originally announced April 2012.

Comments: 14 pages, 5 figures

Random Symmetry Breaking and Freezing in Chaotic Networks

Authors: Y. Peleg, W. Kinzel, I. Kanter

Abstract: Parameter space of a driven damped oscillator in a double well potential presents either a chaotic trajectory with sign oscillating amplitude or a non-chaotic trajectory with a fixed sign amplitude. A network of such delay coupled damped oscillators is shown to present chaotic dynamics while the amplitude sign of each damped oscillator is randomly frozen. This phenomenon of random broken global sy… ▽ More Parameter space of a driven damped oscillator in a double well potential presents either a chaotic trajectory with sign oscillating amplitude or a non-chaotic trajectory with a fixed sign amplitude. A network of such delay coupled damped oscillators is shown to present chaotic dynamics while the amplitude sign of each damped oscillator is randomly frozen. This phenomenon of random broken global symmetry of the network simultaneously with random freezing of each degree of freedom is accompanied by the existence of exponentially many randomly frozen chaotic attractors with the ize of the network. Results are exemplified by a network of modified Duffing oscillators with infinite ange pseudo-inverse delayed interactions. △ Less

Submitted 2 April, 2012; originally announced April 2012.

Comments: 5 pages, 4 figures

Synthetic reverberating activity patterns embedded in networks of cortical neurons

Authors: Roni Vardi, Avner Wallach, Evi Kopelowitz, Moshe Abeles, Shimon Marom, Ido Kanter

Abstract: Synthetic reverberating activity patterns are experimentally generated by stimulation of a subset of neurons embedded in a spontaneously active network of cortical cells in-vitro. The neurons are artificially connected by means of conditional stimulation matrix, forming a synthetic local circuit with a predefined programmable connectivity and time-delays. Possible uses of this experimental design… ▽ More Synthetic reverberating activity patterns are experimentally generated by stimulation of a subset of neurons embedded in a spontaneously active network of cortical cells in-vitro. The neurons are artificially connected by means of conditional stimulation matrix, forming a synthetic local circuit with a predefined programmable connectivity and time-delays. Possible uses of this experimental design are demonstrated, analyzing the sensitivity of these deterministic activity patterns to transmission delays and to the nature of ongoing network dynamics. △ Less

Submitted 26 March, 2012; v1 submitted 1 January, 2012; originally announced January 2012.

Comments: 8 pages, 5 figures

Journal ref: EPL (Europhysics Letters) 97, 66002 (2012)

Controlling synchronization in large laser networks using number theory

Authors: Micha Nixon, Moti Fridman, Eitan Ronen, Asher A. Friesem, Nir Davidson, Ido Kanter

Abstract: Synchronization in networks with delayed coupling are ubiquitous in nature and play a key role in almost all fields of science including physics, biology, ecology, climatology and sociology. In general, the published works on network synchronization are based on data analysis and simulations, with little experimental verification. Here we develop and experimentally demonstrate various multi-cluste… ▽ More Synchronization in networks with delayed coupling are ubiquitous in nature and play a key role in almost all fields of science including physics, biology, ecology, climatology and sociology. In general, the published works on network synchronization are based on data analysis and simulations, with little experimental verification. Here we develop and experimentally demonstrate various multi-cluster phase synchronization scenarios within coupled laser networks. Synchronization is controlled by the network connectivity in accordance to number theory, whereby the number of synchronized clusters equals the greatest common divisor of network loops. This dependence enables remote switching mechanisms to control the optical phase coherence among distant lasers by local network connectivity adjustments. Our results serve as a benchmark for a broad range of coupled oscillators in science and technology, and offer feasible routes to achieve multi-user secure protocols in communication networks and parallel distribution of versatile complex combinatorial tasks in optical computers. △ Less

Submitted 18 December, 2011; originally announced December 2011.

Comments: 13 pages, 4 figure and supplementary information

Time-delayed coupled chaotic semiconductor lasers show new types of synchronization: Experiment and theory

Authors: Y. Aviad, I. Reidler, M. Zigzag, M. Rosenbluh, I. Kanter

Abstract: Topologies of two, three and four time-delay-coupled chaotic semiconductor lasers are experimentally and theoretically found to show new types of synchronization. Shifted zero-lag synchronization is observed for two lasers separated by long distances even when their self-feedback delays are not equal. Shifted sub-lattice synchronization is observed for quadrilateral geometries while the equilatera… ▽ More Topologies of two, three and four time-delay-coupled chaotic semiconductor lasers are experimentally and theoretically found to show new types of synchronization. Shifted zero-lag synchronization is observed for two lasers separated by long distances even when their self-feedback delays are not equal. Shifted sub-lattice synchronization is observed for quadrilateral geometries while the equilateral triangle is zero-lag synchronized. Generalized zero-lag synchronization, without the limitation of precisely matched delays, opens possibilities for advanced multi-user communication protocols. △ Less

Submitted 28 November, 2011; originally announced November 2011.

Comments: 4 pages, 4 figures

Coexistence of exponentially many chaotic spin-glass attractors

Authors: Y. Peleg, M. zigzag, W. Kinzel, I. Kanter

Abstract: A chaotic network of size $N$ with delayed interactions which resembles a pseudo-inverse associative memory neural network is investigated. For a load $αあるふぁ=P/N<1$, where $P$ stands for the number of stored patterns, the chaotic network functions as an associative memory of 2P attractors with macroscopic basin of attractions which decrease with $αあるふぁ$. At finite $αあるふぁ$, a chaotic spin glass phase exists, w… ▽ More A chaotic network of size $N$ with delayed interactions which resembles a pseudo-inverse associative memory neural network is investigated. For a load $αあるふぁ=P/N<1$, where $P$ stands for the number of stored patterns, the chaotic network functions as an associative memory of 2P attractors with macroscopic basin of attractions which decrease with $αあるふぁ$. At finite $αあるふぁ$, a chaotic spin glass phase exists, where the number of distinct chaotic attractors scales exponentially with $N$. Each attractor is characterized by a coexistence of chaotic behavior and freezing of each one of the $N$ chaotic units or freezing with respect to the $P$ patterns. Results are supported by large scale simulations of networks composed of Bernoulli map units and Mackey-Glass time delay differential equations. △ Less

Submitted 9 November, 2011; originally announced November 2011.

Comments: 6 pages and 5 figures

Strong and weak chaos in nonlinear networks with time-delayed couplings

Authors: Sven Heiligenthal, Thomas Dahms, Serhiy Yanchuk, Thomas Jüngling, Valentin Flunkert, Ido Kanter, Eckehard Schöll, Wolfgang Kinzel

Abstract: We study chaotic synchronization in networks with time-delayed coupling. We introduce the notion of strong and weak chaos, distinguished by the scaling properties of the maximum Lyapunov exponent within the synchronization manifold for large delay times, and relate this to the condition for stable or unstable chaotic synchronization, respectively. In simulations of laser models and experiments wit… ▽ More We study chaotic synchronization in networks with time-delayed coupling. We introduce the notion of strong and weak chaos, distinguished by the scaling properties of the maximum Lyapunov exponent within the synchronization manifold for large delay times, and relate this to the condition for stable or unstable chaotic synchronization, respectively. In simulations of laser models and experiments with electronic circuits, we identify transitions from weak to strong and back to weak chaos upon monotonically increasing the coupling strength. △ Less

Submitted 5 September, 2011; originally announced September 2011.

Journal ref: Phys. Rev. Lett. 107, 234102 (2011)

Nonlocal mechanism for cluster synchronization in neural circuits

Authors: I. Kanter, E. Kopelowitz, R. Vardi, M. Zigzag, W. Kinzel, M. Abeles, D. Cohen

Abstract: The interplay between the topology of cortical circuits and synchronized activity modes in distinct cortical areas is a key enigma in neuroscience. We present a new nonlocal mechanism governing the periodic activity mode: the greatest common divisor (GCD) of network loops. For a stimulus to one node, the network splits into GCD-clusters in which cluster neurons are in zero-lag synchronization. For… ▽ More The interplay between the topology of cortical circuits and synchronized activity modes in distinct cortical areas is a key enigma in neuroscience. We present a new nonlocal mechanism governing the periodic activity mode: the greatest common divisor (GCD) of network loops. For a stimulus to one node, the network splits into GCD-clusters in which cluster neurons are in zero-lag synchronization. For complex external stimuli, the number of clusters can be any common divisor. The synchronized mode and the transients to synchronization pinpoint the type of external stimuli. The findings, supported by an information mixing argument and simulations of Hodgkin Huxley population dynamic networks with unidirectional connectivity and synaptic noise, call for reexamining sources of correlated activity in cortex and shorter information processing time scales. △ Less

Submitted 18 March, 2011; originally announced March 2011.

Comments: 8 pges, 6 figures

Journal ref: EPL 93, 66001 (2011)

Synchronization of chaotic networks with time-delayed couplings: An analytic study

Authors: A. Englert, S. Heiligenthal, W. Kinzel, I. Kanter

Abstract: Networks of nonlinear units with time-delayed couplings can synchronize to a common chaotic trajectory. Although the delay time may be very large, the units can synchronize completely without time shift. For networks of coupled Bernoulli maps, analytic results are derived for the stability of the chaotic synchronization manifold. For a single delay time, chaos synchronization is related to the spe… ▽ More Networks of nonlinear units with time-delayed couplings can synchronize to a common chaotic trajectory. Although the delay time may be very large, the units can synchronize completely without time shift. For networks of coupled Bernoulli maps, analytic results are derived for the stability of the chaotic synchronization manifold. For a single delay time, chaos synchronization is related to the spectral gap of the coupling matrix. For networks with multiple delay times, analytic results are obtained from the theory of polynomials. Finally, the analytic results are compared with networks of iterated tent maps and Lang-Kobayashi equations which imitate the behaviour of networks of semiconductor lasers. △ Less

Submitted 20 January, 2011; originally announced January 2011.

Journal ref: Phys. Rev. E 83, 046222 (2011)

Synchronization of unidirectional time delay chaotic networks and the greatest common divisor

Authors: I. Kanter, M. Zigzag, A. Englert, F. Geissler, W. Kinzel

Abstract: We present the interplay between synchronization of unidirectional coupled chaotic nodes with heterogeneous delays and the greatest common divisor (GCD) of loops composing the oriented graph. In the weak chaos region and for GCD=1 the network is in chaotic zero-lag synchronization, whereas for GCD=m>1 synchronization of m-sublattices emerges. Complete synchronization can be achieved when all chaot… ▽ More We present the interplay between synchronization of unidirectional coupled chaotic nodes with heterogeneous delays and the greatest common divisor (GCD) of loops composing the oriented graph. In the weak chaos region and for GCD=1 the network is in chaotic zero-lag synchronization, whereas for GCD=m>1 synchronization of m-sublattices emerges. Complete synchronization can be achieved when all chaotic nodes are influenced by an identical set of delays and in particular for the limiting case of homogeneous delays. Results are supported by simulations of chaotic systems, self-consistent and mixing arguments, as well as analytical solutions of Bernoulli maps. △ Less

Submitted 28 March, 2011; v1 submitted 5 December, 2010; originally announced December 2010.

Comments: 7 pages, 5 figures

Journal ref: EPL 93, 60003 (2011)

An efficient CDMA decoder for correlated information sources

Authors: Hadar Efraim, Nadav Yacov, Ori Shental, Ido Kanter

Abstract: We consider the detection of correlated information sources in the ubiquitous Code-Division Multiple-Access (CDMA) scheme. We propose a message-passing based scheme for detecting correlated sources directly, with no need for source coding. The detection is done simultaneously over a block of transmitted binary symbols (word). Simulation results are provided demonstrating a substantial improvemen… ▽ More We consider the detection of correlated information sources in the ubiquitous Code-Division Multiple-Access (CDMA) scheme. We propose a message-passing based scheme for detecting correlated sources directly, with no need for source coding. The detection is done simultaneously over a block of transmitted binary symbols (word). Simulation results are provided demonstrating a substantial improvement in bit-error-rate in comparison with the unmodified detector and the alternative of source compression. The robustness of the error-performance improvement is shown under practical model settings, including wrong estimation of the generating Markov transition matrix and finite-length spreading codes. △ Less

Submitted 28 January, 2010; originally announced January 2010.

Comments: 11 pages

Journal ref: J. Stat. Mech. (2009) P07039

Mutual information via thermodynamics: Three different approaches

Authors: Yitzhak Peleg, Hadar Efraim, Ori Shental, Ido Kanter

Abstract: Three different approaches to derive mutual information via thermodynamics are presented where the temperature-dependent energy is given by: (a) $βべーた\mathcal{E} = -\ln[P(X,Y)]$, (b) $βべーた\mathcal{E} =-\ln[P(Y|X)]$ or (c) $βべーた\mathcal{E} =-\ln[P(X|Y)]$. All approaches require the extension of the traditional physical framework and the modification of the 2nd law of thermodynamics. A realization of a phy… ▽ More Three different approaches to derive mutual information via thermodynamics are presented where the temperature-dependent energy is given by: (a) $βべーた\mathcal{E} = -\ln[P(X,Y)]$, (b) $βべーた\mathcal{E} =-\ln[P(Y|X)]$ or (c) $βべーた\mathcal{E} =-\ln[P(X|Y)]$. All approaches require the extension of the traditional physical framework and the modification of the 2nd law of thermodynamics. A realization of a physical system with an effective temperature-dependent Hamiltonian is discussed followed by a suggestion of a physical information-heat engine. △ Less

Submitted 19 November, 2009; originally announced November 2009.

Comments: 11 pages, 4 figures

Journal ref: J. Stat. Mech. (2010) P01014

Pulses of chaos synchronization in coupled map chains with delayed transmission

Authors: Bernhard Schmitzer, Wolfgang Kinzel, Ido Kanter

Abstract: Pulses of synchronization in chaotic coupled map lattices are discussed in the context of transmission of information. Synchronization and desynchronization propagate along the chain with different velocities which are calculated analytically from the spectrum of convective Lyapunov exponents. Since the front of synchronization travels slower than the front of desynchronization, the maximal possib… ▽ More Pulses of synchronization in chaotic coupled map lattices are discussed in the context of transmission of information. Synchronization and desynchronization propagate along the chain with different velocities which are calculated analytically from the spectrum of convective Lyapunov exponents. Since the front of synchronization travels slower than the front of desynchronization, the maximal possible chain length for which information can be transmitted by modulating the first unit of the chain is bounded. △ Less

Submitted 11 March, 2010; v1 submitted 30 June, 2009; originally announced June 2009.

Comments: 4 pages, 6 figures, updated version as published in PRE

Journal ref: Phys. Rev. E 80, 047203 (2009)

Emergence of zero-lag synchronization in generic mutually coupled chaotic systems

Authors: Meital Zigzag, Maria Butkovski, Anja Englert, Wolfgang Kinzel, Ido Kanter

Abstract: Zero-lag synchronization (ZLS) is achieved in a very restricted mutually coupled chaotic systems, where the delays of the self-coupling and the mutual coupling are identical or fulfil some restricted ratios. Using a set of multiple self-feedbacks we demonstrate both analytically and numerically that ZLS is achieved for a wide range of mutual delays. It indicates that ZLS can be achieved without… ▽ More Zero-lag synchronization (ZLS) is achieved in a very restricted mutually coupled chaotic systems, where the delays of the self-coupling and the mutual coupling are identical or fulfil some restricted ratios. Using a set of multiple self-feedbacks we demonstrate both analytically and numerically that ZLS is achieved for a wide range of mutual delays. It indicates that ZLS can be achieved without the knowledge of the mutual distance between the communicating partners and has an important implication in the possible use of ZLS in communications networks as well as in the understanding of the emergence of such synchronization in the neuronal activities. △ Less

Submitted 25 November, 2008; originally announced November 2008.

Carnot in the Information Age: Discrete Symmetric Channels

Authors: Ido Kanter, Ori Shental, Hadar Efraim, Nadav Yacov

Abstract: Modeling communication channels as thermal systems results in Hamiltonians which are an explicit function of the temperature. The first two authors have recently generalized the second thermodynamic law to encompass systems with temperature-dependent energy levels, $dQ=TdS+<d\mathcal{E}/dT>dT$, where {$<\cdot>$} denotes averaging over the Boltzmann distribution, recomputing the mutual informatio… ▽ More Modeling communication channels as thermal systems results in Hamiltonians which are an explicit function of the temperature. The first two authors have recently generalized the second thermodynamic law to encompass systems with temperature-dependent energy levels, $dQ=TdS+<d\mathcal{E}/dT>dT$, where {$<\cdot>$} denotes averaging over the Boltzmann distribution, recomputing the mutual information and other main properties of the popular Gaussian channel. Here the mutual information for the binary symmetric channel as well as for the discrete symmetric channel consisting of 4 input/output (I/O) symbols is explicitly calculated using the generalized second law of thermodynamics. For equiprobable I/O the mutual information of the examined channels has a very simple form, -$γがんまU(γがんま)|_0^βべーた$, where $U$ denotes the internal energy of the channel. We prove that this simple form of the mutual information governs the class of discrete memoryless symmetric communication channels with equiprobable I/O symbols. △ Less

Submitted 27 July, 2008; originally announced July 2008.

Chaos Pass Filter: Linear Response of Synchronized Chaotic Systems

Authors: Wolfgang Kinzel, Johannes Kestler, Ido Kanter

Abstract: The linear response of synchronized chaotic units with delayed couplings and feedback to small external perturbations is investigated in the context of communication with chaos synchronization. For iterated chaotic maps, the distribution of distances is calculated numerically and, for some special cases, analytically as well. Depending on model parameters, this distribution has power law tails l… ▽ More The linear response of synchronized chaotic units with delayed couplings and feedback to small external perturbations is investigated in the context of communication with chaos synchronization. For iterated chaotic maps, the distribution of distances is calculated numerically and, for some special cases, analytically as well. Depending on model parameters, this distribution has power law tails leading to diverging moments of distances in the region of synchronization. The corresponding linear equations have multiplicative and additive noise due to perturbations and chaos. The response to small harmonic perturbations shows resonances related to coupling and feedback delay times. For perturbation from a binary message the bit error rate is calculated. The bit error rate is not related to the transverse Lyapunov exponents, and it can be reduced when additional noise is added to the transmitted signal. For some special cases, the bit error rate as a function of coupling strength has the structure of a devil's staircase, related to an iterated function system. Finally, the security of communication is discussed by comparing uni- and bi-directional couplings. △ Less

Submitted 4 February, 2013; v1 submitted 26 June, 2008; originally announced June 2008.

Comments: Outdated by http://arxiv.org/abs/1301.6929

Shannon Meets Carnot: Generalized Second Thermodynamic Law

Authors: Ori Shental, Ido Kanter

Abstract: The classical thermodynamic laws fail to capture the behavior of systems with energy Hamiltonian which is an explicit function of the temperature. Such Hamiltonian arises, for example, in modeling information processing systems, like communication channels, as thermal systems. Here we generalize the second thermodynamic law to encompass systems with temperature-dependent energy levels,… ▽ More The classical thermodynamic laws fail to capture the behavior of systems with energy Hamiltonian which is an explicit function of the temperature. Such Hamiltonian arises, for example, in modeling information processing systems, like communication channels, as thermal systems. Here we generalize the second thermodynamic law to encompass systems with temperature-dependent energy levels, $dQ=TdS+<d\mathcal{E}/dT>dT$, where $<\cdot>$ denotes averaging over the Boltzmann distribution and reveal a new definition to the basic notion of temperature. This generalization enables to express, for instance, the mutual information of the Gaussian channel as a consequence of the fundamental laws of nature - the laws of thermodynamics. △ Less

Submitted 23 June, 2008; originally announced June 2008.

Shannon Meets Carnot: Mutual Information Via Thermodynamics

Authors: Ori Shental, Ido Kanter

Abstract: In this contribution, the Gaussian channel is represented as an equivalent thermal system allowing to express its input-output mutual information in terms of thermodynamic quantities. This thermodynamic description of the mutual information is based upon a generalization of the $2^{nd}$ thermodynamic law and provides an alternative proof to the Guo-Shamai-Verdú theorem, giving an intriguing conn… ▽ More In this contribution, the Gaussian channel is represented as an equivalent thermal system allowing to express its input-output mutual information in terms of thermodynamic quantities. This thermodynamic description of the mutual information is based upon a generalization of the $2^{nd}$ thermodynamic law and provides an alternative proof to the Guo-Shamai-Verdú theorem, giving an intriguing connection between this remarkable theorem and the most fundamental laws of nature - the laws of thermodynamics. △ Less

Submitted 19 June, 2008; originally announced June 2008.