Showing 1–50 of 66 results for author: Aggarwal, S

Semantic Communication in Multi-team Dynamic Games: A Mean Field Perspective

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar

Abstract: Coordinating communication and control is a key component in the stability and performance of networked multi-agent systems. While single user networked control systems have gained a lot of attention within this domain, in this work, we address the more challenging problem of large population multi-team dynamic games. In particular, each team constitutes two decision makers (namely, the sensor and… ▽ More Coordinating communication and control is a key component in the stability and performance of networked multi-agent systems. While single user networked control systems have gained a lot of attention within this domain, in this work, we address the more challenging problem of large population multi-team dynamic games. In particular, each team constitutes two decision makers (namely, the sensor and the controller) who coordinate over a shared network to control a dynamically evolving state of interest under costs on both actuation and sensing/communication. Due to the shared nature of the wireless channel, the overall cost of each team depends on other teams' policies, thereby leading to a noncooperative game setup. Due to the presence of a large number of teams, we compute approximate decentralized Nash equilibrium policies for each team using the paradigm of (extended) mean-field games, which is governed by (1) the mean traffic flowing over the channel, and (2) the value of information at the sensor, which highlights the semantic nature of the ensuing communication. In the process, we compute optimal controller policies and approximately optimal sensor policies for each representative team of the mean-field system to alleviate the problem of general non-contractivity of the mean-field fixed point operator associated with the finite cardinality of the sensor action space. Consequently, we also prove the $εいぷしろん$--Nash property of the mean-field equilibrium solution which essentially characterizes how well the solution derived using mean-field analysis performs on the finite-team system. Finally, we provide extensive numerical simulations, which corroborate the theoretical findings and lead to additional insights on the properties of the results presented. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: Submitted to IEEE for possible publication

From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries

Authors: Hitesh Wadhwa, Rahul Seetharaman, Somyaa Aggarwal, Reshmi Ghosh, Samyadeep Basu, Soundararajan Srinivasan, Wenlong Zhao, Shreyas Chaudhari, Ehsan Aghazadeh

Abstract: Retrieval Augmented Generation (RAG) enriches the ability of language models to reason using external context to augment responses for a given user prompt. This approach has risen in popularity due to practical applications in various applications of language models in search, question/answering, and chat-bots. However, the exact nature of how this approach works isn't clearly understood. In this… ▽ More Retrieval Augmented Generation (RAG) enriches the ability of language models to reason using external context to augment responses for a given user prompt. This approach has risen in popularity due to practical applications in various applications of language models in search, question/answering, and chat-bots. However, the exact nature of how this approach works isn't clearly understood. In this paper, we mechanistically examine the RAG pipeline to highlight that language models take shortcut and have a strong bias towards utilizing only the context information to answer the question, while relying minimally on their parametric memory. We probe this mechanistic behavior in language models with: (i) Causal Mediation Analysis to show that the parametric memory is minimally utilized when answering a question and (ii) Attention Contributions and Knockouts to show that the last token residual stream do not get enriched from the subject token in the question, but gets enriched from other informative tokens in the context. We find this pronounced shortcut behaviour true across both LLaMa and Phi family of models. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Best Response Strategies for Asymmetric Sensing in Linear-Quadratic Differential Games

Authors: Shubham Aggarwal, Tamer Başar, Dipankar Maity

Abstract: In this paper, we revisit the two-player continuous-time infinite-horizon linear quadratic differential game problem, where one of the players can sample the state of the system only intermittently due to a sensing constraint while the other player can do so continuously. Under these asymmetric sensing limitations between the players, we analyze the optimal sensing and control strategies for the p… ▽ More In this paper, we revisit the two-player continuous-time infinite-horizon linear quadratic differential game problem, where one of the players can sample the state of the system only intermittently due to a sensing constraint while the other player can do so continuously. Under these asymmetric sensing limitations between the players, we analyze the optimal sensing and control strategies for the player at a disadvantage while the other player continues to play its security strategy. We derive an optimal sensor policy within the class of stationary randomized policies. Finally, using simulations, we show that the expected cost accrued by the first player approaches its security level as its sensing limitation is relaxed. △ Less

Submitted 8 June, 2024; originally announced June 2024.

Comments: Accepted to IEEE L-CSS

An Automated Validation Framework for Power Management and Data Retention Logic Kits of Standard Cell Library

Authors: Akshay Karkal Kamath, Bharath Kumar, Sunil Aggarwal, Subramanian Parameswaran, Parag Lonkar, Debi Prasanna, Somasunder Sreenath

Abstract: The development of a standard cell library involves characterization of a number of gate-level circuits at various cell-level abstractions. Verifying the behavior of these cells largely depends on the manual skills of the circuit designers. Especially challenging are the power management and data retention cells which must be checked thoroughly for voltage and power configurations in addition to t… ▽ More The development of a standard cell library involves characterization of a number of gate-level circuits at various cell-level abstractions. Verifying the behavior of these cells largely depends on the manual skills of the circuit designers. Especially challenging are the power management and data retention cells which must be checked thoroughly for voltage and power configurations in addition to their logic functionality. Also, when standard cells are extracted into various models, any inconsistencies in these models typically goes unchecked during library development. Thus, validating these cells exhaustively prior to customer delivery is highly advantageous to not only improve customer satisfaction but also to reduce design costs. We address this challenge by presenting a methodology to validate the power management and data retention cells that are used in the logical design flow of low-power chips. For a quick adoption by standard cell library design teams, the framework is fully automated and runs out-of-the-box. The proposed framework has been implemented and deployed within the Samsung Foundry ecosystem to enhance the overall quality of library design kit deliverables. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Comments: 33rd Design and Verification Conference and Exhibition United States (DVCon U.S. 2021)

Robot Safety Monitoring using Programmable Light Curtains

Authors: Karnik Ram, Shobhit Aggarwal, Robert Tamburo, Siddharth Ancha, Srinivasa Narasimhan

Abstract: As factories continue to evolve into collaborative spaces with multiple robots working together with human supervisors in the loop, ensuring safety for all actors involved becomes critical. Currently, laser-based light curtain sensors are widely used in factories for safety monitoring. While these conventional safety sensors meet high accuracy standards, they are difficult to reconfigure and can o… ▽ More As factories continue to evolve into collaborative spaces with multiple robots working together with human supervisors in the loop, ensuring safety for all actors involved becomes critical. Currently, laser-based light curtain sensors are widely used in factories for safety monitoring. While these conventional safety sensors meet high accuracy standards, they are difficult to reconfigure and can only monitor a fixed user-defined region of space. Furthermore, they are typically expensive. Instead, we leverage a controllable depth sensor, programmable light curtains (PLC), to develop an inexpensive and flexible real-time safety monitoring system for collaborative robot workspaces. Our system projects virtual dynamic safety envelopes that tightly envelop the moving robot at all times and detect any objects that intrude the envelope. Furthermore, we develop an instrumentation algorithm that optimally places (multiple) PLCs in a workspace to maximize the visibility coverage of robots. Our work enables fence-less human-robot collaboration, while scaling to monitor multiple robots with few sensors. We analyze our system in a real manufacturing testbed with four robot arms and demonstrate its capabilities as a fast, accurate, and inexpensive safety monitoring solution. △ Less

Submitted 4 April, 2024; originally announced April 2024.

Comments: Under review for IROS '24. Webpage http://cmu-mfi.github.io/plc-safety

A Mean Field Game Model for Timely Computation in Edge Computing Systems

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Sennur Ulukus, Tamer Başar

Abstract: We consider the problem of task offloading in multi-access edge computing (MEC) systems constituting $N$ devices assisted by an edge server (ES), where the devices can split task execution between a local processor and the ES. Since the local task execution and communication with the ES both consume power, each device must judiciously choose between the two. We model the problem as a large populat… ▽ More We consider the problem of task offloading in multi-access edge computing (MEC) systems constituting $N$ devices assisted by an edge server (ES), where the devices can split task execution between a local processor and the ES. Since the local task execution and communication with the ES both consume power, each device must judiciously choose between the two. We model the problem as a large population non-cooperative game among the $N$ devices. Since computation of an equilibrium in this scenario is difficult due to the presence of a large number of devices, we employ the mean-field game framework to reduce the finite-agent game problem to a generic user's multi-objective optimization problem, with a coupled consistency condition. By leveraging the novel age of information (AoI) metric, we invoke techniques from stochastic hybrid systems (SHS) theory and study the tradeoffs between increasing information freshness and reducing power consumption. In numerical simulations, we validate that a higher load at the ES may lead devices to upload their task to the ES less often. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Comments: Submitted to IEEE for possible publication

Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Games

Authors: Muhammad Aneeq uz Zaman, Shubham Aggarwal, Melih Bastopcu, Tamer Başar

Abstract: In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum $N$-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies. Moreover, it delineates sufficient conditions, contingent upon the adequacy of entropy regularization, for the uniqueness of the NE within the game. As Policy Optimizatio… ▽ More In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum $N$-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies. Moreover, it delineates sufficient conditions, contingent upon the adequacy of entropy regularization, for the uniqueness of the NE within the game. As Policy Optimization serves as a foundational approach for Reinforcement Learning (RL) techniques aimed at finding the NE, in this work we prove the linear convergence of a policy optimization algorithm which (subject to the adequacy of entropy regularization) is capable of provably attaining the NE. Furthermore, in scenarios where the entropy regularization proves insufficient, we present a $δでるた$-augmentation technique, which facilitates the achievement of an $εいぷしろん$-NE within the game. △ Less

Submitted 25 March, 2024; originally announced April 2024.

Computational Study on the Impact of Gasoline-Ethanol Blending on Autoignition and Soot/NOx Emissions under Gasoline Compression Ignition Conditions

Authors: Krishna C. Kalvakala, Harsimran Singh, Pinaki Pal, Jorge P. Gonzalez, Christopher P. Kolodziej, Suresh K. Aggarwal

Abstract: Computational fluid dynamics (CFD) simulations of a single-cylinder gasoline compression ignition engine are performed to investigate the impact of gasoline-ethanol blending on autoignition, nitrogen oxide (NOx), and soot emissions under low-load conditions. A four-component toluene primary reference fuel (TPRF) + ethanol (ETPRF) surrogate (with 10% ethanol by volume; E10) is employed to represent… ▽ More Computational fluid dynamics (CFD) simulations of a single-cylinder gasoline compression ignition engine are performed to investigate the impact of gasoline-ethanol blending on autoignition, nitrogen oxide (NOx), and soot emissions under low-load conditions. A four-component toluene primary reference fuel (TPRF) + ethanol (ETPRF) surrogate (with 10% ethanol by volume; E10) is employed to represent the test gasoline (RD5-87). A 3D engine CFD model employing finite-rate chemistry with a skeletal kinetic mechanism, adaptive mesh refinement (AMR), and hybrid method of moments (HMOM) is adopted to capture in-cylinder combustion and soot/NOx emissions. The engine CFD model is validated against experimental data for three gasoline-ethanol blends: E10, E30 and E100, with varying ethanol content by volume. Model validation is carried out for multiple start-of-injection (SOI) timings (-21, -27, -36, and -45 crank angle degrees after top-dead-center (aTDC)) with respect to in-cylinder pressure, heat release rate, combustion phasing, NOx and soot emissions. For late injection timings (-21 and -27oaTDC), E30 yields higher soot than E10; while the trend reverses for early injection cases (-36 and -45oaTDC). E100 yields the lowest amount of soot among all fuels irrespective of SOI timing. Further, E10 shows a non-monotonic trend in soot emissions with SOI timing: SOI-36>SOI-45>SOI-21>SOI-27, while soot emissions from E30 exhibit monotonic decrease with advancing SOI timing. NOx emissions from various fuels follow a trend of E10>E30>E100. NOx emissions increase as SOI timing is advanced for all fuels, with an anomaly for E10 and E100 where NOx decreases when SOI is advanced beyond -36oaTDC. Detailed analysis of the numerical results is performed to investigate the emission trends and elucidate the impact of chemical composition and physical properties on autoignition and emissions characteristics. △ Less

Submitted 15 March, 2024; originally announced March 2024.

Emergent self-adaptation in an integrated photonic neural network for backpropagation-free learning

Authors: Alessio Lugnan, Samarth Aggarwal, Frank Brückerhoff-Plückelmann, C. David Wright, Wolfram H. P. Pernice, Harish Bhaskaran, Peter Bienstman

Abstract: Plastic self-adaptation, nonlinear recurrent dynamics and multi-scale memory are desired features in hardware implementations of neural networks, because they enable them to learn, adapt and process information similarly to the way biological brains do. In this work, we experimentally demonstrate these properties occurring in arrays of photonic neurons. Importantly, this is realised autonomously i… ▽ More Plastic self-adaptation, nonlinear recurrent dynamics and multi-scale memory are desired features in hardware implementations of neural networks, because they enable them to learn, adapt and process information similarly to the way biological brains do. In this work, we experimentally demonstrate these properties occurring in arrays of photonic neurons. Importantly, this is realised autonomously in an emergent fashion, without the need for an external controller setting weights and without explicit feedback of a global reward signal. Using a hierarchy of such arrays coupled to a backpropagation-free training algorithm based on simple logistic regression, we are able to achieve a performance of 98.2% on the MNIST task, a popular benchmark task looking at classification of written digits. The plastic nodes consist of silicon photonics microring resonators covered by a patch of phase-change material that implements nonvolatile memory. The system is compact, robust, and straightforward to scale up through the use of multiple wavelengths. Moreover, it constitutes a unique platform to test and efficiently implement biologically plausible learning schemes at a high processing speed. △ Less

Submitted 6 December, 2023; originally announced December 2023.

CRISP: Hybrid Structured Sparsity for Class-aware Model Pruning

Authors: Shivam Aggarwal, Kuluhan Binici, Tulika Mitra

Abstract: Machine learning pipelines for classification tasks often train a universal model to achieve accuracy across a broad range of classes. However, a typical user encounters only a limited selection of classes regularly. This disparity provides an opportunity to enhance computational efficiency by tailoring models to focus on user-specific classes. Existing works rely on unstructured pruning, which in… ▽ More Machine learning pipelines for classification tasks often train a universal model to achieve accuracy across a broad range of classes. However, a typical user encounters only a limited selection of classes regularly. This disparity provides an opportunity to enhance computational efficiency by tailoring models to focus on user-specific classes. Existing works rely on unstructured pruning, which introduces randomly distributed non-zero values in the model, making it unsuitable for hardware acceleration. Alternatively, some approaches employ structured pruning, such as channel pruning, but these tend to provide only minimal compression and may lead to reduced model accuracy. In this work, we propose CRISP, a novel pruning framework leveraging a hybrid structured sparsity pattern that combines both fine-grained N:M structured sparsity and coarse-grained block sparsity. Our pruning strategy is guided by a gradient-based class-aware saliency score, allowing us to retain weights crucial for user-specific classes. CRISP achieves high accuracy with minimal memory consumption for popular models like ResNet-50, VGG-16, and MobileNetV2 on ImageNet and CIFAR-100 datasets. Moreover, CRISP delivers up to 14$\times$ reduction in latency and energy consumption compared to existing pruning methods while maintaining comparable accuracy. Our code is available at https://github.com/shivmgg/CRISP/. △ Less

Submitted 18 March, 2024; v1 submitted 23 November, 2023; originally announced November 2023.

Comments: 6 pages, accepted in Design, Automation & Test in Europe Conference & Exhibition (DATE) 2024

Shedding the Bits: Pushing the Boundaries of Quantization with Minifloats on FPGAs

Authors: Shivam Aggarwal, Hans Jakob Damsgaard, Alessandro Pappalardo, Giuseppe Franco, Thomas B. Preußer, Michaela Blott, Tulika Mitra

Abstract: Post-training quantization (PTQ) is a powerful technique for model compression, reducing the numerical precision in neural networks without additional training overhead. Recent works have investigated adopting 8-bit floating-point formats(FP8) in the context of PTQ for model inference. However, floating-point formats smaller than 8 bits and their relative comparison in terms of accuracy-hardware c… ▽ More Post-training quantization (PTQ) is a powerful technique for model compression, reducing the numerical precision in neural networks without additional training overhead. Recent works have investigated adopting 8-bit floating-point formats(FP8) in the context of PTQ for model inference. However, floating-point formats smaller than 8 bits and their relative comparison in terms of accuracy-hardware cost with integers remains unexplored on FPGAs. In this work, we present minifloats, which are reduced-precision floating-point formats capable of further reducing the memory footprint, latency, and energy cost of a model while approaching full-precision model accuracy. We implement a custom FPGA-based multiply-accumulate operator library and explore the vast design space, comparing minifloat and integer representations across 3 to 8 bits for both weights and activations. We also examine the applicability of various integerbased quantization techniques to minifloats. Our experiments show that minifloats offer a promising alternative for emerging workloads such as vision transformers. △ Less

Submitted 5 July, 2024; v1 submitted 21 November, 2023; originally announced November 2023.

Comments: Accepted in FPL (International Conference on Field-Programmable Logic and Applications) 2024 conference. Revised with updated results

Multi-Purpose NLP Chatbot : Design, Methodology & Conclusion

Authors: Shivom Aggarwal, Shourya Mehra, Pritha Mitra

Abstract: With a major focus on its history, difficulties, and promise, this research paper provides a thorough analysis of the chatbot technology environment as it exists today. It provides a very flexible chatbot system that makes use of reinforcement learning strategies to improve user interactions and conversational experiences. Additionally, this system makes use of sentiment analysis and natural langu… ▽ More With a major focus on its history, difficulties, and promise, this research paper provides a thorough analysis of the chatbot technology environment as it exists today. It provides a very flexible chatbot system that makes use of reinforcement learning strategies to improve user interactions and conversational experiences. Additionally, this system makes use of sentiment analysis and natural language processing to determine user moods. The chatbot is a valuable tool across many fields thanks to its amazing characteristics, which include voice-to-voice conversation, multilingual support [12], advising skills, offline functioning, and quick help features. The complexity of chatbot technology development is also explored in this study, along with the causes that have propelled these developments and their far-reaching effects on a range of sectors. According to the study, three crucial elements are crucial: 1) Even without explicit profile information, the chatbot system is built to adeptly understand unique consumer preferences and fluctuating satisfaction levels. With the use of this capacity, user interactions are made to meet their wants and preferences. 2) Using a complex method that interlaces Multiview voice chat information, the chatbot may precisely simulate users' actual experiences. This aids in developing more genuine and interesting discussions. 3) The study presents an original method for improving the black-box deep learning models' capacity for prediction. This improvement is made possible by introducing dynamic satisfaction measurements that are theory-driven, which leads to more precise forecasts of consumer reaction. △ Less

Submitted 13 October, 2023; originally announced October 2023.

Comments: Multilingual , Voice Conversion , Emotion Recognition , Offline Service , Financial Advisor , Product Preference , Customer Reaction Prediction

Severe flooding and cause-specific hospitalization in the United States

Authors: Sarika Aggarwal, Jie K. Hu, Jonathan A. Sullivan, Robbie M. Parks, Rachel C. Nethery

Abstract: Flooding is one of the most disruptive and costliest climate-related disasters and presents an escalating threat to population health due to climate change and urbanization patterns. Previous studies have investigated the consequences of flood exposures on only a handful of health outcomes and focus on a single flood event or affected region. To address this gap, we conducted a nationwide, multi-d… ▽ More Flooding is one of the most disruptive and costliest climate-related disasters and presents an escalating threat to population health due to climate change and urbanization patterns. Previous studies have investigated the consequences of flood exposures on only a handful of health outcomes and focus on a single flood event or affected region. To address this gap, we conducted a nationwide, multi-decade analysis of the impacts of severe floods on a wide range of health outcomes in the United States by linking a novel satellite-based high-resolution flood exposure database with Medicare cause-specific hospitalization records over the period 2000- 2016. Using a self-matched study design with a distributed lag model, we examined how cause-specific hospitalization rates deviate from expected rates during and up to four weeks after severe flood exposure. Our results revealed that risk of hospitalization was consistently elevated during and for at least four weeks following severe flood exposure for nervous system diseases (3.5 %; 95 % confidence interval [CI]: 0.6 %, 6.4 %), skin and subcutaneous tissue diseases (3.4 %; 95 % CI: 0.3 %, 6.7 %), and injury and poisoning (1.5 %; 95 % CI: -0.07 %, 3.2 %). Increases in hospitalization rate for these causes, musculoskeletal system diseases, and mental health-related impacts varied based on proportion of Black residents in each ZIP Code. Our findings demonstrate the need for targeted preparedness strategies for hospital personnel before, during, and after severe flooding. △ Less

Submitted 22 September, 2023; originally announced September 2023.

Neuromorphic Hebbian learning with magnetic tunnel junction synapses

Authors: Peng Zhou, Alexander J. Edwards, Frederick B. Mancoff, Sanjeev Aggarwal, Stephen K. Heinrich-Barna, Joseph S. Friedman

Abstract: Neuromorphic computing aims to mimic both the function and structure of biological neural networks to provide artificial intelligence with extreme efficiency. Conventional approaches store synaptic weights in non-volatile memory devices with analog resistance states, permitting in-memory computation of neural network operations while avoiding the costs associated with transferring synaptic weights… ▽ More Neuromorphic computing aims to mimic both the function and structure of biological neural networks to provide artificial intelligence with extreme efficiency. Conventional approaches store synaptic weights in non-volatile memory devices with analog resistance states, permitting in-memory computation of neural network operations while avoiding the costs associated with transferring synaptic weights from a memory array. However, the use of analog resistance states for storing weights in neuromorphic systems is impeded by stochastic writing, weights drifting over time through stochastic processes, and limited endurance that reduces the precision of synapse weights. Here we propose and experimentally demonstrate neuromorphic networks that provide high-accuracy inference thanks to the binary resistance states of magnetic tunnel junctions (MTJs), while leveraging the analog nature of their stochastic spin-transfer torque (STT) switching for unsupervised Hebbian learning. We performed the first experimental demonstration of a neuromorphic network directly implemented with MTJ synapses, for both inference and spike-timing-dependent plasticity learning. We also demonstrated through simulation that the proposed system for unsupervised Hebbian learning with stochastic STT-MTJ synapses can achieve competitive accuracies for MNIST handwritten digit recognition. By appropriately applying neuromorphic principles through hardware-aware design, the proposed STT-MTJ neuromorphic learning networks provide a pathway toward artificial intelligence hardware that learns autonomously with extreme efficiency. △ Less

Submitted 21 August, 2023; originally announced August 2023.

SecureTrack- A contact tracing IoT platform for monitoring infectious diseases

Authors: Shobhit Aggarwal, Arnab Purkayastha

Abstract: The COVID-19 pandemic has highlighted the need for innovative solutions to monitor and control the spread of infectious diseases. With the potential for future pandemics and the risk of outbreaks particularly in academic institutions, there is a pressing need for effective approaches to monitor and manage such diseases. Contact tracing using Global Positioning Systems (GPS) has been found to be th… ▽ More The COVID-19 pandemic has highlighted the need for innovative solutions to monitor and control the spread of infectious diseases. With the potential for future pandemics and the risk of outbreaks particularly in academic institutions, there is a pressing need for effective approaches to monitor and manage such diseases. Contact tracing using Global Positioning Systems (GPS) has been found to be the most prevalent method to detect and tackle the extent of outbreaks during the pandemic. However, these services suffer from the inherent problems of infringement of data privacy that creates hindrance in adoption of the technology. Non-cellular wireless technologies on the other hand are well-suited to provide secure contact tracing methods. Such approaches integrated with the Internet of Things (IoT) have a great potential to aid in the fight against any type of infectious diseases. In response, we present a unique approach that utilizes an IoT based generic framework to identify individuals who may have been exposed to the virus, using contact tracing methods, without compromising the privacy aspect. We develop the architecture of our platform, including both the frontend and backend components, and demonstrate its effectiveness in identifying potential COVID-19 exposures (as a test case) through a proof-of-concept implementation. We also implement and verify a prototype of the device. Our framework is easily deployable and can be scaled up as needed with the existing infrastructure. △ Less

Submitted 18 July, 2023; originally announced July 2023.

Comments: 22 Pages, 8 figures, To be published in "The Global Interdisciplinary Green Cities Conference 2023 Business, Engineering, Art, Architecture, Design, Political Science, International Relations, Applied Science & Technology. "

Theoretical proposal for the experimental realization of realignment operation

Authors: Shruti Aggarwal, Satyabrata Adhikari

Abstract: Realignment operation has a significant role in detecting bound as well as free entanglement. Just like partial transposition, it is also based on permutations of the matrix elements. However, the physical implementation of realignment operation is not known yet. In this letter, we address the problem of experimental realization of realignment operation and to achieve this aim, we propose a theore… ▽ More Realignment operation has a significant role in detecting bound as well as free entanglement. Just like partial transposition, it is also based on permutations of the matrix elements. However, the physical implementation of realignment operation is not known yet. In this letter, we address the problem of experimental realization of realignment operation and to achieve this aim, we propose a theoretical proposal for the same. We first show that after applying the realignment operation on a bipartite state, the resulting matrix can be expressed in terms of the partial transposition operation along with column interchange operations. We observed that these column interchange operations forms a permutation matrix which can be implemented via SWAP operator acting on the density matrix. This mathematical framework is used to exactly determine the first moment of the realignment matrix experimentally. This has been done by showing that the first moment of the realignment matrix can be expressed as the expectation value of a SWAP operator which indicates the possibility of its measurement. Further, we have provided an estimation of the higher order realigned moments in terms of the first realigned moment and thus pave a way to estimate the higher order moments experimentally. Next, we develop moments based entanglement detection criteria that detect positive partial transpose entangled states (PPTES) as well as negative partial transpose entangled states (NPTES). Moreover, we define a new matrix realignment operation for three-qubit states and have devised an entanglement criteria that is able to detect three-qubit fully entangled states. We have developed various methods and techniques in the detection of bipartite and tripartite entangled states that may be realized in the current technology. △ Less

Submitted 8 May, 2024; v1 submitted 16 July, 2023; originally announced July 2023.

Comments: 10 Pages, 3 figures

Journal ref: Quantum Inf Process 23, 223 (2024)

High-quality amorphous Silicon Carbide for hybrid photonic integration at low temperature

Authors: Bruno Lopez-Rodriguez, Roald Van Der Kolk, Samarth Aggarwal, Naresh Sharma, Zizheng Li, Daniel Van Der Plaats, Thomas Scholte, Jin Chang, Silvania F. Pereira, Simon Groeblacher, Harish Bhaskaran, Iman Esmaeil Zadeh Zadeh

Abstract: Integrated photonic platforms have proliferated in recent years, each demonstrating its own unique strengths and shortcomings. However, given the processing incompatibilities of different platforms, a formidable challenge in the field of integrated photonics still remains for combining the strength of different optical materials in one hybrid integrated platform. Silicon carbide is a material of g… ▽ More Integrated photonic platforms have proliferated in recent years, each demonstrating its own unique strengths and shortcomings. However, given the processing incompatibilities of different platforms, a formidable challenge in the field of integrated photonics still remains for combining the strength of different optical materials in one hybrid integrated platform. Silicon carbide is a material of great interest because of its high refractive index, strong second and third-order non-linearities and broad transparecy window in the visible and near infrared. However, integrating SiC has been difficult, and current approaches rely on transfer bonding techniques, that are time consuming, expensive and lacking precision in layer thickness. Here, we demonstrate high index Amorphous Silicon Carbide (a-SiC) films deposited at 150$^{\circ}$C and verify the high performance of the platform by fabricating standard photonic waveguides and ring resonators. The intrinsic quality factors of single-mode ring resonators were in the range of $Q_{int} = (4.7-5.7)\times10^5$ corresponding to optical losses between 0.78-1.06 dBでしべる/cm. We then demonstrate the potential of this platform for future heterogeneous integration with ultralow loss thin SiN and LiNbO$_3$ platforms. △ Less

Submitted 7 June, 2023; originally announced June 2023.

Large Population Games on Constrained Unreliable Networks

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar

Abstract: This paper studies an $N$--agent cost-coupled game where the agents are connected via an unreliable capacity constrained network. Each agent receives state information over that network which loses packets with probability $p$. A Base station (BS) actively schedules agent communications over the network by minimizing a weighted Age of Information (WAoI) based cost function under a capacity limit… ▽ More This paper studies an $N$--agent cost-coupled game where the agents are connected via an unreliable capacity constrained network. Each agent receives state information over that network which loses packets with probability $p$. A Base station (BS) actively schedules agent communications over the network by minimizing a weighted Age of Information (WAoI) based cost function under a capacity limit $\mathcal{C} < N$ on the number of transmission attempts at each instant. Under a standard information structure, we show that the problem can be decoupled into a scheduling problem for the BS and a game problem for the $N$ agents. Since the scheduling problem is an NP hard combinatorics problem, we propose an approximately optimal solution which approaches the optimal solution as $N \rightarrow \infty$. In the process, we also provide some insights on the case without channel erasure. Next, to solve the large population game problem, we use the mean-field game framework to compute an approximate decentralized Nash equilibrium. Finally, we validate the theoretical results using a numerical example. △ Less

Submitted 16 March, 2023; originally announced March 2023.

Comments: Submitted to IEEE for possible publication

MABNet: Master Assistant Buddy Network with Hybrid Learning for Image Retrieval

Authors: Rohit Agarwal, Gyanendra Das, Saksham Aggarwal, Alexander Horsch, Dilip K. Prasad

Abstract: Image retrieval has garnered growing interest in recent times. The current approaches are either supervised or self-supervised. These methods do not exploit the benefits of hybrid learning using both supervision and self-supervision. We present a novel Master Assistant Buddy Network (MABNet) for image retrieval which incorporates both learning mechanisms. MABNet consists of master and assistant bl… ▽ More Image retrieval has garnered growing interest in recent times. The current approaches are either supervised or self-supervised. These methods do not exploit the benefits of hybrid learning using both supervision and self-supervision. We present a novel Master Assistant Buddy Network (MABNet) for image retrieval which incorporates both learning mechanisms. MABNet consists of master and assistant blocks, both learning independently through supervision and collectively via self-supervision. The master guides the assistant by providing its knowledge base as a reference for self-supervision and the assistant reports its knowledge back to the master by weight transfer. We perform extensive experiments on public datasets with and without post-processing. △ Less

Submitted 6 March, 2023; originally announced March 2023.

Comments: Accepted at International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2023

Mimetic Muscle Rehabilitation Analysis Using Clustering of Low Dimensional 3D Kinect Data

Authors: Sumit Kumar Vishwakarma, Sanjeev Kumar, Shrey Aggarwal, Jan Mareš

Abstract: Facial nerve paresis is a severe complication that arises post-head and neck surgery; This results in articulation problems, facial asymmetry, and severe problems in non-verbal communication. To overcome the side effects of post-surgery facial paralysis, rehabilitation requires which last for several weeks. This paper discusses an unsupervised approach to rehabilitating patients who have temporary… ▽ More Facial nerve paresis is a severe complication that arises post-head and neck surgery; This results in articulation problems, facial asymmetry, and severe problems in non-verbal communication. To overcome the side effects of post-surgery facial paralysis, rehabilitation requires which last for several weeks. This paper discusses an unsupervised approach to rehabilitating patients who have temporary facial paralysis due to damage in mimetic muscles. The work aims to make the rehabilitation process objective compared to the current subjective approach, such as House-Brackmann (HB) scale. Also, the approach will assist clinicians by reducing their workload in assessing the improvement during rehabilitation. This paper focuses on the clustering approach to monitor the rehabilitation process. We compare the results obtained from different clustering algorithms on various forms of the same data set, namely dynamic form, data expressed as functional data using B-spline basis expansion, and by finding the functional principal components of the functional data. The study contains data set of 85 distinct patients with 120 measurements obtained using a Kinect stereo-vision camera. The method distinguish effectively between patients with the least and greatest degree of facial paralysis, however patients with adjacent degrees of paralysis provide some challenges. In addition, we compared the cluster results to the HB scale outputs. △ Less

Submitted 15 February, 2023; originally announced February 2023.

Entanglement detection in arbitrary dimensional bipartite quantum systems through partial realigned moments

Authors: Shruti Aggarwal, Satyabrata Adhikari, A. S. Majumdar

Abstract: Detection of entanglement through partial knowledge of the quantum state is a challenge to implement efficiently. Here we propose a separability criterion for detecting bipartite entanglement in arbitrary dimensional quantum states using partial moments of the realigned density matrix. Our approach enables detection of both distillable and bound entangled states through a common framework. We illu… ▽ More Detection of entanglement through partial knowledge of the quantum state is a challenge to implement efficiently. Here we propose a separability criterion for detecting bipartite entanglement in arbitrary dimensional quantum states using partial moments of the realigned density matrix. Our approach enables detection of both distillable and bound entangled states through a common framework. We illustrate the efficiency of our method through examples of states belonging to both the above categories, which are not detectable using comparable other schemes relying on partial state information. The formalism of employing partial realigned moments proposed here is further shown to be effective for two-qubit systems too, with a slight modification of our separability criterion. △ Less

Submitted 8 March, 2024; v1 submitted 9 February, 2023; originally announced February 2023.

Comments: 11 pages, 3 figures

Journal ref: Phys. Rev. A 109, 012404, (2024)

Towards Precision in Appearance-based Gaze Estimation in the Wild

Authors: Murthy L. R. D., Abhishek Mukhopadhyay, Shambhavi Aggarwal, Ketan Anand, Pradipta Biswas

Abstract: Appearance-based gaze estimation systems have shown great progress recently, yet the performance of these techniques depend on the datasets used for training. Most of the existing gaze estimation datasets setup in interactive settings were recorded in laboratory conditions and those recorded in the wild conditions display limited head pose and illumination variations. Further, we observed little a… ▽ More Appearance-based gaze estimation systems have shown great progress recently, yet the performance of these techniques depend on the datasets used for training. Most of the existing gaze estimation datasets setup in interactive settings were recorded in laboratory conditions and those recorded in the wild conditions display limited head pose and illumination variations. Further, we observed little attention so far towards precision evaluations of existing gaze estimation approaches. In this work, we present a large gaze estimation dataset, PARKS-Gaze, with wider head pose and illumination variation and with multiple samples for a single Point of Gaze (PoG). The dataset contains 974 minutes of data from 28 participants with a head pose range of 60 degrees in both yaw and pitch directions. Our within-dataset and cross-dataset evaluations and precision evaluations indicate that the proposed dataset is more challenging and enable models to generalize on unseen participants better than the existing in-the-wild datasets. The project page can be accessed here: https://github.com/lrdmurthy/PARKS-Gaze △ Less

Submitted 13 February, 2023; v1 submitted 5 February, 2023; originally announced February 2023.

Physical realization of realignment criteria using structural physical approximation

Authors: Shruti Aggarwal, Anu Kumari, Satyabrata Adhikari

Abstract: Entanglement detection is an important problem in quantum information theory because quantum entanglement is a key resource in quantum information processing. Realignment criteria is a powerful tool for detection of entangled states in bipartite and multipartite quantum system. It is an important criteria for entanglement detection because it works well; not only for negative partial transpose ent… ▽ More Entanglement detection is an important problem in quantum information theory because quantum entanglement is a key resource in quantum information processing. Realignment criteria is a powerful tool for detection of entangled states in bipartite and multipartite quantum system. It is an important criteria for entanglement detection because it works well; not only for negative partial transpose entangled states (NPTES) but also for positive partial transpose entangled states (PPTES). Since the matrix corresponding to realignment map is indefinite so the experimental implementation of the map is an obscure task. In this work, firstly, we have approximated the realignment map to a positive map using the method of structural physical approximation (SPA) and then we have shown that the structural physical approximation of realignment map (SPA-R) is completely positive. Positivity of the constructed map is characterized using moments which can be physically measured. Next, we develop a separability criterion based on our SPA-R map in the form of an inequality and have shown that the developed criterion not only detect NPTES but also PPTES. We have provided some examples to support the results obtained. Moreover, we have analysed the error that may occur because of approximating the realignment map. △ Less

Submitted 8 July, 2023; v1 submitted 24 January, 2023; originally announced January 2023.

Comments: 13 pages, 7 figures, 1 table, Appendix, Accepted in Physical Review A

Journal ref: Physical Review A 108, 012422 (2023)

On Designing Light-Weight Object Trackers through Network Pruning: Use CNNs or Transformers?

Authors: Saksham Aggarwal, Taneesh Gupta, Pawan Kumar Sahu, Arnav Chavan, Rishabh Tiwari, Dilip K. Prasad, Deepak K. Gupta

Abstract: Object trackers deployed on low-power devices need to be light-weight, however, most of the current state-of-the-art (SOTA) methods rely on using compute-heavy backbones built using CNNs or transformers. Large sizes of such models do not allow their deployment in low-power conditions and designing compressed variants of large tracking models is of great importance. This paper demonstrates how high… ▽ More Object trackers deployed on low-power devices need to be light-weight, however, most of the current state-of-the-art (SOTA) methods rely on using compute-heavy backbones built using CNNs or transformers. Large sizes of such models do not allow their deployment in low-power conditions and designing compressed variants of large tracking models is of great importance. This paper demonstrates how highly compressed light-weight object trackers can be designed using neural architectural pruning of large CNN and transformer based trackers. Further, a comparative study on architectural choices best suited to design light-weight trackers is provided. A comparison between SOTA trackers using CNNs, transformers as well as the combination of the two is presented to study their stability at various compression ratios. Finally results for extreme pruning scenarios going as low as 1% in some cases are shown to study the limits of network pruning in object tracking. This work provides deeper insights into designing highly efficient trackers from existing SOTA methods. △ Less

Submitted 26 March, 2023; v1 submitted 24 November, 2022; originally announced November 2022.

Comments: Accepted at IEEE ICASSP 2023

Partial Binarization of Neural Networks for Budget-Aware Efficient Learning

Authors: Udbhav Bamba, Neeraj Anand, Saksham Aggarwal, Dilip K. Prasad, Deepak K. Gupta

Abstract: Binarization is a powerful compression technique for neural networks, significantly reducing FLOPs, but often results in a significant drop in model performance. To address this issue, partial binarization techniques have been developed, but a systematic approach to mixing binary and full-precision parameters in a single network is still lacking. In this paper, we propose a controlled approach to… ▽ More Binarization is a powerful compression technique for neural networks, significantly reducing FLOPs, but often results in a significant drop in model performance. To address this issue, partial binarization techniques have been developed, but a systematic approach to mixing binary and full-precision parameters in a single network is still lacking. In this paper, we propose a controlled approach to partial binarization, creating a budgeted binary neural network (B2NN) with our MixBin strategy. This method optimizes the mixing of binary and full-precision components, allowing for explicit selection of the fraction of the network to remain binary. Our experiments show that B2NNs created using MixBin outperform those from random or iterative searches and state-of-the-art layer selection methods by up to 3% on the ImageNet-1K dataset. We also show that B2NNs outperform the structured pruning baseline by approximately 23% at the extreme FLOP budget of 15%, and perform well in object tracking, with up to a 12.4% relative improvement over other baselines. Additionally, we demonstrate that B2NNs developed by MixBin can be transferred across datasets, with some cases showing improved performance over directly applying MixBin on the downstream data. △ Less

Submitted 8 November, 2023; v1 submitted 12 November, 2022; originally announced November 2022.

Comments: Accepted at WACV 2023 Conference

Weighted Age of Information based Scheduling for Large Population Games on Networks

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar

Abstract: In this paper, we consider a discrete-time multi-agent system involving $N$ cost-coupled networked rational agents solving a consensus problem and a central Base Station (BS), scheduling agent communications over a network. Due to a hard bandwidth constraint on the number of transmissions through the network, at most $R_d < N$ agents can concurrently access their state information through the netw… ▽ More In this paper, we consider a discrete-time multi-agent system involving $N$ cost-coupled networked rational agents solving a consensus problem and a central Base Station (BS), scheduling agent communications over a network. Due to a hard bandwidth constraint on the number of transmissions through the network, at most $R_d < N$ agents can concurrently access their state information through the network. Under standard assumptions on the information structure of the agents and the BS, we first show that the control actions of the agents are free of any dual effect, allowing for separation between estimation and control problems at each agent. Next, we propose a weighted age of information (WAoI) metric for the scheduling problem of the BS, where the weights depend on the estimation error of the agents. The BS aims to find the optimum scheduling policy that minimizes the WAoI, subject to the hard bandwidth constraint. Since this problem is NP hard, we first relax the hard constraint to a soft update rate constraint, and then compute an optimal policy for the relaxed problem by reformulating it into a Markov Decision Process (MDP). This then inspires a sub-optimal policy for the bandwidth constrained problem, which is shown to approach the optimal policy as $N \rightarrow \infty$. Next, we solve the consensus problem using the mean-field game framework wherein we first design decentralized control policies for a limiting case of the $N$-agent system (as $N \rightarrow \infty$). By explicitly constructing the mean-field system, we prove the existence and uniqueness of the mean-field equilibrium. Consequently, we show that the obtained equilibrium policies constitute an $εいぷしろん$-Nash equilibrium for the finite agent system. Finally, we validate the performance of both the scheduling and the control policies through numerical simulations. △ Less

Submitted 26 December, 2022; v1 submitted 26 September, 2022; originally announced September 2022.

Comments: This work has been submitted to IEEE for possible publication

GPTs at Factify 2022: Prompt Aided Fact-Verification

Authors: Pawan Kumar Sahu, Saksham Aggarwal, Taneesh Gupta, Gyanendra Das

Abstract: One of the most pressing societal issues is the fight against false news. The false claims, as difficult as they are to expose, create a lot of damage. To tackle the problem, fact verification becomes crucial and thus has been a topic of interest among diverse research communities. Using only the textual form of data we propose our solution to the problem and achieve competitive results with other… ▽ More One of the most pressing societal issues is the fight against false news. The false claims, as difficult as they are to expose, create a lot of damage. To tackle the problem, fact verification becomes crucial and thus has been a topic of interest among diverse research communities. Using only the textual form of data we propose our solution to the problem and achieve competitive results with other approaches. We present our solution based on two approaches - PLM (pre-trained language model) based method and Prompt based method. The PLM-based approach uses the traditional supervised learning, where the model is trained to take 'x' as input and output prediction 'y' as P(y|x). Whereas, Prompt-based learning reflects the idea to design input to fit the model such that the original objective may be re-framed as a problem of (masked) language modeling. We may further stimulate the rich knowledge provided by PLMs to better serve downstream tasks by employing extra prompts to fine-tune PLMs. Our experiments showed that the proposed method performs better than just fine-tuning PLMs. We achieved an F1 score of 0.6946 on the FACTIFY dataset and a 7th position on the competition leader-board. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Comments: Accepted in AAAI'22: First Workshop on Multimodal Fact-Checking and Hate Speech Detection, Februrary 22 - March 1, 2022,Vancouver, BC, Canada

How does a Rational Agent Act in an Epidemic?

Authors: S. Yagiz Olmez, Shubham Aggarwal, Jin Won Kim, Erik Miehling, Tamer Başar, Matthew West, Prashant G. Mehta

Abstract: Evolution of disease in a large population is a function of the top-down policy measures from a centralized planner, as well as the self-interested decisions (to be socially active) of individual agents in a large heterogeneous population. This paper is concerned with understanding the latter based on a mean-field type optimal control model. Specifically, the model is used to investigate the role… ▽ More Evolution of disease in a large population is a function of the top-down policy measures from a centralized planner, as well as the self-interested decisions (to be socially active) of individual agents in a large heterogeneous population. This paper is concerned with understanding the latter based on a mean-field type optimal control model. Specifically, the model is used to investigate the role of partial information on an agent's decision-making, and study the impact of such decisions by a large number of agents on the spread of the virus in the population. The motivation comes from the presymptomatic and asymptomatic spread of the COVID-19 virus where an agent unwittingly spreads the virus. We show that even in a setting with fully rational agents, limited information on the viral state can result in an epidemic growth. △ Less

Submitted 5 June, 2022; originally announced June 2022.

Comments: arXiv admin note: text overlap with arXiv:2111.10422

Search for an efficient entanglement witness operator for bound entangled states in bipartite quantum systems

Authors: Shruti Aggarwal, Satyabrata Adhikari

Abstract: Entanglement detection problem is one of the important problem in quantum information theory. Gurvit showed that this problem is NP complete and thus this may be the possible reason that only one criterion is not sufficient to detect all entangled states. There are some powerful entanglement detection criterion such as partial transposition criterion, realignment criterion but it may not be possib… ▽ More Entanglement detection problem is one of the important problem in quantum information theory. Gurvit showed that this problem is NP complete and thus this may be the possible reason that only one criterion is not sufficient to detect all entangled states. There are some powerful entanglement detection criterion such as partial transposition criterion, realignment criterion but it may not be possible to implement them successfully in the experiment. This situation can be avoided if the entanglement is detected through the construction of witness operator method. In this work, we take an analytical approach to construct a witness operator. To achieve this task, we first construct a linear map using partial transposition and realignment operation. Then we find some conditions on the parameters of the map for which the map represent a positive map. Further, we have constructed a Choi matrix corresponding to the map and have shown that it is not completely positive. We then construct an entanglement witness operator, which is based on the linear combination of the function of Choi matrix and the identity matrix and it can detect both NPTES and PPTES. Finally, we prove its efficiency by detecting several bipartite bound entangled states which were previously undetected by some well-known separability criteria. We also compared the detection power of our witness operator with three well-known powerful entanglement detection criteria, namely, dV criterion, CCNR criterion and the separability criteria based on correlation tensor (CT) proposed by Sarbicki et. al. and find that our witness operator detect more entangled states than these criterion. △ Less

Submitted 20 July, 2022; v1 submitted 13 April, 2022; originally announced April 2022.

Comments: Accepted in Annals of Physics. In this work, we propose a witness operator detects more entangled states than the other detection criteria, namely, dV criterion, CCNR criterion and separability criterion based on correlation tensor

Journal ref: Annals of Physics 444 (2022) 169043

Lower Bounds for Restricted Schemes in the Two-Adaptive Bitprobe Model

Authors: Sreshth Aggarwal, Deepanjan Kesh, Divyam Singal

Abstract: In the adaptive bitprobe model answering membership queries in two bitprobes, we consider the class of restricted schemes as introduced by Kesh and Sharma (Discrete Applied Mathematics 2021). In that paper, the authors showed that such restricted schemes storing subsets of size 2 require $Ωおめが(m^\frac{2}{3})$ space. In this paper, we generalise the result to arbitrary subsets of size $n$, and prove t… ▽ More In the adaptive bitprobe model answering membership queries in two bitprobes, we consider the class of restricted schemes as introduced by Kesh and Sharma (Discrete Applied Mathematics 2021). In that paper, the authors showed that such restricted schemes storing subsets of size 2 require $Ωおめが(m^\frac{2}{3})$ space. In this paper, we generalise the result to arbitrary subsets of size $n$, and prove that the space required for such restricted schemes will be $Ωおめが(\left(\frac{m}{n}\right)^{1 - \frac{1}{\lfloor n / 4 \rfloor + 2}})$. △ Less

Submitted 8 April, 2022; v1 submitted 7 April, 2022; originally announced April 2022.

Comments: 19 pages, 1 figure, IWOCA2022, full version of paper

Linear Quadratic Mean-Field Games with Communication Constraints

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Tamer Başar

Abstract: In this paper, we study a large population game with heterogeneous dynamics and cost functions solving a consensus problem. Moreover, the agents have communication constraints which appear as: (1) an Additive-White Gaussian Noise (AWGN) channel, and (2) asynchronous data transmission via a fixed scheduling policy. Since the complexity of solving the game increases with the number of agents, we use… ▽ More In this paper, we study a large population game with heterogeneous dynamics and cost functions solving a consensus problem. Moreover, the agents have communication constraints which appear as: (1) an Additive-White Gaussian Noise (AWGN) channel, and (2) asynchronous data transmission via a fixed scheduling policy. Since the complexity of solving the game increases with the number of agents, we use the Mean-Field Game paradigm to solve it. Under standard assumptions on the information structure of the agents, we prove that the control of the agent in the MFG setting is free of the dual effect. This allows us to obtain an equilibrium control policy for the generic agent, which is a function of only the local observation of the agent. Furthermore, the equilibrium mean-field trajectory is shown to follow linear dynamics, hence making it computable. We show that in the finite population game, the equilibrium control policy prescribed by the MFG analysis constitutes an $εいぷしろん$-Nash equilibrium, where $εいぷしろん$ tends to zero as the number of agents goes to infinity. The paper is concluded with simulations demonstrating the performance of the equilibrium control policy. △ Less

Submitted 25 August, 2022; v1 submitted 10 March, 2022; originally announced March 2022.

Comments: Accepted in American Control Conference 2022

Shear-induced mixing of granular materials featuring broad granule size distributions

Authors: Joyjit Chattoraj, Nguyen Hoang Huy, Saurabh Aggarwal, Mohamed Salahuddin Habibullah, Farzam Farbiz

Abstract: Granular flows during a shear-induced mixing process are studied using Discrete Element Methods. The aim is to understand the underlying elementary mechanisms of transition from unmixed to mixed phases for a granular material featuring a broad distribution of particles, which we investigate systematically by varying the strain rate and system size. Here the strain rate varies over four orders of m… ▽ More Granular flows during a shear-induced mixing process are studied using Discrete Element Methods. The aim is to understand the underlying elementary mechanisms of transition from unmixed to mixed phases for a granular material featuring a broad distribution of particles, which we investigate systematically by varying the strain rate and system size. Here the strain rate varies over four orders of magnitude and the system size varies from ten thousand to more than a million granules. A strain rate-dependent transition from quasistatic to purely inertial flow is observed. At the macroscopic scale, the contact stresses drop due to the formation of shear-induced instabilities that serves as an onset of granular flows and initiates mixing between the granules. The stress-drop displays a profound system size dependence. At the granular scale, mixing dynamics are correlated with the formation of shear bands, which result in significantly different timescales of mixing, especially for those regions that are close to the system walls and the bulk. Overall, our results reveal that although the transient dynamics display a generic behavior these have a significant finite-size effect. In contrast, macroscopic behaviors at steady states have negligible system size dependence. △ Less

Submitted 13 February, 2022; originally announced February 2022.

Journal ref: 104, 2021, 044910

Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay

Authors: Kuluhan Binici, Shivam Aggarwal, Nam Trung Pham, Karianto Leman, Tulika Mitra

Abstract: Data-Free Knowledge Distillation (KD) allows knowledge transfer from a trained neural network (teacher) to a more compact one (student) in the absence of original training data. Existing works use a validation set to monitor the accuracy of the student over real data and report the highest performance throughout the entire process. However, validation data may not be available at distillation time… ▽ More Data-Free Knowledge Distillation (KD) allows knowledge transfer from a trained neural network (teacher) to a more compact one (student) in the absence of original training data. Existing works use a validation set to monitor the accuracy of the student over real data and report the highest performance throughout the entire process. However, validation data may not be available at distillation time either, making it infeasible to record the student snapshot that achieved the peak accuracy. Therefore, a practical data-free KD method should be robust and ideally provide monotonically increasing student accuracy during distillation. This is challenging because the student experiences knowledge degradation due to the distribution shift of the synthetic data. A straightforward approach to overcome this issue is to store and rehearse the generated samples periodically, which increases the memory footprint and creates privacy concerns. We propose to model the distribution of the previously observed synthetic samples with a generative network. In particular, we design a Variational Autoencoder (VAE) with a training objective that is customized to learn the synthetic data representations optimally. The student is rehearsed by the generative pseudo replay technique, with samples produced by the VAE. Hence knowledge degradation can be prevented without storing any samples. Experiments on image classification benchmarks show that our method optimizes the expected value of the distilled model accuracy while eliminating the large memory overhead incurred by the sample-storing methods. △ Less

Submitted 21 March, 2022; v1 submitted 9 January, 2022; originally announced January 2022.

Comments: Accepted by the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22)

Experimental Demonstration of Neuromorphic Network with STT MTJ Synapses

Authors: Peng Zhou, Alexander J. Edwards, Fred B. Mancoff, Dimitri Houssameddine, Sanjeev Aggarwal, Joseph S. Friedman

Abstract: We present the first experimental demonstration of a neuromorphic network with magnetic tunnel junction (MTJ) synapses, which performs image recognition via vector-matrix multiplication. We also simulate a large MTJ network performing MNIST handwritten digit recognition, demonstrating that MTJ crossbars can match memristor accuracy while providing increased precision, stability, and endurance. We present the first experimental demonstration of a neuromorphic network with magnetic tunnel junction (MTJ) synapses, which performs image recognition via vector-matrix multiplication. We also simulate a large MTJ network performing MNIST handwritten digit recognition, demonstrating that MTJ crossbars can match memristor accuracy while providing increased precision, stability, and endurance. △ Less

Submitted 9 December, 2021; originally announced December 2021.

Non-uniform magnetic field as a booster for quantum speed limit: faster quantum information processing

Authors: Srishty Aggarwal, Subhashish Banerjee, Arindam Ghosh, Banibrata Mukhopadhyay

Abstract: We probe the quantum speed limit (QSL) of an electron when it is trapped in a non-uniform magnetic field. We show that the QSL increases to a large value, but within the regime of causality, by choosing a proper variation in magnetic fields. We also probe the dependence of QSL on spin of electron and find that it is higher for spin-down electron in the relativistic regime. This can be useful in ac… ▽ More We probe the quantum speed limit (QSL) of an electron when it is trapped in a non-uniform magnetic field. We show that the QSL increases to a large value, but within the regime of causality, by choosing a proper variation in magnetic fields. We also probe the dependence of QSL on spin of electron and find that it is higher for spin-down electron in the relativistic regime. This can be useful in achieving a faster speed of transmission of quantum information. Further, we use the Bremermann--Bekenstein bound to find a critical magnetic field that bridges the gap between non-relativistic and relativistic treatments and relates to the stability of matter. An analytical framework is developed. We also provide a plausible experimental design to supplement our theory. △ Less

Submitted 23 August, 2022; v1 submitted 8 December, 2021; originally announced December 2021.

Comments: 12 pages including 8 figures

Journal ref: New J. Phys. 24 (2022) 085001

Modeling Presymptomatic Spread in Epidemics via Mean-Field Games

Authors: S. Yagiz Olmez, Shubham Aggarwal, Jin Won Kim, Erik Miehling, Tamer Başar, Matthew West, Prashant G. Mehta

Abstract: This paper is concerned with developing mean-field game models for the evolution of epidemics. Specifically, an agent's decision -- to be socially active in the midst of an epidemic -- is modeled as a mean-field game with health-related costs and activity-related rewards. By considering the fully and partially observed versions of this problem, the role of information in guiding an agent's rationa… ▽ More This paper is concerned with developing mean-field game models for the evolution of epidemics. Specifically, an agent's decision -- to be socially active in the midst of an epidemic -- is modeled as a mean-field game with health-related costs and activity-related rewards. By considering the fully and partially observed versions of this problem, the role of information in guiding an agent's rational decision is highlighted. The main contributions of the paper are to derive the equations for the mean-field game in both fully and partially observed settings of the problem, to present a complete analysis of the fully observed case, and to present some analytical results for the partially observed case. △ Less

Submitted 19 November, 2021; originally announced November 2021.

Dynamics of relativistic electrons in non-uniform magnetic fields and its applications in quantum computing and astrophysics

Authors: Srishty Aggarwal, Banibrata Mukhopadhyay

Abstract: We explore the two-dimensional motion of relativistic electrons when they are trapped in magnetic fields having spatial power-law variation. Its impacts include lifting of degeneracy that emerged in the case of the constant magnetic field, special alignment of Landau levels of spin-up and spin-down electrons depending on whether the magnetic field is increasing or decreasing from the centre, split… ▽ More We explore the two-dimensional motion of relativistic electrons when they are trapped in magnetic fields having spatial power-law variation. Its impacts include lifting of degeneracy that emerged in the case of the constant magnetic field, special alignment of Landau levels of spin-up and spin-down electrons depending on whether the magnetic field is increasing or decreasing from the centre, splitting of Landau levels of electrons with zero angular momentum from that of positive one and the change in the equation of state of matter. Landau quantization (LQ) in variable magnetic fields has interdisciplinary applications in a variety of disciplines ranging from condensed matter to quantum information. As examples, we discuss the increase in quantum speed of the electron in presence of spatially increasing magnetic field; and the attainment of super Chandrasekhar mass of white dwarfs by taking into account LQ and Lorentz force simultaneously. △ Less

Submitted 17 November, 2021; originally announced November 2021.

Comments: 12 pages including 6 figures; based on the talk given in the parallel session "Strong Electromagnetic and Gravitational Field Physics: From Laboratories to Early Universe" in the Sixteenth Marcel Grossmann Meeting held online during July 5-10, 2021; to appear in the proceedings of Sixteenth Marcel Grossmann Meeting

Report number: The Sixteenth Marcel Grossmann Meeting, pp. 4362-4373 (2023)

Journal ref: World Scientific, The Sixteenth Marcel Grossmann Meeting, pp. 4362-4373 (2023)

Relativistic Landau quantization in non-uniform magnetic field and its applications to white dwarfs and quantum information

Authors: Srishty Aggarwal, Banibrata Mukhopadhyay, Gianluca Gregori

Abstract: We investigate the two-dimensional motion of relativistic cold electrons in the presence of `strictly' spatially varying magnetic fields satisfying, however, no magnetic monopole condition. We find that the degeneracy of Landau levels, which arises in the case of the constant magnetic field, lifts out when the field is variable and the energy levels of spin-up and spin-down electrons align in an i… ▽ More We investigate the two-dimensional motion of relativistic cold electrons in the presence of `strictly' spatially varying magnetic fields satisfying, however, no magnetic monopole condition. We find that the degeneracy of Landau levels, which arises in the case of the constant magnetic field, lifts out when the field is variable and the energy levels of spin-up and spin-down electrons align in an interesting way depending on the nature of change of field. Also the varying magnetic field splits Landau levels of electrons with zero angular momentum from positive angular momentum, unlike the constant field which only can split the levels between positive and negative angular momenta. Exploring Landau quantization in non-uniform magnetic fields is a unique venture on its own and has interdisciplinary implications in the fields ranging from condensed matter to astrophysics to quantum information. As examples, we show magnetized white dwarfs, with varying magnetic fields, involved simultaneously with Lorentz force and Landau quantization affecting the underlying degenerate electron gas, exhibiting a significant violation of the Chandrasekhar mass-limit; and an increase in quantum speed of electrons in the presence of a spatially growing magnetic field. △ Less

Submitted 18 October, 2021; originally announced October 2021.

Comments: 30 pages including 12 figures; accepted for publication in SciPost Physics

Writers Gonna Wait: The Effectiveness of Notifications to Initiate Aversive Action in Writing Procrastination

Authors: Chatchai Wangwiwattana, Sunjoli Aggarwal, Eric C. Larson

Abstract: This paper evaluates the use of notifications to reduce aversive-task-procrastination by helping initiate action. Specifically, we focus on aversion to graded writing tasks. We evaluate software designs commonly used by behavior change applications, such as goal setting and action support systems. We conduct a two-phase control trial experiment with 21 college students tasked to write two 3000-wor… ▽ More This paper evaluates the use of notifications to reduce aversive-task-procrastination by helping initiate action. Specifically, we focus on aversion to graded writing tasks. We evaluate software designs commonly used by behavior change applications, such as goal setting and action support systems. We conduct a two-phase control trial experiment with 21 college students tasked to write two 3000-word writing assignments (14 students fully completed the experiment). Participants use a customized text editor designed to continuously collect writing behavior. The results from the study reveal that notifications have minimal effect in encouraging users to get started. They can also increase negative effects on participants. Other techniques, such as eliminating distraction and showing simple writing statistics, yield higher satisfaction among participants as they complete the writing task. Furthermore, the incorporation of text mining decreases aversion to the task and helps participants overcome writer's block. Finally, we discuss lessons learned from our evaluation that help quantify the difficulty of behavior change for writing procrastination, with emphasis on goals for the HCI community. △ Less

Submitted 25 January, 2021; originally announced January 2021.

ACM Class: H.5.m

Evaluating Empathetic Chatbots in Customer Service Settings

Authors: Akshay Agarwal, Shashank Maiya, Sonu Aggarwal

Abstract: Customer service is a setting that calls for empathy in live human agent responses. Recent advances have demonstrated how open-domain chatbots can be trained to demonstrate empathy when responding to live human utterances. We show that a blended skills chatbot model that responds to customer queries is more likely to resemble actual human agent response if it is trained to recognize emotion and ex… ▽ More Customer service is a setting that calls for empathy in live human agent responses. Recent advances have demonstrated how open-domain chatbots can be trained to demonstrate empathy when responding to live human utterances. We show that a blended skills chatbot model that responds to customer queries is more likely to resemble actual human agent response if it is trained to recognize emotion and exhibit appropriate empathy, than a model without such training. For our analysis, we leverage a Twitter customer service dataset containing several million customer<->agent dialog examples in customer service contexts from 20 well-known brands. △ Less

Submitted 4 January, 2021; originally announced January 2021.

Comments: 8 pages, 7 figures

Get It Scored Using AutoSAS -- An Automated System for Scoring Short Answers

Authors: Yaman Kumar, Swati Aggarwal, Debanjan Mahata, Rajiv Ratn Shah, Ponnurangam Kumaraguru, Roger Zimmermann

Abstract: In the era of MOOCs, online exams are taken by millions of candidates, where scoring short answers is an integral part. It becomes intractable to evaluate them by human graders. Thus, a generic automated system capable of grading these responses should be designed and deployed. In this paper, we present a fast, scalable, and accurate approach towards automated Short Answer Scoring (SAS). We propos… ▽ More In the era of MOOCs, online exams are taken by millions of candidates, where scoring short answers is an integral part. It becomes intractable to evaluate them by human graders. Thus, a generic automated system capable of grading these responses should be designed and deployed. In this paper, we present a fast, scalable, and accurate approach towards automated Short Answer Scoring (SAS). We propose and explain the design and development of a system for SAS, namely AutoSAS. Given a question along with its graded samples, AutoSAS can learn to grade that prompt successfully. This paper further lays down the features such as lexical diversity, Word2Vec, prompt, and content overlap that plays a pivotal role in building our proposed model. We also present a methodology for indicating the factors responsible for scoring an answer. The trained model is evaluated on an extensively used public dataset, namely Automated Student Assessment Prize Short Answer Scoring (ASAP-SAS). AutoSAS shows state-of-the-art performance and achieves better results by over 8% in some of the question prompts as measured by Quadratic Weighted Kappa (QWK), showing performance comparable to humans. △ Less

Submitted 21 December, 2020; originally announced December 2020.

Goal-driven Command Recommendations for Analysts

Authors: Samarth Aggarwal, Rohin Garg, Abhilasha Sancheti, Bhanu Prakash Reddy Guda, Iftikhar Ahamath Burhanuddin

Abstract: Recent times have seen data analytics software applications become an integral part of the decision-making process of analysts. The users of these software applications generate a vast amount of unstructured log data. These logs contain clues to the user's goals, which traditional recommender systems may find difficult to model implicitly from the log data. With this assumption, we would like to a… ▽ More Recent times have seen data analytics software applications become an integral part of the decision-making process of analysts. The users of these software applications generate a vast amount of unstructured log data. These logs contain clues to the user's goals, which traditional recommender systems may find difficult to model implicitly from the log data. With this assumption, we would like to assist the analytics process of a user through command recommendations. We categorize the commands into software and data categories based on their purpose to fulfill the task at hand. On the premise that the sequence of commands leading up to a data command is a good predictor of the latter, we design, develop, and validate various sequence modeling techniques. In this paper, we propose a framework to provide goal-driven data command recommendations to the user by leveraging unstructured logs. We use the log data of a web-based analytics software to train our neural network models and quantify their performance, in comparison to relevant and competitive baselines. We propose a custom loss function to tailor the recommended data commands according to the goal information provided exogenously. We also propose an evaluation metric that captures the degree of goal orientation of the recommendations. We demonstrate the promise of our approach by evaluating the models with the proposed metric and showcasing the robustness of our models in the case of adversarial examples, where the user activity is misaligned with selected goal, through offline evaluation. △ Less

Submitted 12 November, 2020; originally announced November 2020.

Comments: 14th ACM Conference on Recommender Systems (RecSys 2020)

Witness Operator Provides Better Estimate of the Lower Bound of Concurrence of Bipartite Bound Entangled States in $d_{1}\otimes d_{2}$ Dimensional System

Authors: Shruti Aggarwal, Satyabrata Adhikari

Abstract: It is known that the witness operator is useful in the detection and quantification of entangled states. This motivated us for the construction of the family of witness operators that can detect many mixed entangled states. This family of witness operators is then used to estimate the lower bound of concurrence of the detected mixed entangled states. Our method of construction of witness operator… ▽ More It is known that the witness operator is useful in the detection and quantification of entangled states. This motivated us for the construction of the family of witness operators that can detect many mixed entangled states. This family of witness operators is then used to estimate the lower bound of concurrence of the detected mixed entangled states. Our method of construction of witness operator is important in the sense that it will estimate a better lower bound of concurrence of the entangled states in arbitrary $d_{1}\otimes d_{2} (d_{1}\leq d_{2})$ dimensional system compared to the lower bound of the concurrence given in \cite{kchen}. We have shown the significance of our constructed witness operator by detecting many bound entangled states that are not detected by the earlier methods and then we use the expectation value of the witness operator to estimate the lower bound of the concurrence of those bound entangled states. △ Less

Submitted 29 January, 2021; v1 submitted 10 October, 2020; originally announced October 2020.

Comments: 12 pages, 5 figures, 4 Tables. Accepted in Quantum Information Processing, 2021

Journal ref: Quan. Inf. Proc. 20, 83 (2021)

OpenIE6: Iterative Grid Labeling and Coordination Analysis for Open Information Extraction

Authors: Keshav Kolluru, Vaibhav Adlakha, Samarth Aggarwal, Mausam, Soumen Chakrabarti

Abstract: A recent state-of-the-art neural open information extraction (OpenIE) system generates extractions iteratively, requiring repeated encoding of partial outputs. This comes at a significant computational cost. On the other hand, sequence labeling approaches for OpenIE are much faster, but worse in extraction quality. In this paper, we bridge this trade-off by presenting an iterative labeling-based s… ▽ More A recent state-of-the-art neural open information extraction (OpenIE) system generates extractions iteratively, requiring repeated encoding of partial outputs. This comes at a significant computational cost. On the other hand, sequence labeling approaches for OpenIE are much faster, but worse in extraction quality. In this paper, we bridge this trade-off by presenting an iterative labeling-based system that establishes a new state of the art for OpenIE, while extracting 10x faster. This is achieved through a novel Iterative Grid Labeling (IGL) architecture, which treats OpenIE as a 2-D grid labeling task. We improve its performance further by applying coverage (soft) constraints on the grid at training time. Moreover, on observing that the best OpenIE systems falter at handling coordination structures, our OpenIE system also incorporates a new coordination analyzer built with the same IGL architecture. This IGL based coordination analyzer helps our OpenIE system handle complicated coordination structures, while also establishing a new state of the art on the task of coordination analysis, with a 12.3 pts improvement in F1 over previous analyzers. Our OpenIE system, OpenIE6, beats the previous systems by as much as 4 pts in F1, while being much faster. △ Less

Submitted 7 October, 2020; originally announced October 2020.

Comments: EMNLP 2020 (Long)

Visual Exploration and Knowledge Discovery from Biomedical Dark Data

Authors: Shashwat Aggarwal, Ramesh Singh

Abstract: Data visualization techniques proffer efficient means to organize and present data in graphically appealing formats, which not only speeds up the process of decision making and pattern recognition but also enables decision-makers to fully understand data insights and make informed decisions. Over time, with the rise in technological and computational resources, there has been an exponential increa… ▽ More Data visualization techniques proffer efficient means to organize and present data in graphically appealing formats, which not only speeds up the process of decision making and pattern recognition but also enables decision-makers to fully understand data insights and make informed decisions. Over time, with the rise in technological and computational resources, there has been an exponential increase in the world's scientific knowledge. However, most of it lacks structure and cannot be easily categorized and imported into regular databases. This type of data is often termed as Dark Data. Data visualization techniques provide a promising solution to explore such data by allowing quick comprehension of information, the discovery of emerging trends, identification of relationships and patterns, etc. In this empirical research study, we use the rich corpus of PubMed comprising of more than 30 million citations from biomedical literature to visually explore and understand the underlying key-insights using various information visualization techniques. We employ a natural language processing based pipeline to discover knowledge out of the biomedical dark data. The pipeline comprises of different lexical analysis techniques like Topic Modeling to extract inherent topics and major focus areas, Network Graphs to study the relationships between various entities like scientific documents and journals, researchers, and, keywords and terms, etc. With this analytical research, we aim to proffer a potential solution to overcome the problem of analyzing overwhelming amounts of information and diminish the limitation of human cognition and perception in handling and examining such large volumes of data. △ Less

Submitted 28 September, 2020; originally announced September 2020.

Metaphor Detection using Deep Contextualized Word Embeddings

Authors: Shashwat Aggarwal, Ramesh Singh

Abstract: Metaphors are ubiquitous in natural language, and their detection plays an essential role in many natural language processing tasks, such as language understanding, sentiment analysis, etc. Most existing approaches for metaphor detection rely on complex, hand-crafted and fine-tuned feature pipelines, which greatly limit their applicability. In this work, we present an end-to-end method composed of… ▽ More Metaphors are ubiquitous in natural language, and their detection plays an essential role in many natural language processing tasks, such as language understanding, sentiment analysis, etc. Most existing approaches for metaphor detection rely on complex, hand-crafted and fine-tuned feature pipelines, which greatly limit their applicability. In this work, we present an end-to-end method composed of deep contextualized word embeddings, bidirectional LSTMs and multi-head attention mechanism to address the task of automatic metaphor detection. Our method, unlike many other existing approaches, requires only the raw text sequences as input features to detect the metaphoricity of a phrase. We compare the performance of our method against the existing baselines on two benchmark datasets, TroFi, and MOH-X respectively. Experimental evaluations confirm the effectiveness of our approach. △ Less

Submitted 26 September, 2020; originally announced September 2020.

Password Guessers Under a Microscope: An In-Depth Analysis to Inform Deployments

Authors: Zach Parish, Connor Cushing, Shourya Aggarwal, Amirali Salehi-Abari, Julie Thorpe

Abstract: Password guessers are instrumental for assessing the strength of passwords. Despite their diversity and abundance, little is known about how different guessers compare to each other. We perform in-depth analyses and comparisons of the guessing abilities and behavior of password guessers. To extend analyses beyond number of passwords cracked, we devise an analytical framework to compare the types o… ▽ More Password guessers are instrumental for assessing the strength of passwords. Despite their diversity and abundance, little is known about how different guessers compare to each other. We perform in-depth analyses and comparisons of the guessing abilities and behavior of password guessers. To extend analyses beyond number of passwords cracked, we devise an analytical framework to compare the types of passwords that guessers generate under various conditions (e.g., limited training data, limited number of guesses, and dissimilar training and target data). Our results show that guessers often produce dissimilar guesses, even when trained on the same data. We leverage this result to show that combinations of computationally-cheap guessers are as effective as computationally intensive guessers, but more efficient. Our insights allow us to provide a concrete set of recommendations for system administrators when performing password checking. △ Less

Submitted 20 February, 2021; v1 submitted 18 August, 2020; originally announced August 2020.

Comments: 14 pages, 20 individual figures including 9 scale bars; clarified that default settings were used for the guessing tools; modified table formats

"Notic My Speech" -- Blending Speech Patterns With Multimedia

Authors: Dhruva Sahrawat, Yaman Kumar, Shashwat Aggarwal, Yifang Yin, Rajiv Ratn Shah, Roger Zimmermann

Abstract: Speech as a natural signal is composed of three parts - visemes (visual part of speech), phonemes (spoken part of speech), and language (the imposed structure). However, video as a medium for the delivery of speech and a multimedia construct has mostly ignored the cognitive aspects of speech delivery. For example, video applications like transcoding and compression have till now ignored the fact h… ▽ More Speech as a natural signal is composed of three parts - visemes (visual part of speech), phonemes (spoken part of speech), and language (the imposed structure). However, video as a medium for the delivery of speech and a multimedia construct has mostly ignored the cognitive aspects of speech delivery. For example, video applications like transcoding and compression have till now ignored the fact how speech is delivered and heard. To close the gap between speech understanding and multimedia video applications, in this paper, we show the initial experiments by modelling the perception on visual speech and showing its use case on video compression. On the other hand, in the visual speech recognition domain, existing studies have mostly modeled it as a classification problem, while ignoring the correlations between views, phonemes, visemes, and speech perception. This results in solutions which are further away from how human perception works. To bridge this gap, we propose a view-temporal attention mechanism to model both the view dependence and the visemic importance in speech recognition and understanding. We conduct experiments on three public visual speech recognition datasets. The experimental results show that our proposed method outperformed the existing work by 4.99% in terms of the viseme error rate. Moreover, we show that there is a strong correlation between our model's understanding of multi-view speech and the human perception. This characteristic benefits downstream applications such as video compression and streaming where a significant number of less important frames can be compressed or eliminated while being able to maximally preserve human speech understanding with good user experience. △ Less

Submitted 12 June, 2020; originally announced June 2020.

Comments: Under Review

IMoJIE: Iterative Memory-Based Joint Open Information Extraction

Authors: Keshav Kolluru, Samarth Aggarwal, Vipul Rathore, Mausam, Soumen Chakrabarti

Abstract: While traditional systems for Open Information Extraction were statistical and rule-based, recently neural models have been introduced for the task. Our work builds upon CopyAttention, a sequence generation OpenIE model (Cui et. al., 2018). Our analysis reveals that CopyAttention produces a constant number of extractions per sentence, and its extracted tuples often express redundant information.… ▽ More While traditional systems for Open Information Extraction were statistical and rule-based, recently neural models have been introduced for the task. Our work builds upon CopyAttention, a sequence generation OpenIE model (Cui et. al., 2018). Our analysis reveals that CopyAttention produces a constant number of extractions per sentence, and its extracted tuples often express redundant information. We present IMoJIE, an extension to CopyAttention, which produces the next extraction conditioned on all previously extracted tuples. This approach overcomes both shortcomings of CopyAttention, resulting in a variable number of diverse extractions per sentence. We train IMoJIE on training data bootstrapped from extractions of several non-neural systems, which have been automatically filtered to reduce redundancy and noise. IMoJIE outperforms CopyAttention by about 18 F1 pts, and a BERT-based strong baseline by 2 F1 pts, establishing a new state of the art for the task. △ Less

Submitted 17 May, 2020; originally announced May 2020.

Journal ref: ACL 2020, Long paper

Synthetic Video Generation for Robust Hand Gesture Recognition in Augmented Reality Applications

Authors: Varun Jain, Shivam Aggarwal, Suril Mehta, Ramya Hebbalaguppe

Abstract: Hand gestures are a natural means of interaction in Augmented Reality and Virtual Reality (AR/VR) applications. Recently, there has been an increased focus on removing the dependence of accurate hand gesture recognition on complex sensor setup found in expensive proprietary devices such as the Microsoft HoloLens, Daqri and Meta Glasses. Most such solutions either rely on multi-modal sensor data or… ▽ More Hand gestures are a natural means of interaction in Augmented Reality and Virtual Reality (AR/VR) applications. Recently, there has been an increased focus on removing the dependence of accurate hand gesture recognition on complex sensor setup found in expensive proprietary devices such as the Microsoft HoloLens, Daqri and Meta Glasses. Most such solutions either rely on multi-modal sensor data or deep neural networks that can benefit greatly from abundance of labelled data. Datasets are an integral part of any deep learning based research. They have been the principal reason for the substantial progress in this field, both, in terms of providing enough data for the training of these models, and, for benchmarking competing algorithms. However, it is becoming increasingly difficult to generate enough labelled data for complex tasks such as hand gesture recognition. The goal of this work is to introduce a framework capable of generating photo-realistic videos that have labelled hand bounding box and fingertip that can help in designing, training, and benchmarking models for hand-gesture recognition in AR/VR applications. We demonstrate the efficacy of our framework in generating videos with diverse backgrounds. △ Less

Submitted 5 December, 2019; v1 submitted 4 November, 2019; originally announced November 2019.

Comments: Presented at the ICCV 2019 Workshop: The 5th International Workshop on Observing And Understanding Hands In Action