Showing 1–44 of 44 results for author: Aggarwal, S

Semantic Communication in Multi-team Dynamic Games: A Mean Field Perspective

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar

Abstract: Coordinating communication and control is a key component in the stability and performance of networked multi-agent systems. While single user networked control systems have gained a lot of attention within this domain, in this work, we address the more challenging problem of large population multi-team dynamic games. In particular, each team constitutes two decision makers (namely, the sensor and… ▽ More Coordinating communication and control is a key component in the stability and performance of networked multi-agent systems. While single user networked control systems have gained a lot of attention within this domain, in this work, we address the more challenging problem of large population multi-team dynamic games. In particular, each team constitutes two decision makers (namely, the sensor and the controller) who coordinate over a shared network to control a dynamically evolving state of interest under costs on both actuation and sensing/communication. Due to the shared nature of the wireless channel, the overall cost of each team depends on other teams' policies, thereby leading to a noncooperative game setup. Due to the presence of a large number of teams, we compute approximate decentralized Nash equilibrium policies for each team using the paradigm of (extended) mean-field games, which is governed by (1) the mean traffic flowing over the channel, and (2) the value of information at the sensor, which highlights the semantic nature of the ensuing communication. In the process, we compute optimal controller policies and approximately optimal sensor policies for each representative team of the mean-field system to alleviate the problem of general non-contractivity of the mean-field fixed point operator associated with the finite cardinality of the sensor action space. Consequently, we also prove the $εいぷしろん$--Nash property of the mean-field equilibrium solution which essentially characterizes how well the solution derived using mean-field analysis performs on the finite-team system. Finally, we provide extensive numerical simulations, which corroborate the theoretical findings and lead to additional insights on the properties of the results presented. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: Submitted to IEEE for possible publication

From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries

Authors: Hitesh Wadhwa, Rahul Seetharaman, Somyaa Aggarwal, Reshmi Ghosh, Samyadeep Basu, Soundararajan Srinivasan, Wenlong Zhao, Shreyas Chaudhari, Ehsan Aghazadeh

Abstract: Retrieval Augmented Generation (RAG) enriches the ability of language models to reason using external context to augment responses for a given user prompt. This approach has risen in popularity due to practical applications in various applications of language models in search, question/answering, and chat-bots. However, the exact nature of how this approach works isn't clearly understood. In this… ▽ More Retrieval Augmented Generation (RAG) enriches the ability of language models to reason using external context to augment responses for a given user prompt. This approach has risen in popularity due to practical applications in various applications of language models in search, question/answering, and chat-bots. However, the exact nature of how this approach works isn't clearly understood. In this paper, we mechanistically examine the RAG pipeline to highlight that language models take shortcut and have a strong bias towards utilizing only the context information to answer the question, while relying minimally on their parametric memory. We probe this mechanistic behavior in language models with: (i) Causal Mediation Analysis to show that the parametric memory is minimally utilized when answering a question and (ii) Attention Contributions and Knockouts to show that the last token residual stream do not get enriched from the subject token in the question, but gets enriched from other informative tokens in the context. We find this pronounced shortcut behaviour true across both LLaMa and Phi family of models. △ Less

Submitted 18 June, 2024; originally announced June 2024.

An Automated Validation Framework for Power Management and Data Retention Logic Kits of Standard Cell Library

Authors: Akshay Karkal Kamath, Bharath Kumar, Sunil Aggarwal, Subramanian Parameswaran, Parag Lonkar, Debi Prasanna, Somasunder Sreenath

Abstract: The development of a standard cell library involves characterization of a number of gate-level circuits at various cell-level abstractions. Verifying the behavior of these cells largely depends on the manual skills of the circuit designers. Especially challenging are the power management and data retention cells which must be checked thoroughly for voltage and power configurations in addition to t… ▽ More The development of a standard cell library involves characterization of a number of gate-level circuits at various cell-level abstractions. Verifying the behavior of these cells largely depends on the manual skills of the circuit designers. Especially challenging are the power management and data retention cells which must be checked thoroughly for voltage and power configurations in addition to their logic functionality. Also, when standard cells are extracted into various models, any inconsistencies in these models typically goes unchecked during library development. Thus, validating these cells exhaustively prior to customer delivery is highly advantageous to not only improve customer satisfaction but also to reduce design costs. We address this challenge by presenting a methodology to validate the power management and data retention cells that are used in the logical design flow of low-power chips. For a quick adoption by standard cell library design teams, the framework is fully automated and runs out-of-the-box. The proposed framework has been implemented and deployed within the Samsung Foundry ecosystem to enhance the overall quality of library design kit deliverables. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Comments: 33rd Design and Verification Conference and Exhibition United States (DVCon U.S. 2021)

Robot Safety Monitoring using Programmable Light Curtains

Authors: Karnik Ram, Shobhit Aggarwal, Robert Tamburo, Siddharth Ancha, Srinivasa Narasimhan

Abstract: As factories continue to evolve into collaborative spaces with multiple robots working together with human supervisors in the loop, ensuring safety for all actors involved becomes critical. Currently, laser-based light curtain sensors are widely used in factories for safety monitoring. While these conventional safety sensors meet high accuracy standards, they are difficult to reconfigure and can o… ▽ More As factories continue to evolve into collaborative spaces with multiple robots working together with human supervisors in the loop, ensuring safety for all actors involved becomes critical. Currently, laser-based light curtain sensors are widely used in factories for safety monitoring. While these conventional safety sensors meet high accuracy standards, they are difficult to reconfigure and can only monitor a fixed user-defined region of space. Furthermore, they are typically expensive. Instead, we leverage a controllable depth sensor, programmable light curtains (PLC), to develop an inexpensive and flexible real-time safety monitoring system for collaborative robot workspaces. Our system projects virtual dynamic safety envelopes that tightly envelop the moving robot at all times and detect any objects that intrude the envelope. Furthermore, we develop an instrumentation algorithm that optimally places (multiple) PLCs in a workspace to maximize the visibility coverage of robots. Our work enables fence-less human-robot collaboration, while scaling to monitor multiple robots with few sensors. We analyze our system in a real manufacturing testbed with four robot arms and demonstrate its capabilities as a fast, accurate, and inexpensive safety monitoring solution. △ Less

Submitted 4 April, 2024; originally announced April 2024.

Comments: Under review for IROS '24. Webpage http://cmu-mfi.github.io/plc-safety

A Mean Field Game Model for Timely Computation in Edge Computing Systems

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Sennur Ulukus, Tamer Başar

Abstract: We consider the problem of task offloading in multi-access edge computing (MEC) systems constituting $N$ devices assisted by an edge server (ES), where the devices can split task execution between a local processor and the ES. Since the local task execution and communication with the ES both consume power, each device must judiciously choose between the two. We model the problem as a large populat… ▽ More We consider the problem of task offloading in multi-access edge computing (MEC) systems constituting $N$ devices assisted by an edge server (ES), where the devices can split task execution between a local processor and the ES. Since the local task execution and communication with the ES both consume power, each device must judiciously choose between the two. We model the problem as a large population non-cooperative game among the $N$ devices. Since computation of an equilibrium in this scenario is difficult due to the presence of a large number of devices, we employ the mean-field game framework to reduce the finite-agent game problem to a generic user's multi-objective optimization problem, with a coupled consistency condition. By leveraging the novel age of information (AoI) metric, we invoke techniques from stochastic hybrid systems (SHS) theory and study the tradeoffs between increasing information freshness and reducing power consumption. In numerical simulations, we validate that a higher load at the ES may lead devices to upload their task to the ES less often. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Comments: Submitted to IEEE for possible publication

Policy Optimization finds Nash Equilibrium in Regularized General-Sum LQ Games

Authors: Muhammad Aneeq uz Zaman, Shubham Aggarwal, Melih Bastopcu, Tamer Başar

Abstract: In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum $N$-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies. Moreover, it delineates sufficient conditions, contingent upon the adequacy of entropy regularization, for the uniqueness of the NE within the game. As Policy Optimizatio… ▽ More In this paper, we investigate the impact of introducing relative entropy regularization on the Nash Equilibria (NE) of General-Sum $N$-agent games, revealing the fact that the NE of such games conform to linear Gaussian policies. Moreover, it delineates sufficient conditions, contingent upon the adequacy of entropy regularization, for the uniqueness of the NE within the game. As Policy Optimization serves as a foundational approach for Reinforcement Learning (RL) techniques aimed at finding the NE, in this work we prove the linear convergence of a policy optimization algorithm which (subject to the adequacy of entropy regularization) is capable of provably attaining the NE. Furthermore, in scenarios where the entropy regularization proves insufficient, we present a $δでるた$-augmentation technique, which facilitates the achievement of an $εいぷしろん$-NE within the game. △ Less

Submitted 25 March, 2024; originally announced April 2024.

CRISP: Hybrid Structured Sparsity for Class-aware Model Pruning

Authors: Shivam Aggarwal, Kuluhan Binici, Tulika Mitra

Abstract: Machine learning pipelines for classification tasks often train a universal model to achieve accuracy across a broad range of classes. However, a typical user encounters only a limited selection of classes regularly. This disparity provides an opportunity to enhance computational efficiency by tailoring models to focus on user-specific classes. Existing works rely on unstructured pruning, which in… ▽ More Machine learning pipelines for classification tasks often train a universal model to achieve accuracy across a broad range of classes. However, a typical user encounters only a limited selection of classes regularly. This disparity provides an opportunity to enhance computational efficiency by tailoring models to focus on user-specific classes. Existing works rely on unstructured pruning, which introduces randomly distributed non-zero values in the model, making it unsuitable for hardware acceleration. Alternatively, some approaches employ structured pruning, such as channel pruning, but these tend to provide only minimal compression and may lead to reduced model accuracy. In this work, we propose CRISP, a novel pruning framework leveraging a hybrid structured sparsity pattern that combines both fine-grained N:M structured sparsity and coarse-grained block sparsity. Our pruning strategy is guided by a gradient-based class-aware saliency score, allowing us to retain weights crucial for user-specific classes. CRISP achieves high accuracy with minimal memory consumption for popular models like ResNet-50, VGG-16, and MobileNetV2 on ImageNet and CIFAR-100 datasets. Moreover, CRISP delivers up to 14$\times$ reduction in latency and energy consumption compared to existing pruning methods while maintaining comparable accuracy. Our code is available at https://github.com/shivmgg/CRISP/. △ Less

Submitted 18 March, 2024; v1 submitted 23 November, 2023; originally announced November 2023.

Comments: 6 pages, accepted in Design, Automation & Test in Europe Conference & Exhibition (DATE) 2024

Shedding the Bits: Pushing the Boundaries of Quantization with Minifloats on FPGAs

Authors: Shivam Aggarwal, Hans Jakob Damsgaard, Alessandro Pappalardo, Giuseppe Franco, Thomas B. Preußer, Michaela Blott, Tulika Mitra

Abstract: Post-training quantization (PTQ) is a powerful technique for model compression, reducing the numerical precision in neural networks without additional training overhead. Recent works have investigated adopting 8-bit floating-point formats(FP8) in the context of PTQ for model inference. However, floating-point formats smaller than 8 bits and their relative comparison in terms of accuracy-hardware c… ▽ More Post-training quantization (PTQ) is a powerful technique for model compression, reducing the numerical precision in neural networks without additional training overhead. Recent works have investigated adopting 8-bit floating-point formats(FP8) in the context of PTQ for model inference. However, floating-point formats smaller than 8 bits and their relative comparison in terms of accuracy-hardware cost with integers remains unexplored on FPGAs. In this work, we present minifloats, which are reduced-precision floating-point formats capable of further reducing the memory footprint, latency, and energy cost of a model while approaching full-precision model accuracy. We implement a custom FPGA-based multiply-accumulate operator library and explore the vast design space, comparing minifloat and integer representations across 3 to 8 bits for both weights and activations. We also examine the applicability of various integerbased quantization techniques to minifloats. Our experiments show that minifloats offer a promising alternative for emerging workloads such as vision transformers. △ Less

Submitted 5 July, 2024; v1 submitted 21 November, 2023; originally announced November 2023.

Comments: Accepted in FPL (International Conference on Field-Programmable Logic and Applications) 2024 conference. Revised with updated results

Multi-Purpose NLP Chatbot : Design, Methodology & Conclusion

Authors: Shivom Aggarwal, Shourya Mehra, Pritha Mitra

Abstract: With a major focus on its history, difficulties, and promise, this research paper provides a thorough analysis of the chatbot technology environment as it exists today. It provides a very flexible chatbot system that makes use of reinforcement learning strategies to improve user interactions and conversational experiences. Additionally, this system makes use of sentiment analysis and natural langu… ▽ More With a major focus on its history, difficulties, and promise, this research paper provides a thorough analysis of the chatbot technology environment as it exists today. It provides a very flexible chatbot system that makes use of reinforcement learning strategies to improve user interactions and conversational experiences. Additionally, this system makes use of sentiment analysis and natural language processing to determine user moods. The chatbot is a valuable tool across many fields thanks to its amazing characteristics, which include voice-to-voice conversation, multilingual support [12], advising skills, offline functioning, and quick help features. The complexity of chatbot technology development is also explored in this study, along with the causes that have propelled these developments and their far-reaching effects on a range of sectors. According to the study, three crucial elements are crucial: 1) Even without explicit profile information, the chatbot system is built to adeptly understand unique consumer preferences and fluctuating satisfaction levels. With the use of this capacity, user interactions are made to meet their wants and preferences. 2) Using a complex method that interlaces Multiview voice chat information, the chatbot may precisely simulate users' actual experiences. This aids in developing more genuine and interesting discussions. 3) The study presents an original method for improving the black-box deep learning models' capacity for prediction. This improvement is made possible by introducing dynamic satisfaction measurements that are theory-driven, which leads to more precise forecasts of consumer reaction. △ Less

Submitted 13 October, 2023; originally announced October 2023.

Comments: Multilingual , Voice Conversion , Emotion Recognition , Offline Service , Financial Advisor , Product Preference , Customer Reaction Prediction

Neuromorphic Hebbian learning with magnetic tunnel junction synapses

Authors: Peng Zhou, Alexander J. Edwards, Frederick B. Mancoff, Sanjeev Aggarwal, Stephen K. Heinrich-Barna, Joseph S. Friedman

Abstract: Neuromorphic computing aims to mimic both the function and structure of biological neural networks to provide artificial intelligence with extreme efficiency. Conventional approaches store synaptic weights in non-volatile memory devices with analog resistance states, permitting in-memory computation of neural network operations while avoiding the costs associated with transferring synaptic weights… ▽ More Neuromorphic computing aims to mimic both the function and structure of biological neural networks to provide artificial intelligence with extreme efficiency. Conventional approaches store synaptic weights in non-volatile memory devices with analog resistance states, permitting in-memory computation of neural network operations while avoiding the costs associated with transferring synaptic weights from a memory array. However, the use of analog resistance states for storing weights in neuromorphic systems is impeded by stochastic writing, weights drifting over time through stochastic processes, and limited endurance that reduces the precision of synapse weights. Here we propose and experimentally demonstrate neuromorphic networks that provide high-accuracy inference thanks to the binary resistance states of magnetic tunnel junctions (MTJs), while leveraging the analog nature of their stochastic spin-transfer torque (STT) switching for unsupervised Hebbian learning. We performed the first experimental demonstration of a neuromorphic network directly implemented with MTJ synapses, for both inference and spike-timing-dependent plasticity learning. We also demonstrated through simulation that the proposed system for unsupervised Hebbian learning with stochastic STT-MTJ synapses can achieve competitive accuracies for MNIST handwritten digit recognition. By appropriately applying neuromorphic principles through hardware-aware design, the proposed STT-MTJ neuromorphic learning networks provide a pathway toward artificial intelligence hardware that learns autonomously with extreme efficiency. △ Less

Submitted 21 August, 2023; originally announced August 2023.

SecureTrack- A contact tracing IoT platform for monitoring infectious diseases

Authors: Shobhit Aggarwal, Arnab Purkayastha

Abstract: The COVID-19 pandemic has highlighted the need for innovative solutions to monitor and control the spread of infectious diseases. With the potential for future pandemics and the risk of outbreaks particularly in academic institutions, there is a pressing need for effective approaches to monitor and manage such diseases. Contact tracing using Global Positioning Systems (GPS) has been found to be th… ▽ More The COVID-19 pandemic has highlighted the need for innovative solutions to monitor and control the spread of infectious diseases. With the potential for future pandemics and the risk of outbreaks particularly in academic institutions, there is a pressing need for effective approaches to monitor and manage such diseases. Contact tracing using Global Positioning Systems (GPS) has been found to be the most prevalent method to detect and tackle the extent of outbreaks during the pandemic. However, these services suffer from the inherent problems of infringement of data privacy that creates hindrance in adoption of the technology. Non-cellular wireless technologies on the other hand are well-suited to provide secure contact tracing methods. Such approaches integrated with the Internet of Things (IoT) have a great potential to aid in the fight against any type of infectious diseases. In response, we present a unique approach that utilizes an IoT based generic framework to identify individuals who may have been exposed to the virus, using contact tracing methods, without compromising the privacy aspect. We develop the architecture of our platform, including both the frontend and backend components, and demonstrate its effectiveness in identifying potential COVID-19 exposures (as a test case) through a proof-of-concept implementation. We also implement and verify a prototype of the device. Our framework is easily deployable and can be scaled up as needed with the existing infrastructure. △ Less

Submitted 18 July, 2023; originally announced July 2023.

Comments: 22 Pages, 8 figures, To be published in "The Global Interdisciplinary Green Cities Conference 2023 Business, Engineering, Art, Architecture, Design, Political Science, International Relations, Applied Science & Technology. "

Large Population Games on Constrained Unreliable Networks

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar

Abstract: This paper studies an $N$--agent cost-coupled game where the agents are connected via an unreliable capacity constrained network. Each agent receives state information over that network which loses packets with probability $p$. A Base station (BS) actively schedules agent communications over the network by minimizing a weighted Age of Information (WAoI) based cost function under a capacity limit… ▽ More This paper studies an $N$--agent cost-coupled game where the agents are connected via an unreliable capacity constrained network. Each agent receives state information over that network which loses packets with probability $p$. A Base station (BS) actively schedules agent communications over the network by minimizing a weighted Age of Information (WAoI) based cost function under a capacity limit $\mathcal{C} < N$ on the number of transmission attempts at each instant. Under a standard information structure, we show that the problem can be decoupled into a scheduling problem for the BS and a game problem for the $N$ agents. Since the scheduling problem is an NP hard combinatorics problem, we propose an approximately optimal solution which approaches the optimal solution as $N \rightarrow \infty$. In the process, we also provide some insights on the case without channel erasure. Next, to solve the large population game problem, we use the mean-field game framework to compute an approximate decentralized Nash equilibrium. Finally, we validate the theoretical results using a numerical example. △ Less

Submitted 16 March, 2023; originally announced March 2023.

Comments: Submitted to IEEE for possible publication

MABNet: Master Assistant Buddy Network with Hybrid Learning for Image Retrieval

Authors: Rohit Agarwal, Gyanendra Das, Saksham Aggarwal, Alexander Horsch, Dilip K. Prasad

Abstract: Image retrieval has garnered growing interest in recent times. The current approaches are either supervised or self-supervised. These methods do not exploit the benefits of hybrid learning using both supervision and self-supervision. We present a novel Master Assistant Buddy Network (MABNet) for image retrieval which incorporates both learning mechanisms. MABNet consists of master and assistant bl… ▽ More Image retrieval has garnered growing interest in recent times. The current approaches are either supervised or self-supervised. These methods do not exploit the benefits of hybrid learning using both supervision and self-supervision. We present a novel Master Assistant Buddy Network (MABNet) for image retrieval which incorporates both learning mechanisms. MABNet consists of master and assistant blocks, both learning independently through supervision and collectively via self-supervision. The master guides the assistant by providing its knowledge base as a reference for self-supervision and the assistant reports its knowledge back to the master by weight transfer. We perform extensive experiments on public datasets with and without post-processing. △ Less

Submitted 6 March, 2023; originally announced March 2023.

Comments: Accepted at International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2023

Mimetic Muscle Rehabilitation Analysis Using Clustering of Low Dimensional 3D Kinect Data

Authors: Sumit Kumar Vishwakarma, Sanjeev Kumar, Shrey Aggarwal, Jan Mareš

Abstract: Facial nerve paresis is a severe complication that arises post-head and neck surgery; This results in articulation problems, facial asymmetry, and severe problems in non-verbal communication. To overcome the side effects of post-surgery facial paralysis, rehabilitation requires which last for several weeks. This paper discusses an unsupervised approach to rehabilitating patients who have temporary… ▽ More Facial nerve paresis is a severe complication that arises post-head and neck surgery; This results in articulation problems, facial asymmetry, and severe problems in non-verbal communication. To overcome the side effects of post-surgery facial paralysis, rehabilitation requires which last for several weeks. This paper discusses an unsupervised approach to rehabilitating patients who have temporary facial paralysis due to damage in mimetic muscles. The work aims to make the rehabilitation process objective compared to the current subjective approach, such as House-Brackmann (HB) scale. Also, the approach will assist clinicians by reducing their workload in assessing the improvement during rehabilitation. This paper focuses on the clustering approach to monitor the rehabilitation process. We compare the results obtained from different clustering algorithms on various forms of the same data set, namely dynamic form, data expressed as functional data using B-spline basis expansion, and by finding the functional principal components of the functional data. The study contains data set of 85 distinct patients with 120 measurements obtained using a Kinect stereo-vision camera. The method distinguish effectively between patients with the least and greatest degree of facial paralysis, however patients with adjacent degrees of paralysis provide some challenges. In addition, we compared the cluster results to the HB scale outputs. △ Less

Submitted 15 February, 2023; originally announced February 2023.

Towards Precision in Appearance-based Gaze Estimation in the Wild

Authors: Murthy L. R. D., Abhishek Mukhopadhyay, Shambhavi Aggarwal, Ketan Anand, Pradipta Biswas

Abstract: Appearance-based gaze estimation systems have shown great progress recently, yet the performance of these techniques depend on the datasets used for training. Most of the existing gaze estimation datasets setup in interactive settings were recorded in laboratory conditions and those recorded in the wild conditions display limited head pose and illumination variations. Further, we observed little a… ▽ More Appearance-based gaze estimation systems have shown great progress recently, yet the performance of these techniques depend on the datasets used for training. Most of the existing gaze estimation datasets setup in interactive settings were recorded in laboratory conditions and those recorded in the wild conditions display limited head pose and illumination variations. Further, we observed little attention so far towards precision evaluations of existing gaze estimation approaches. In this work, we present a large gaze estimation dataset, PARKS-Gaze, with wider head pose and illumination variation and with multiple samples for a single Point of Gaze (PoG). The dataset contains 974 minutes of data from 28 participants with a head pose range of 60 degrees in both yaw and pitch directions. Our within-dataset and cross-dataset evaluations and precision evaluations indicate that the proposed dataset is more challenging and enable models to generalize on unseen participants better than the existing in-the-wild datasets. The project page can be accessed here: https://github.com/lrdmurthy/PARKS-Gaze △ Less

Submitted 13 February, 2023; v1 submitted 5 February, 2023; originally announced February 2023.

On Designing Light-Weight Object Trackers through Network Pruning: Use CNNs or Transformers?

Authors: Saksham Aggarwal, Taneesh Gupta, Pawan Kumar Sahu, Arnav Chavan, Rishabh Tiwari, Dilip K. Prasad, Deepak K. Gupta

Abstract: Object trackers deployed on low-power devices need to be light-weight, however, most of the current state-of-the-art (SOTA) methods rely on using compute-heavy backbones built using CNNs or transformers. Large sizes of such models do not allow their deployment in low-power conditions and designing compressed variants of large tracking models is of great importance. This paper demonstrates how high… ▽ More Object trackers deployed on low-power devices need to be light-weight, however, most of the current state-of-the-art (SOTA) methods rely on using compute-heavy backbones built using CNNs or transformers. Large sizes of such models do not allow their deployment in low-power conditions and designing compressed variants of large tracking models is of great importance. This paper demonstrates how highly compressed light-weight object trackers can be designed using neural architectural pruning of large CNN and transformer based trackers. Further, a comparative study on architectural choices best suited to design light-weight trackers is provided. A comparison between SOTA trackers using CNNs, transformers as well as the combination of the two is presented to study their stability at various compression ratios. Finally results for extreme pruning scenarios going as low as 1% in some cases are shown to study the limits of network pruning in object tracking. This work provides deeper insights into designing highly efficient trackers from existing SOTA methods. △ Less

Submitted 26 March, 2023; v1 submitted 24 November, 2022; originally announced November 2022.

Comments: Accepted at IEEE ICASSP 2023

Partial Binarization of Neural Networks for Budget-Aware Efficient Learning

Authors: Udbhav Bamba, Neeraj Anand, Saksham Aggarwal, Dilip K. Prasad, Deepak K. Gupta

Abstract: Binarization is a powerful compression technique for neural networks, significantly reducing FLOPs, but often results in a significant drop in model performance. To address this issue, partial binarization techniques have been developed, but a systematic approach to mixing binary and full-precision parameters in a single network is still lacking. In this paper, we propose a controlled approach to… ▽ More Binarization is a powerful compression technique for neural networks, significantly reducing FLOPs, but often results in a significant drop in model performance. To address this issue, partial binarization techniques have been developed, but a systematic approach to mixing binary and full-precision parameters in a single network is still lacking. In this paper, we propose a controlled approach to partial binarization, creating a budgeted binary neural network (B2NN) with our MixBin strategy. This method optimizes the mixing of binary and full-precision components, allowing for explicit selection of the fraction of the network to remain binary. Our experiments show that B2NNs created using MixBin outperform those from random or iterative searches and state-of-the-art layer selection methods by up to 3% on the ImageNet-1K dataset. We also show that B2NNs outperform the structured pruning baseline by approximately 23% at the extreme FLOP budget of 15%, and perform well in object tracking, with up to a 12.4% relative improvement over other baselines. Additionally, we demonstrate that B2NNs developed by MixBin can be transferred across datasets, with some cases showing improved performance over directly applying MixBin on the downstream data. △ Less

Submitted 8 November, 2023; v1 submitted 12 November, 2022; originally announced November 2022.

Comments: Accepted at WACV 2023 Conference

Weighted Age of Information based Scheduling for Large Population Games on Networks

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Melih Bastopcu, Tamer Başar

Abstract: In this paper, we consider a discrete-time multi-agent system involving $N$ cost-coupled networked rational agents solving a consensus problem and a central Base Station (BS), scheduling agent communications over a network. Due to a hard bandwidth constraint on the number of transmissions through the network, at most $R_d < N$ agents can concurrently access their state information through the netw… ▽ More In this paper, we consider a discrete-time multi-agent system involving $N$ cost-coupled networked rational agents solving a consensus problem and a central Base Station (BS), scheduling agent communications over a network. Due to a hard bandwidth constraint on the number of transmissions through the network, at most $R_d < N$ agents can concurrently access their state information through the network. Under standard assumptions on the information structure of the agents and the BS, we first show that the control actions of the agents are free of any dual effect, allowing for separation between estimation and control problems at each agent. Next, we propose a weighted age of information (WAoI) metric for the scheduling problem of the BS, where the weights depend on the estimation error of the agents. The BS aims to find the optimum scheduling policy that minimizes the WAoI, subject to the hard bandwidth constraint. Since this problem is NP hard, we first relax the hard constraint to a soft update rate constraint, and then compute an optimal policy for the relaxed problem by reformulating it into a Markov Decision Process (MDP). This then inspires a sub-optimal policy for the bandwidth constrained problem, which is shown to approach the optimal policy as $N \rightarrow \infty$. Next, we solve the consensus problem using the mean-field game framework wherein we first design decentralized control policies for a limiting case of the $N$-agent system (as $N \rightarrow \infty$). By explicitly constructing the mean-field system, we prove the existence and uniqueness of the mean-field equilibrium. Consequently, we show that the obtained equilibrium policies constitute an $εいぷしろん$-Nash equilibrium for the finite agent system. Finally, we validate the performance of both the scheduling and the control policies through numerical simulations. △ Less

Submitted 26 December, 2022; v1 submitted 26 September, 2022; originally announced September 2022.

Comments: This work has been submitted to IEEE for possible publication

GPTs at Factify 2022: Prompt Aided Fact-Verification

Authors: Pawan Kumar Sahu, Saksham Aggarwal, Taneesh Gupta, Gyanendra Das

Abstract: One of the most pressing societal issues is the fight against false news. The false claims, as difficult as they are to expose, create a lot of damage. To tackle the problem, fact verification becomes crucial and thus has been a topic of interest among diverse research communities. Using only the textual form of data we propose our solution to the problem and achieve competitive results with other… ▽ More One of the most pressing societal issues is the fight against false news. The false claims, as difficult as they are to expose, create a lot of damage. To tackle the problem, fact verification becomes crucial and thus has been a topic of interest among diverse research communities. Using only the textual form of data we propose our solution to the problem and achieve competitive results with other approaches. We present our solution based on two approaches - PLM (pre-trained language model) based method and Prompt based method. The PLM-based approach uses the traditional supervised learning, where the model is trained to take 'x' as input and output prediction 'y' as P(y|x). Whereas, Prompt-based learning reflects the idea to design input to fit the model such that the original objective may be re-framed as a problem of (masked) language modeling. We may further stimulate the rich knowledge provided by PLMs to better serve downstream tasks by employing extra prompts to fine-tune PLMs. Our experiments showed that the proposed method performs better than just fine-tuning PLMs. We achieved an F1 score of 0.6946 on the FACTIFY dataset and a 7th position on the competition leader-board. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Comments: Accepted in AAAI'22: First Workshop on Multimodal Fact-Checking and Hate Speech Detection, Februrary 22 - March 1, 2022,Vancouver, BC, Canada

How does a Rational Agent Act in an Epidemic?

Authors: S. Yagiz Olmez, Shubham Aggarwal, Jin Won Kim, Erik Miehling, Tamer Başar, Matthew West, Prashant G. Mehta

Abstract: Evolution of disease in a large population is a function of the top-down policy measures from a centralized planner, as well as the self-interested decisions (to be socially active) of individual agents in a large heterogeneous population. This paper is concerned with understanding the latter based on a mean-field type optimal control model. Specifically, the model is used to investigate the role… ▽ More Evolution of disease in a large population is a function of the top-down policy measures from a centralized planner, as well as the self-interested decisions (to be socially active) of individual agents in a large heterogeneous population. This paper is concerned with understanding the latter based on a mean-field type optimal control model. Specifically, the model is used to investigate the role of partial information on an agent's decision-making, and study the impact of such decisions by a large number of agents on the spread of the virus in the population. The motivation comes from the presymptomatic and asymptomatic spread of the COVID-19 virus where an agent unwittingly spreads the virus. We show that even in a setting with fully rational agents, limited information on the viral state can result in an epidemic growth. △ Less

Submitted 5 June, 2022; originally announced June 2022.

Comments: arXiv admin note: text overlap with arXiv:2111.10422

Lower Bounds for Restricted Schemes in the Two-Adaptive Bitprobe Model

Authors: Sreshth Aggarwal, Deepanjan Kesh, Divyam Singal

Abstract: In the adaptive bitprobe model answering membership queries in two bitprobes, we consider the class of restricted schemes as introduced by Kesh and Sharma (Discrete Applied Mathematics 2021). In that paper, the authors showed that such restricted schemes storing subsets of size 2 require $Ωおめが(m^\frac{2}{3})$ space. In this paper, we generalise the result to arbitrary subsets of size $n$, and prove t… ▽ More In the adaptive bitprobe model answering membership queries in two bitprobes, we consider the class of restricted schemes as introduced by Kesh and Sharma (Discrete Applied Mathematics 2021). In that paper, the authors showed that such restricted schemes storing subsets of size 2 require $Ωおめが(m^\frac{2}{3})$ space. In this paper, we generalise the result to arbitrary subsets of size $n$, and prove that the space required for such restricted schemes will be $Ωおめが(\left(\frac{m}{n}\right)^{1 - \frac{1}{\lfloor n / 4 \rfloor + 2}})$. △ Less

Submitted 8 April, 2022; v1 submitted 7 April, 2022; originally announced April 2022.

Comments: 19 pages, 1 figure, IWOCA2022, full version of paper

Linear Quadratic Mean-Field Games with Communication Constraints

Authors: Shubham Aggarwal, Muhammad Aneeq uz Zaman, Tamer Başar

Abstract: In this paper, we study a large population game with heterogeneous dynamics and cost functions solving a consensus problem. Moreover, the agents have communication constraints which appear as: (1) an Additive-White Gaussian Noise (AWGN) channel, and (2) asynchronous data transmission via a fixed scheduling policy. Since the complexity of solving the game increases with the number of agents, we use… ▽ More In this paper, we study a large population game with heterogeneous dynamics and cost functions solving a consensus problem. Moreover, the agents have communication constraints which appear as: (1) an Additive-White Gaussian Noise (AWGN) channel, and (2) asynchronous data transmission via a fixed scheduling policy. Since the complexity of solving the game increases with the number of agents, we use the Mean-Field Game paradigm to solve it. Under standard assumptions on the information structure of the agents, we prove that the control of the agent in the MFG setting is free of the dual effect. This allows us to obtain an equilibrium control policy for the generic agent, which is a function of only the local observation of the agent. Furthermore, the equilibrium mean-field trajectory is shown to follow linear dynamics, hence making it computable. We show that in the finite population game, the equilibrium control policy prescribed by the MFG analysis constitutes an $εいぷしろん$-Nash equilibrium, where $εいぷしろん$ tends to zero as the number of agents goes to infinity. The paper is concluded with simulations demonstrating the performance of the equilibrium control policy. △ Less

Submitted 25 August, 2022; v1 submitted 10 March, 2022; originally announced March 2022.

Comments: Accepted in American Control Conference 2022

Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay

Authors: Kuluhan Binici, Shivam Aggarwal, Nam Trung Pham, Karianto Leman, Tulika Mitra

Abstract: Data-Free Knowledge Distillation (KD) allows knowledge transfer from a trained neural network (teacher) to a more compact one (student) in the absence of original training data. Existing works use a validation set to monitor the accuracy of the student over real data and report the highest performance throughout the entire process. However, validation data may not be available at distillation time… ▽ More Data-Free Knowledge Distillation (KD) allows knowledge transfer from a trained neural network (teacher) to a more compact one (student) in the absence of original training data. Existing works use a validation set to monitor the accuracy of the student over real data and report the highest performance throughout the entire process. However, validation data may not be available at distillation time either, making it infeasible to record the student snapshot that achieved the peak accuracy. Therefore, a practical data-free KD method should be robust and ideally provide monotonically increasing student accuracy during distillation. This is challenging because the student experiences knowledge degradation due to the distribution shift of the synthetic data. A straightforward approach to overcome this issue is to store and rehearse the generated samples periodically, which increases the memory footprint and creates privacy concerns. We propose to model the distribution of the previously observed synthetic samples with a generative network. In particular, we design a Variational Autoencoder (VAE) with a training objective that is customized to learn the synthetic data representations optimally. The student is rehearsed by the generative pseudo replay technique, with samples produced by the VAE. Hence knowledge degradation can be prevented without storing any samples. Experiments on image classification benchmarks show that our method optimizes the expected value of the distilled model accuracy while eliminating the large memory overhead incurred by the sample-storing methods. △ Less

Submitted 21 March, 2022; v1 submitted 9 January, 2022; originally announced January 2022.

Comments: Accepted by the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22)

Experimental Demonstration of Neuromorphic Network with STT MTJ Synapses

Authors: Peng Zhou, Alexander J. Edwards, Fred B. Mancoff, Dimitri Houssameddine, Sanjeev Aggarwal, Joseph S. Friedman

Abstract: We present the first experimental demonstration of a neuromorphic network with magnetic tunnel junction (MTJ) synapses, which performs image recognition via vector-matrix multiplication. We also simulate a large MTJ network performing MNIST handwritten digit recognition, demonstrating that MTJ crossbars can match memristor accuracy while providing increased precision, stability, and endurance. We present the first experimental demonstration of a neuromorphic network with magnetic tunnel junction (MTJ) synapses, which performs image recognition via vector-matrix multiplication. We also simulate a large MTJ network performing MNIST handwritten digit recognition, demonstrating that MTJ crossbars can match memristor accuracy while providing increased precision, stability, and endurance. △ Less

Submitted 9 December, 2021; originally announced December 2021.

Modeling Presymptomatic Spread in Epidemics via Mean-Field Games

Authors: S. Yagiz Olmez, Shubham Aggarwal, Jin Won Kim, Erik Miehling, Tamer Başar, Matthew West, Prashant G. Mehta

Abstract: This paper is concerned with developing mean-field game models for the evolution of epidemics. Specifically, an agent's decision -- to be socially active in the midst of an epidemic -- is modeled as a mean-field game with health-related costs and activity-related rewards. By considering the fully and partially observed versions of this problem, the role of information in guiding an agent's rationa… ▽ More This paper is concerned with developing mean-field game models for the evolution of epidemics. Specifically, an agent's decision -- to be socially active in the midst of an epidemic -- is modeled as a mean-field game with health-related costs and activity-related rewards. By considering the fully and partially observed versions of this problem, the role of information in guiding an agent's rational decision is highlighted. The main contributions of the paper are to derive the equations for the mean-field game in both fully and partially observed settings of the problem, to present a complete analysis of the fully observed case, and to present some analytical results for the partially observed case. △ Less

Submitted 19 November, 2021; originally announced November 2021.

Writers Gonna Wait: The Effectiveness of Notifications to Initiate Aversive Action in Writing Procrastination

Authors: Chatchai Wangwiwattana, Sunjoli Aggarwal, Eric C. Larson

Abstract: This paper evaluates the use of notifications to reduce aversive-task-procrastination by helping initiate action. Specifically, we focus on aversion to graded writing tasks. We evaluate software designs commonly used by behavior change applications, such as goal setting and action support systems. We conduct a two-phase control trial experiment with 21 college students tasked to write two 3000-wor… ▽ More This paper evaluates the use of notifications to reduce aversive-task-procrastination by helping initiate action. Specifically, we focus on aversion to graded writing tasks. We evaluate software designs commonly used by behavior change applications, such as goal setting and action support systems. We conduct a two-phase control trial experiment with 21 college students tasked to write two 3000-word writing assignments (14 students fully completed the experiment). Participants use a customized text editor designed to continuously collect writing behavior. The results from the study reveal that notifications have minimal effect in encouraging users to get started. They can also increase negative effects on participants. Other techniques, such as eliminating distraction and showing simple writing statistics, yield higher satisfaction among participants as they complete the writing task. Furthermore, the incorporation of text mining decreases aversion to the task and helps participants overcome writer's block. Finally, we discuss lessons learned from our evaluation that help quantify the difficulty of behavior change for writing procrastination, with emphasis on goals for the HCI community. △ Less

Submitted 25 January, 2021; originally announced January 2021.

ACM Class: H.5.m

Evaluating Empathetic Chatbots in Customer Service Settings

Authors: Akshay Agarwal, Shashank Maiya, Sonu Aggarwal

Abstract: Customer service is a setting that calls for empathy in live human agent responses. Recent advances have demonstrated how open-domain chatbots can be trained to demonstrate empathy when responding to live human utterances. We show that a blended skills chatbot model that responds to customer queries is more likely to resemble actual human agent response if it is trained to recognize emotion and ex… ▽ More Customer service is a setting that calls for empathy in live human agent responses. Recent advances have demonstrated how open-domain chatbots can be trained to demonstrate empathy when responding to live human utterances. We show that a blended skills chatbot model that responds to customer queries is more likely to resemble actual human agent response if it is trained to recognize emotion and exhibit appropriate empathy, than a model without such training. For our analysis, we leverage a Twitter customer service dataset containing several million customer<->agent dialog examples in customer service contexts from 20 well-known brands. △ Less

Submitted 4 January, 2021; originally announced January 2021.

Comments: 8 pages, 7 figures

Get It Scored Using AutoSAS -- An Automated System for Scoring Short Answers

Authors: Yaman Kumar, Swati Aggarwal, Debanjan Mahata, Rajiv Ratn Shah, Ponnurangam Kumaraguru, Roger Zimmermann

Abstract: In the era of MOOCs, online exams are taken by millions of candidates, where scoring short answers is an integral part. It becomes intractable to evaluate them by human graders. Thus, a generic automated system capable of grading these responses should be designed and deployed. In this paper, we present a fast, scalable, and accurate approach towards automated Short Answer Scoring (SAS). We propos… ▽ More In the era of MOOCs, online exams are taken by millions of candidates, where scoring short answers is an integral part. It becomes intractable to evaluate them by human graders. Thus, a generic automated system capable of grading these responses should be designed and deployed. In this paper, we present a fast, scalable, and accurate approach towards automated Short Answer Scoring (SAS). We propose and explain the design and development of a system for SAS, namely AutoSAS. Given a question along with its graded samples, AutoSAS can learn to grade that prompt successfully. This paper further lays down the features such as lexical diversity, Word2Vec, prompt, and content overlap that plays a pivotal role in building our proposed model. We also present a methodology for indicating the factors responsible for scoring an answer. The trained model is evaluated on an extensively used public dataset, namely Automated Student Assessment Prize Short Answer Scoring (ASAP-SAS). AutoSAS shows state-of-the-art performance and achieves better results by over 8% in some of the question prompts as measured by Quadratic Weighted Kappa (QWK), showing performance comparable to humans. △ Less

Submitted 21 December, 2020; originally announced December 2020.

Goal-driven Command Recommendations for Analysts

Authors: Samarth Aggarwal, Rohin Garg, Abhilasha Sancheti, Bhanu Prakash Reddy Guda, Iftikhar Ahamath Burhanuddin

Abstract: Recent times have seen data analytics software applications become an integral part of the decision-making process of analysts. The users of these software applications generate a vast amount of unstructured log data. These logs contain clues to the user's goals, which traditional recommender systems may find difficult to model implicitly from the log data. With this assumption, we would like to a… ▽ More Recent times have seen data analytics software applications become an integral part of the decision-making process of analysts. The users of these software applications generate a vast amount of unstructured log data. These logs contain clues to the user's goals, which traditional recommender systems may find difficult to model implicitly from the log data. With this assumption, we would like to assist the analytics process of a user through command recommendations. We categorize the commands into software and data categories based on their purpose to fulfill the task at hand. On the premise that the sequence of commands leading up to a data command is a good predictor of the latter, we design, develop, and validate various sequence modeling techniques. In this paper, we propose a framework to provide goal-driven data command recommendations to the user by leveraging unstructured logs. We use the log data of a web-based analytics software to train our neural network models and quantify their performance, in comparison to relevant and competitive baselines. We propose a custom loss function to tailor the recommended data commands according to the goal information provided exogenously. We also propose an evaluation metric that captures the degree of goal orientation of the recommendations. We demonstrate the promise of our approach by evaluating the models with the proposed metric and showcasing the robustness of our models in the case of adversarial examples, where the user activity is misaligned with selected goal, through offline evaluation. △ Less

Submitted 12 November, 2020; originally announced November 2020.

Comments: 14th ACM Conference on Recommender Systems (RecSys 2020)

OpenIE6: Iterative Grid Labeling and Coordination Analysis for Open Information Extraction

Authors: Keshav Kolluru, Vaibhav Adlakha, Samarth Aggarwal, Mausam, Soumen Chakrabarti

Abstract: A recent state-of-the-art neural open information extraction (OpenIE) system generates extractions iteratively, requiring repeated encoding of partial outputs. This comes at a significant computational cost. On the other hand, sequence labeling approaches for OpenIE are much faster, but worse in extraction quality. In this paper, we bridge this trade-off by presenting an iterative labeling-based s… ▽ More A recent state-of-the-art neural open information extraction (OpenIE) system generates extractions iteratively, requiring repeated encoding of partial outputs. This comes at a significant computational cost. On the other hand, sequence labeling approaches for OpenIE are much faster, but worse in extraction quality. In this paper, we bridge this trade-off by presenting an iterative labeling-based system that establishes a new state of the art for OpenIE, while extracting 10x faster. This is achieved through a novel Iterative Grid Labeling (IGL) architecture, which treats OpenIE as a 2-D grid labeling task. We improve its performance further by applying coverage (soft) constraints on the grid at training time. Moreover, on observing that the best OpenIE systems falter at handling coordination structures, our OpenIE system also incorporates a new coordination analyzer built with the same IGL architecture. This IGL based coordination analyzer helps our OpenIE system handle complicated coordination structures, while also establishing a new state of the art on the task of coordination analysis, with a 12.3 pts improvement in F1 over previous analyzers. Our OpenIE system, OpenIE6, beats the previous systems by as much as 4 pts in F1, while being much faster. △ Less

Submitted 7 October, 2020; originally announced October 2020.

Comments: EMNLP 2020 (Long)

Visual Exploration and Knowledge Discovery from Biomedical Dark Data

Authors: Shashwat Aggarwal, Ramesh Singh

Abstract: Data visualization techniques proffer efficient means to organize and present data in graphically appealing formats, which not only speeds up the process of decision making and pattern recognition but also enables decision-makers to fully understand data insights and make informed decisions. Over time, with the rise in technological and computational resources, there has been an exponential increa… ▽ More Data visualization techniques proffer efficient means to organize and present data in graphically appealing formats, which not only speeds up the process of decision making and pattern recognition but also enables decision-makers to fully understand data insights and make informed decisions. Over time, with the rise in technological and computational resources, there has been an exponential increase in the world's scientific knowledge. However, most of it lacks structure and cannot be easily categorized and imported into regular databases. This type of data is often termed as Dark Data. Data visualization techniques provide a promising solution to explore such data by allowing quick comprehension of information, the discovery of emerging trends, identification of relationships and patterns, etc. In this empirical research study, we use the rich corpus of PubMed comprising of more than 30 million citations from biomedical literature to visually explore and understand the underlying key-insights using various information visualization techniques. We employ a natural language processing based pipeline to discover knowledge out of the biomedical dark data. The pipeline comprises of different lexical analysis techniques like Topic Modeling to extract inherent topics and major focus areas, Network Graphs to study the relationships between various entities like scientific documents and journals, researchers, and, keywords and terms, etc. With this analytical research, we aim to proffer a potential solution to overcome the problem of analyzing overwhelming amounts of information and diminish the limitation of human cognition and perception in handling and examining such large volumes of data. △ Less

Submitted 28 September, 2020; originally announced September 2020.

Metaphor Detection using Deep Contextualized Word Embeddings

Authors: Shashwat Aggarwal, Ramesh Singh

Abstract: Metaphors are ubiquitous in natural language, and their detection plays an essential role in many natural language processing tasks, such as language understanding, sentiment analysis, etc. Most existing approaches for metaphor detection rely on complex, hand-crafted and fine-tuned feature pipelines, which greatly limit their applicability. In this work, we present an end-to-end method composed of… ▽ More Metaphors are ubiquitous in natural language, and their detection plays an essential role in many natural language processing tasks, such as language understanding, sentiment analysis, etc. Most existing approaches for metaphor detection rely on complex, hand-crafted and fine-tuned feature pipelines, which greatly limit their applicability. In this work, we present an end-to-end method composed of deep contextualized word embeddings, bidirectional LSTMs and multi-head attention mechanism to address the task of automatic metaphor detection. Our method, unlike many other existing approaches, requires only the raw text sequences as input features to detect the metaphoricity of a phrase. We compare the performance of our method against the existing baselines on two benchmark datasets, TroFi, and MOH-X respectively. Experimental evaluations confirm the effectiveness of our approach. △ Less

Submitted 26 September, 2020; originally announced September 2020.

Password Guessers Under a Microscope: An In-Depth Analysis to Inform Deployments

Authors: Zach Parish, Connor Cushing, Shourya Aggarwal, Amirali Salehi-Abari, Julie Thorpe

Abstract: Password guessers are instrumental for assessing the strength of passwords. Despite their diversity and abundance, little is known about how different guessers compare to each other. We perform in-depth analyses and comparisons of the guessing abilities and behavior of password guessers. To extend analyses beyond number of passwords cracked, we devise an analytical framework to compare the types o… ▽ More Password guessers are instrumental for assessing the strength of passwords. Despite their diversity and abundance, little is known about how different guessers compare to each other. We perform in-depth analyses and comparisons of the guessing abilities and behavior of password guessers. To extend analyses beyond number of passwords cracked, we devise an analytical framework to compare the types of passwords that guessers generate under various conditions (e.g., limited training data, limited number of guesses, and dissimilar training and target data). Our results show that guessers often produce dissimilar guesses, even when trained on the same data. We leverage this result to show that combinations of computationally-cheap guessers are as effective as computationally intensive guessers, but more efficient. Our insights allow us to provide a concrete set of recommendations for system administrators when performing password checking. △ Less

Submitted 20 February, 2021; v1 submitted 18 August, 2020; originally announced August 2020.

Comments: 14 pages, 20 individual figures including 9 scale bars; clarified that default settings were used for the guessing tools; modified table formats

"Notic My Speech" -- Blending Speech Patterns With Multimedia

Authors: Dhruva Sahrawat, Yaman Kumar, Shashwat Aggarwal, Yifang Yin, Rajiv Ratn Shah, Roger Zimmermann

Abstract: Speech as a natural signal is composed of three parts - visemes (visual part of speech), phonemes (spoken part of speech), and language (the imposed structure). However, video as a medium for the delivery of speech and a multimedia construct has mostly ignored the cognitive aspects of speech delivery. For example, video applications like transcoding and compression have till now ignored the fact h… ▽ More Speech as a natural signal is composed of three parts - visemes (visual part of speech), phonemes (spoken part of speech), and language (the imposed structure). However, video as a medium for the delivery of speech and a multimedia construct has mostly ignored the cognitive aspects of speech delivery. For example, video applications like transcoding and compression have till now ignored the fact how speech is delivered and heard. To close the gap between speech understanding and multimedia video applications, in this paper, we show the initial experiments by modelling the perception on visual speech and showing its use case on video compression. On the other hand, in the visual speech recognition domain, existing studies have mostly modeled it as a classification problem, while ignoring the correlations between views, phonemes, visemes, and speech perception. This results in solutions which are further away from how human perception works. To bridge this gap, we propose a view-temporal attention mechanism to model both the view dependence and the visemic importance in speech recognition and understanding. We conduct experiments on three public visual speech recognition datasets. The experimental results show that our proposed method outperformed the existing work by 4.99% in terms of the viseme error rate. Moreover, we show that there is a strong correlation between our model's understanding of multi-view speech and the human perception. This characteristic benefits downstream applications such as video compression and streaming where a significant number of less important frames can be compressed or eliminated while being able to maximally preserve human speech understanding with good user experience. △ Less

Submitted 12 June, 2020; originally announced June 2020.

Comments: Under Review

IMoJIE: Iterative Memory-Based Joint Open Information Extraction

Authors: Keshav Kolluru, Samarth Aggarwal, Vipul Rathore, Mausam, Soumen Chakrabarti

Abstract: While traditional systems for Open Information Extraction were statistical and rule-based, recently neural models have been introduced for the task. Our work builds upon CopyAttention, a sequence generation OpenIE model (Cui et. al., 2018). Our analysis reveals that CopyAttention produces a constant number of extractions per sentence, and its extracted tuples often express redundant information.… ▽ More While traditional systems for Open Information Extraction were statistical and rule-based, recently neural models have been introduced for the task. Our work builds upon CopyAttention, a sequence generation OpenIE model (Cui et. al., 2018). Our analysis reveals that CopyAttention produces a constant number of extractions per sentence, and its extracted tuples often express redundant information. We present IMoJIE, an extension to CopyAttention, which produces the next extraction conditioned on all previously extracted tuples. This approach overcomes both shortcomings of CopyAttention, resulting in a variable number of diverse extractions per sentence. We train IMoJIE on training data bootstrapped from extractions of several non-neural systems, which have been automatically filtered to reduce redundancy and noise. IMoJIE outperforms CopyAttention by about 18 F1 pts, and a BERT-based strong baseline by 2 F1 pts, establishing a new state of the art for the task. △ Less

Submitted 17 May, 2020; originally announced May 2020.

Journal ref: ACL 2020, Long paper

Synthetic Video Generation for Robust Hand Gesture Recognition in Augmented Reality Applications

Authors: Varun Jain, Shivam Aggarwal, Suril Mehta, Ramya Hebbalaguppe

Abstract: Hand gestures are a natural means of interaction in Augmented Reality and Virtual Reality (AR/VR) applications. Recently, there has been an increased focus on removing the dependence of accurate hand gesture recognition on complex sensor setup found in expensive proprietary devices such as the Microsoft HoloLens, Daqri and Meta Glasses. Most such solutions either rely on multi-modal sensor data or… ▽ More Hand gestures are a natural means of interaction in Augmented Reality and Virtual Reality (AR/VR) applications. Recently, there has been an increased focus on removing the dependence of accurate hand gesture recognition on complex sensor setup found in expensive proprietary devices such as the Microsoft HoloLens, Daqri and Meta Glasses. Most such solutions either rely on multi-modal sensor data or deep neural networks that can benefit greatly from abundance of labelled data. Datasets are an integral part of any deep learning based research. They have been the principal reason for the substantial progress in this field, both, in terms of providing enough data for the training of these models, and, for benchmarking competing algorithms. However, it is becoming increasingly difficult to generate enough labelled data for complex tasks such as hand gesture recognition. The goal of this work is to introduce a framework capable of generating photo-realistic videos that have labelled hand bounding box and fingertip that can help in designing, training, and benchmarking models for hand-gesture recognition in AR/VR applications. We demonstrate the efficacy of our framework in generating videos with diverse backgrounds. △ Less

Submitted 5 December, 2019; v1 submitted 4 November, 2019; originally announced November 2019.

Comments: Presented at the ICCV 2019 Workshop: The 5th International Workshop on Observing And Understanding Hands In Action

BHAAV- A Text Corpus for Emotion Analysis from Hindi Stories

Authors: Yaman Kumar, Debanjan Mahata, Sagar Aggarwal, Anmol Chugh, Rajat Maheshwari, Rajiv Ratn Shah

Abstract: In this paper, we introduce the first and largest Hindi text corpus, named BHAAV, which means emotions in Hindi, for analyzing emotions that a writer expresses through his characters in a story, as perceived by a narrator/reader. The corpus consists of 20,304 sentences collected from 230 different short stories spanning across 18 genres such as Inspirational and Mystery. Each sentence has been ann… ▽ More In this paper, we introduce the first and largest Hindi text corpus, named BHAAV, which means emotions in Hindi, for analyzing emotions that a writer expresses through his characters in a story, as perceived by a narrator/reader. The corpus consists of 20,304 sentences collected from 230 different short stories spanning across 18 genres such as Inspirational and Mystery. Each sentence has been annotated into one of the five emotion categories - anger, joy, suspense, sad, and neutral, by three native Hindi speakers with at least ten years of formal education in Hindi. We also discuss challenges in the annotation of low resource languages such as Hindi, and discuss the scope of the proposed corpus along with its possible uses. We also provide a detailed analysis of the dataset and train strong baseline classifiers reporting their performances. △ Less

Submitted 9 October, 2019; originally announced October 2019.

Fine-grained Apparel Classification and Retrieval without rich annotations

Authors: Aniket Bhatnagar, Sanchit Aggarwal

Abstract: The ability to correctly classify and retrieve apparel images has a variety of applications important to e-commerce, online advertising and internet search. In this work, we propose a robust framework for fine-grained apparel classification, in-shop and cross-domain retrieval which eliminates the requirement of rich annotations like bounding boxes and human-joints or clothing landmarks, and traini… ▽ More The ability to correctly classify and retrieve apparel images has a variety of applications important to e-commerce, online advertising and internet search. In this work, we propose a robust framework for fine-grained apparel classification, in-shop and cross-domain retrieval which eliminates the requirement of rich annotations like bounding boxes and human-joints or clothing landmarks, and training of bounding box/ key-landmark detector for the same. Factors such as subtle appearance differences, variations in human poses, different shooting angles, apparel deformations, and self-occlusion add to the challenges in classification and retrieval of apparel items. Cross-domain retrieval is even harder due to the presence of large variation between online shopping images, usually taken in ideal lighting, pose, positive angle and clean background as compared with street photos captured by users in complicated conditions with poor lighting and cluttered scenes. Our framework uses compact bilinear CNN with tensor sketch algorithm to generate embeddings that capture local pairwise feature interactions in a translationally invariant manner. For apparel classification, we pass the feature embeddings through a softmax classifier, while, the in-shop and cross-domain retrieval pipelines use a triplet-loss based optimization approach, such that squared Euclidean distance between embeddings measures the dissimilarity between the images. Unlike previous works that relied on bounding box, key clothing landmarks or human joint detectors to assist the final deep classifier, proposed framework can be trained directly on the provided category labels or generated triplets for triplet loss optimization. Lastly, Experimental results on the DeepFashion fine-grained categorization, and in-shop and consumer-to-shop retrieval datasets provide a comparative analysis with previous work performed in the domain. △ Less

Submitted 6 November, 2018; originally announced November 2018.

Comments: 14 pages, 6 figures, 3 tables, Submitted to Springer Journal of Applied Intelligence

Automatic Target Recovery for Hindi-English Code Mixed Puns

Authors: Srishti Aggarwal, Kritik Mathur, Radhika Mamidi

Abstract: In order for our computer systems to be more human-like, with a higher emotional quotient, they need to be able to process and understand intrinsic human language phenomena like humour. In this paper, we consider a subtype of humour - puns, which are a common type of wordplay-based jokes. In particular, we consider code-mixed puns which have become increasingly mainstream on social media, in infor… ▽ More In order for our computer systems to be more human-like, with a higher emotional quotient, they need to be able to process and understand intrinsic human language phenomena like humour. In this paper, we consider a subtype of humour - puns, which are a common type of wordplay-based jokes. In particular, we consider code-mixed puns which have become increasingly mainstream on social media, in informal conversations and advertisements and aim to build a system which can automatically identify the pun location and recover the target of such puns. We first study and classify code-mixed puns into two categories namely intra-sentential and intra-word, and then propose a four-step algorithm to recover the pun targets for puns belonging to the intra-sentential category. Our algorithm uses language models, and phonetic similarity-based features to get the desired results. We test our approach on a small set of code-mixed punning advertisements, and observe that our system is successfully able to recover the targets for 67% of the puns. △ Less

Submitted 11 June, 2018; originally announced June 2018.

Strategies for Utility Maximization in Social Groups with Preferential Exploration

Authors: Saurabh Aggarwal, Joy Kuri

Abstract: We consider a \emph{Social Group} of networked nodes, seeking a "universe" of segments for maximization of their utility. Each node has a subset of the universe, and access to an expensive link for downloading data. Nodes can also acquire the universe by exchanging copies of segments among themselves, at low cost, using inter-node links. While exchanges over inter-node links ensure minimum or negl… ▽ More We consider a \emph{Social Group} of networked nodes, seeking a "universe" of segments for maximization of their utility. Each node has a subset of the universe, and access to an expensive link for downloading data. Nodes can also acquire the universe by exchanging copies of segments among themselves, at low cost, using inter-node links. While exchanges over inter-node links ensure minimum or negligible cost, some nodes in the group try to exploit the system. We term such nodes as `non-reciprocating nodes' and prohibit such behavior by proposing the "Give-and-Take" criterion, where exchange is allowed iff each participating node has segments unavailable with the other. Following this criterion for inter-node links, each node wants to maximize its utility, which depends on the node's segment set available with the node. Link activation among nodes requires mutual consent of participating nodes. Each node tries to find a pairing partner by preferentially exploring nodes for link formation and unpaired nodes choose to download a segment using the expensive link with segment aggressive probability. We present various linear complexity decentralized algorithms based on \emph{Stable Roommates Problem} that can be used by nodes (as per their behavioral nature) for choosing the best strategy based on available information. Then, we present decentralized randomized algorithm that performs close to optimal for large number of nodes. We define \emph{Price of Choices} for benchmarking performance for social groups (consisting of non-aggressive nodes only). We evaluate performances of various algorithms and characterize the behavioral regime that will yield best results for node and social group, spending the minimal on expensive link. We consider social group consisting of non-aggressive nodes and benchmark performances of proposed algorithms with the optimal. △ Less

Submitted 10 September, 2014; originally announced September 2014.

Social optimum in Social Groups with Give-and-Take criterion

Authors: Saurabh Aggarwal, Joy Kuri, Rahul Vaze

Abstract: We consider a "Social Group" of networked nodes, seeking a "universe" of segments. Each node has subset of the universe, and access to an expensive resource for downloading data. Alternatively, nodes can also acquire the universe by exchanging segments among themselves, at low cost, using a local network interface. While local exchanges ensure minimum cost, "free riders" in the group can exploit t… ▽ More We consider a "Social Group" of networked nodes, seeking a "universe" of segments. Each node has subset of the universe, and access to an expensive resource for downloading data. Alternatively, nodes can also acquire the universe by exchanging segments among themselves, at low cost, using a local network interface. While local exchanges ensure minimum cost, "free riders" in the group can exploit the system. To prohibit free riding, we propose the "Give-and-Take" criterion, where exchange is allowed if each node has segments unavailable with the other. Under this criterion, we consider the problem of maximizing the aggregate cardinality of the nodes' segment sets. First, we present a randomized algorithm, whose analysis yields a lower bound on the expected aggregate cardinality, as well as an approximation ratio of 1/4 under some conditions. Four other algorithms are presented and analyzed. We identify conditions under which some of these algorithms are optimal △ Less

Submitted 8 August, 2013; originally announced August 2013.

Comments: Submitted for review to INFOCOM 2014

Application Delay Modelling for Variable Length Packets in Single Cell IEEE 802.11 WLANs

Authors: Albert Sunny, Joy Kuri, Saurabh Aggarwal

Abstract: In this paper, we consider the problem of modelling the average delay experienced by an application packets of variable length in a single cell IEEE 802.11 DCF wireless local area network. The packet arrival process at each node i is assumed to be a stationary and independent increment random process with mean ai and second moment a(2) i . The packet lengths at node i are assumed to be i.i.d rando… ▽ More In this paper, we consider the problem of modelling the average delay experienced by an application packets of variable length in a single cell IEEE 802.11 DCF wireless local area network. The packet arrival process at each node i is assumed to be a stationary and independent increment random process with mean ai and second moment a(2) i . The packet lengths at node i are assumed to be i.i.d random variables Pi with finite mean and second moment. A closed form expression has been derived for the same. We assume the input arrival process across queues to be uncorrelated Poison processes. As the nodes share a single channel, they have to contend with one another for a successful transmission. The mean delay for a packet has been approximated by modelling the system as a 1-limited Random Polling system with zero switchover times. Extensive simulations are conducted to verify the analytical results. △ Less

Submitted 27 September, 2010; originally announced September 2010.

Delay Modelling for Single Cell IEEE 802.11 WLANs Using a Random Polling System

Authors: Albert Sunny, Joy Kuri, Saurabh Aggarwal

Abstract: In this paper, we consider the problem of modelling the average delay experienced by a packet in a single cell IEEE 802.11 DCF wireless local area network. The packet arrival process at each node i is assumed to be Poisson with rate parameter λらむだ_i. Since the nodes are sharing a single channel, they have to contend with one another for a successful transmission. The mean delay for a packet has been… ▽ More In this paper, we consider the problem of modelling the average delay experienced by a packet in a single cell IEEE 802.11 DCF wireless local area network. The packet arrival process at each node i is assumed to be Poisson with rate parameter λらむだ_i. Since the nodes are sharing a single channel, they have to contend with one another for a successful transmission. The mean delay for a packet has been approximated by modelling the system as a 1-limited Random Polling system with zero switchover time. We show that even for non-homogeneous packet arrival processes, the mean delay of packets across the queues are same and depends on the system utilization factor and the aggregate throughput of the MAC. Extensive simulations are conducted to verify the analytical results. △ Less

Submitted 22 September, 2010; v1 submitted 17 September, 2010; originally announced September 2010.

Delay Modelling for a Single-hop Wireless Mesh Network under Light Aggregate Traffic

Authors: Albert Sunny, Joy Kuri, Saurabh Aggarwal

Abstract: In this paper, we consider the problem of modelling the average delay in an IEEE 802.11 DCF wireless mesh network with a single root node under light traffic. We derive expression for mean delay for a co-located wireless mesh network, when packet generation is homogeneous Poisson process with rate λらむだ. We also show how our analysis can be extended for non-homogeneous Poisson packet generation. We mo… ▽ More In this paper, we consider the problem of modelling the average delay in an IEEE 802.11 DCF wireless mesh network with a single root node under light traffic. We derive expression for mean delay for a co-located wireless mesh network, when packet generation is homogeneous Poisson process with rate λらむだ. We also show how our analysis can be extended for non-homogeneous Poisson packet generation. We model mean delay by decoupling queues into independent M/M/1 queues. Extensive simulations are conducted to verify the analytical results. △ Less

Submitted 3 September, 2010; v1 submitted 2 September, 2010; originally announced September 2010.