Showing 1–50 of 92 results for author: Chin, T

Weakly Supervised Test-Time Domain Adaptation for Object Detection

Authors: Anh-Dzung Doan, Bach Long Nguyen, Terry Lim, Madhuka Jayawardhana, Surabhi Gupta, Christophe Guettier, Ian Reid, Markus Wagner, Tat-Jun Chin

Abstract: Prior to deployment, an object detector is trained on a dataset compiled from a previous data collection campaign. However, the environment in which the object detector is deployed will invariably evolve, particularly in outdoor settings where changes in lighting, weather and seasons will significantly affect the appearance of the scene and target objects. It is almost impossible for all potential… ▽ More Prior to deployment, an object detector is trained on a dataset compiled from a previous data collection campaign. However, the environment in which the object detector is deployed will invariably evolve, particularly in outdoor settings where changes in lighting, weather and seasons will significantly affect the appearance of the scene and target objects. It is almost impossible for all potential scenarios that the object detector may come across to be present in a finite training dataset. This necessitates continuous updates to the object detector to maintain satisfactory performance. Test-time domain adaptation techniques enable machine learning models to self-adapt based on the distributions of the testing data. However, existing methods mainly focus on fully automated adaptation, which makes sense for applications such as self-driving cars. Despite the prevalence of fully automated approaches, in some applications such as surveillance, there is usually a human operator overseeing the system's operation. We propose to involve the operator in test-time domain adaptation to raise the performance of object detection beyond what is achievable by fully automated adaptation. To reduce manual effort, the proposed method only requires the operator to provide weak labels, which are then used to guide the adaptation process. Furthermore, the proposed method can be performed in a streaming setting, where each online sample is observed only once. We show that the proposed method outperforms existing works, demonstrating a great benefit of human-in-the-loop test-time domain adaptation. Our code is publicly available at https://github.com/dzungdoan6/WSTTA △ Less

Submitted 8 July, 2024; originally announced July 2024.

Panopticon: a telescope for our times

Authors: Will Saunders, Timothy Chin, Michael Goodwin

Abstract: We present a design for a wide-field spectroscopic telescope. The only large powered mirror is spherical, the resulting spherical aberration is corrected for each target separately, giving exceptional image quality. The telescope is a transit design, but still allows all-sky coverage. Three simultaneous modes are proposed: (a) natural seeing multi-object spectroscopy with 12m aperture over 3dg FoV… ▽ More We present a design for a wide-field spectroscopic telescope. The only large powered mirror is spherical, the resulting spherical aberration is corrected for each target separately, giving exceptional image quality. The telescope is a transit design, but still allows all-sky coverage. Three simultaneous modes are proposed: (a) natural seeing multi-object spectroscopy with 12m aperture over 3dg FoV with ~25,000 targets; (b) multi-object AO with 12m aperture over 3dg FoV with ~100 AO-corrected Integral Field Units each with 4 arcsec FoV; (c) ground layer AO-corrected integral field spectroscopy with 15m aperture and 13 arcmin FoV. Such a telescope would be uniquely powerful for large-area follow-up of imaging surveys; in each mode, the AOmega and survey speed exceed all existing facilities combined. The expected cost of this design is relatively modest, much closer to $500M than $1000M. △ Less

Submitted 6 July, 2024; originally announced July 2024.

Comments: 10 pages. SPIE 13094-191, Ground-based and Airborne Telescopes X, Yokohama 2024

The Design, Implementation, and Performance of the LZ Calibration Systems

Authors: J. Aalbers, D. S. Akerib, A. K. Al Musalhi, F. Alder, C. S. Amarasinghe, A. Ames, T. J. Anderson, N. Angelides, H. M. Araújo, J. E. Armstrong, M. Arthurs, A. Baker, S. Balashov, J. Bang, E. E. Barillier, J. W. Bargemann, K. Beattie, T. Benson, A. Bhatti, A. Biekert, T. P. Biesiadzinski, H. J. Birch, E. Bishop, G. M. Blockinger, B. Boxer , et al. (179 additional authors not shown)

Abstract: LUX-ZEPLIN (LZ) is a tonne-scale experiment searching for direct dark matter interactions and other rare events. It is located at the Sanford Underground Research Facility (SURF) in Lead, South Dakota, USA. The core of the LZ detector is a dual-phase xenon time projection chamber (TPC), designed with the primary goal of detecting Weakly Interacting Massive Particles (WIMPs) via their induced low e… ▽ More LUX-ZEPLIN (LZ) is a tonne-scale experiment searching for direct dark matter interactions and other rare events. It is located at the Sanford Underground Research Facility (SURF) in Lead, South Dakota, USA. The core of the LZ detector is a dual-phase xenon time projection chamber (TPC), designed with the primary goal of detecting Weakly Interacting Massive Particles (WIMPs) via their induced low energy nuclear recoils. Surrounding the TPC, two veto detectors immersed in an ultra-pure water tank enable reducing background events to enhance the discovery potential. Intricate calibration systems are purposely designed to precisely understand the responses of these three detector volumes to various types of particle interactions and to demonstrate LZ's ability to discriminate between signals and backgrounds. In this paper, we present a comprehensive discussion of the key features, requirements, and performance of the LZ calibration systems, which play a crucial role in enabling LZ's WIMP-search and its broad science program. The thorough description of these calibration systems, with an emphasis on their novel aspects, is valuable for future calibration efforts in direct dark matter and other rare-event search experiments. △ Less

Submitted 20 June, 2024; v1 submitted 2 May, 2024; originally announced June 2024.

Camera-Pose Robust Crater Detection from Chang'e 5

Authors: Matthew Rodda, Sofia McLeod, Ky Cuong Pham, Tat-Jun Chin

Abstract: As space missions aim to explore increasingly hazardous terrain, accurate and timely position estimates are required to ensure safe navigation. Vision-based navigation achieves this goal through correlating impact craters visible through onboard imagery with a known database to estimate a craft's pose. However, existing literature has not sufficiently evaluated crater-detection algorithm (CDA) per… ▽ More As space missions aim to explore increasingly hazardous terrain, accurate and timely position estimates are required to ensure safe navigation. Vision-based navigation achieves this goal through correlating impact craters visible through onboard imagery with a known database to estimate a craft's pose. However, existing literature has not sufficiently evaluated crater-detection algorithm (CDA) performance from imagery containing off-nadir view angles. In this work, we evaluate the performance of Mask R-CNN for crater detection, comparing models pretrained on simulated data containing off-nadir view angles and to pretraining on real-lunar images. We demonstrate pretraining on real-lunar images is superior despite the lack of images containing off-nadir view angles, achieving detection performance of 63.1 F1-score and ellipse-regression performance of 0.701 intersection over union. This work provides the first quantitative analysis of performance of CDAs on images containing off-nadir view angles. Towards the development of increasingly robust CDAs, we additionally provide the first annotated CDA dataset with off-nadir view angles from the Chang'e 5 Landing Camera. △ Less

Submitted 12 July, 2024; v1 submitted 6 June, 2024; originally announced June 2024.

Probing the Scalar WIMP-Pion Coupling with the first LUX-ZEPLIN data

Authors: J. Aalbers, D. S. Akerib, A. K. Al Musalhi, F. Alder, C. S. Amarasinghe, A. Ames, T. J. Anderson, N. Angelides, H. M. Araújo, J. E. Armstrong, M. Arthurs, A. Baker, S. Balashov, J. Bang, E. E. Barillier, J. W. Bargemann, K. Beattie, T. Benson, A. Bhatti, A. Biekert, T. P. Biesiadzinski, H. J. Birch, E. J. Bishop, G. M. Blockinger, B. Boxer , et al. (178 additional authors not shown)

Abstract: Weakly interacting massive particles (WIMPs) may interact with a virtual pion that is exchanged between nucleons. This interaction channel is important to consider in models where the spin-independent isoscalar channel is suppressed. Using data from the first science run of the LUX-ZEPLIN dark matter experiment, containing 60 live days of data in a 5.5~tonne fiducial mass of liquid xenon, we repor… ▽ More Weakly interacting massive particles (WIMPs) may interact with a virtual pion that is exchanged between nucleons. This interaction channel is important to consider in models where the spin-independent isoscalar channel is suppressed. Using data from the first science run of the LUX-ZEPLIN dark matter experiment, containing 60 live days of data in a 5.5~tonne fiducial mass of liquid xenon, we report the results on a search for WIMP-pion interactions. We observe no significant excess and set an upper limit of $1.5\times10^{-46}$~cm$^2$ at a 90\% confidence level for a WIMP mass of 33~GeV/c$^2$ for this interaction. △ Less

Submitted 4 June, 2024; originally announced June 2024.

The Data Acquisition System of the LZ Dark Matter Detector: FADR

Authors: J. Aalbers, D. S. Akerib, A. K. Al Musalhi, F. Alder, C. S. Amarasinghe, A. Ames, T. J. Anderson, N. Angelides, H. M. Araújo, J. E. Armstrong, M. Arthurs, A. Baker, S. Balashov, J. Bang, E. E. Barillier, J. W. Bargemann, K. Beattie, T. Benson, A. Bhatti, A. Biekert, T. P. Biesiadzinski, H. J. Birch, E. Bishop, G. M. Blockinger, B. Boxer , et al. (190 additional authors not shown)

Abstract: The Data Acquisition System (DAQ) for the LUX-ZEPLIN (LZ) dark matter detector is described. The signals from 745 PMTs, distributed across three subsystems, are sampled with 100-MHz 32-channel digitizers (DDC-32s). A basic waveform analysis is carried out on the on-board Field Programmable Gate Arrays (FPGAs) to extract information about the observed scintillation and electroluminescence signals.… ▽ More The Data Acquisition System (DAQ) for the LUX-ZEPLIN (LZ) dark matter detector is described. The signals from 745 PMTs, distributed across three subsystems, are sampled with 100-MHz 32-channel digitizers (DDC-32s). A basic waveform analysis is carried out on the on-board Field Programmable Gate Arrays (FPGAs) to extract information about the observed scintillation and electroluminescence signals. This information is used to determine if the digitized waveforms should be preserved for offline analysis. The system is designed around the Kintex-7 FPGA. In addition to digitizing the PMT signals and providing basic event selection in real time, the flexibility provided by the use of FPGAs allows us to monitor the performance of the detector and the DAQ in parallel to normal data acquisition. The hardware and software/firmware of this FPGA-based Architecture for Data acquisition and Realtime monitoring (FADR) are discussed and performance measurements are described. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Comments: 18 pages, 24 figures

Event-based Structure-from-Orbit

Authors: Ethan Elms, Yasir Latif, Tae Ha Park, Tat-Jun Chin

Abstract: Event sensors offer high temporal resolution visual sensing, which makes them ideal for perceiving fast visual phenomena without suffering from motion blur. Certain applications in robotics and vision-based navigation require 3D perception of an object undergoing circular or spinning motion in front of a static camera, such as recovering the angular velocity and shape of the object. The setting is… ▽ More Event sensors offer high temporal resolution visual sensing, which makes them ideal for perceiving fast visual phenomena without suffering from motion blur. Certain applications in robotics and vision-based navigation require 3D perception of an object undergoing circular or spinning motion in front of a static camera, such as recovering the angular velocity and shape of the object. The setting is equivalent to observing a static object with an orbiting camera. In this paper, we propose event-based structure-from-orbit (eSfO), where the aim is to simultaneously reconstruct the 3D structure of a fast spinning object observed from a static event camera, and recover the equivalent orbital motion of the camera. Our contributions are threefold: since state-of-the-art event feature trackers cannot handle periodic self-occlusion due to the spinning motion, we develop a novel event feature tracker based on spatio-temporal clustering and data association that can better track the helical trajectories of valid features in the event data. The feature tracks are then fed to our novel factor graph-based structure-from-orbit back-end that calculates the orbital motion parameters (e.g., spin rate, relative rotational axis) that minimize the reprojection error. For evaluation, we produce a new event dataset of objects under spinning motion. Comparisons against ground truth indicate the efficacy of eSfO. △ Less

Submitted 9 May, 2024; originally announced May 2024.

Comments: This work will be published in the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 2024

Real Stability and Log Concavity are coNP-Hard

Authors: Tracy Chin

Abstract: Real-stable, Lorentzian, and log-concave polynomials are well-studied classes of polynomials, and have been powerful tools in resolving several conjectures. We show that the problems of deciding whether a polynomial of fixed degree is real stable or log concave are coNP-hard. On the other hand, while all homogeneous real-stable polynomials are Lorentzian and all Lorentzian polynomials are log conc… ▽ More Real-stable, Lorentzian, and log-concave polynomials are well-studied classes of polynomials, and have been powerful tools in resolving several conjectures. We show that the problems of deciding whether a polynomial of fixed degree is real stable or log concave are coNP-hard. On the other hand, while all homogeneous real-stable polynomials are Lorentzian and all Lorentzian polynomials are log concave on the positive orthant, the problem of deciding whether a polynomial of fixed degree is Lorentzian can be solved in polynomial time. △ Less

Submitted 21 May, 2024; v1 submitted 30 April, 2024; originally announced May 2024.

Comments: 21 pages, 1 figure

Constraints On Covariant WIMP-Nucleon Effective Field Theory Interactions from the First Science Run of the LUX-ZEPLIN Experiment

Authors: J. Aalbers, D. S. Akerib, A. K. Al Musalhi, F. Alder, C. S. Amarasinghe, A. Ames, T. J. Anderson, N. Angelides, H. M. Araújo, J. E. Armstrong, M. Arthurs, A. Baker, S. Balashov, J. Bang, E. E. Barillier, J. W. Bargemann, K. Beattie, T. Benson, A. Bhatti, A. Biekert, T. P. Biesiadzinski, H. J. Birch, E. J. Bishop, G. M. Blockinger, B. Boxer , et al. (179 additional authors not shown)

Abstract: The first science run of the LUX-ZEPLIN (LZ) experiment, a dual-phase xenon time project chamber operating in the Sanford Underground Research Facility in South Dakota, USA, has reported leading limits on spin-independent WIMP-nucleon interactions and interactions described from a non-relativistic effective field theory (NREFT). Using the same 5.5~t fiducial mass and 60 live days of exposure we re… ▽ More The first science run of the LUX-ZEPLIN (LZ) experiment, a dual-phase xenon time project chamber operating in the Sanford Underground Research Facility in South Dakota, USA, has reported leading limits on spin-independent WIMP-nucleon interactions and interactions described from a non-relativistic effective field theory (NREFT). Using the same 5.5~t fiducial mass and 60 live days of exposure we report on the results of a relativistic extension to the NREFT. We present constraints on couplings from covariant interactions arising from the coupling of vector, axial currents, and electric dipole moments of the nucleon to the magnetic and electric dipole moments of the WIMP which cannot be described by recasting previous results described by an NREFT. Using a profile-likelihood ratio analysis, in an energy region between 0~keV$_\text{nr}$ to 270~keV$_\text{nr}$, we report 90% confidence level exclusion limits on the coupling strength of five interactions in both the isoscalar and isovector bases. △ Less

Submitted 26 April, 2024; originally announced April 2024.

Comments: 7 pages, 4 figures

Effective dynamics of qubit networks via phase-covariant quantum ensembles

Authors: Sean Prudhoe, Unnati Akhouri, Tommy Chin, Sarah Shandera

Abstract: We study ensembles of phase-covariant channels. We show that such ensembles arise naturally from familiar spin-chain models (e.g., XXZ) with a special class of initial states, and that the disorder-averaged map of disordered spin chains is phase-covariant under a weak symmetry constraint on the distribution. We use those examples to motivate a broader class of phase-covariant ensembles, which incl… ▽ More We study ensembles of phase-covariant channels. We show that such ensembles arise naturally from familiar spin-chain models (e.g., XXZ) with a special class of initial states, and that the disorder-averaged map of disordered spin chains is phase-covariant under a weak symmetry constraint on the distribution. We use those examples to motivate a broader class of phase-covariant ensembles, which include both unital and non-unital channels. We demonstrate the physical properties captured by the late-time limit of the average map over the ensemble. △ Less

Submitted 23 April, 2024; originally announced April 2024.

Comments: 41 pages, 8 figures

GHOST Commissioning Science Results III: Characterizing an iron-poor damped Lyman $αあるふぁ$ system

Authors: Trystyn A. M. Berg, Christian R. Hayes, Stefano Cristiani, Alan McConnachie, J. Gordon Robertson, Federico Sestito, Chris Simpson, Fletcher Waller, Timothy Chin, Adam Densmore, Ruben J. Diaz, Michael L. Edgar, Javier Fuentes Lettura, Manuel Gómez-Jiménez, Venu M. Kalari, Jon Lawrence, Steven Margheim, John Pazder, Roque Ruiz-Carmona, Ricardo Salinas, Karleyne M. G. Silva, Katherine Silversides, Kim A. Venn

Abstract: The Gemini High-resolution Optical SpecTrograph (GHOST) is a new echelle spectrograph available on the Gemini-South telescope as of Semester 2024A. We present the first high resolution spectrum of the quasar J1449-1227 (redshift z_em=3.27) using data taken during the commissioning of GHOST. The observed quasar hosts an intervening iron-poor ([Fe/H] = -2.5) damped Lyman alpha (DLA) system at redshi… ▽ More The Gemini High-resolution Optical SpecTrograph (GHOST) is a new echelle spectrograph available on the Gemini-South telescope as of Semester 2024A. We present the first high resolution spectrum of the quasar J1449-1227 (redshift z_em=3.27) using data taken during the commissioning of GHOST. The observed quasar hosts an intervening iron-poor ([Fe/H] = -2.5) damped Lyman alpha (DLA) system at redshift z=2.904. Taking advantage of the high spectral resolving power of GHOST (R~55000), we are able to accurately model the metal absorption lines of the metal-poor DLA and find a supersolar [Si/Fe], suggesting the DLA gas is in an early stage of chemical enrichment. Using simple ionization models, we find that the large range in the C IV/Si IV column density ratio of individual components within the DLA's high ionization absorption profile can be reproduced by several metal-poor Lyman limit systems surrounding the low-ionization gas of the DLA. It is possible that this metal-poor DLA resides within a complex system of metal-poor galaxies or filaments with inflowing gas. The high spectral resolution, wavelength coverage and sensitivity of GHOST makes it an ideal spectrograph for characterizing the chemistry and kinematics of quasar absorption lines. △ Less

Submitted 18 April, 2024; v1 submitted 15 April, 2024; originally announced April 2024.

Comments: Accepted for publication in MNRAS. 8 Pages, 5 figures

The Science Performance of the Gemini High Resolution Optical Spectrograph

Authors: Alan W. McConnachie, Christian R. Hayes, J. Gordon Robertson, John Pazder, Michael Ireland, Greg Burley, Vladimir Churilov, Jordan Lothrop, Ross Zhelem, Venu Kalari, André Anthony, Gabriella Baker, Trystyn Berg, Edward L. Chapin, Timothy Chin, Adam Densmore, Ruben Diaz, Jennifer Dunn, Michael L. Edgar, Tony Farrell, Veronica Firpo, Javier Fuentes, Manuel Gomez-Jimenez, Tim Hardy, David Henderson , et al. (24 additional authors not shown)

Abstract: The Gemini High Resolution Optical Spectrograph (GHOST) is a fiber-fed spectrograph system on the Gemini South telescope that provides simultaneous wavelength coverage from 348 - 1061nm, and designed for optimal performance between 363 - 950nm. It can observe up to two objects simultaneously in a 7.5 arcmin diameter field of regard at R = 56,000 or a single object at R = 75,000. The spectral resol… ▽ More The Gemini High Resolution Optical Spectrograph (GHOST) is a fiber-fed spectrograph system on the Gemini South telescope that provides simultaneous wavelength coverage from 348 - 1061nm, and designed for optimal performance between 363 - 950nm. It can observe up to two objects simultaneously in a 7.5 arcmin diameter field of regard at R = 56,000 or a single object at R = 75,000. The spectral resolution modes are obtained by using integral field units to image slice a 1.2" aperture by a factor of five in width using 19 fibers in the high resolution mode and by a factor of three in width using 7 fibers in the standard resolution mode. GHOST is equipped with hardware to allow for precision radial velocity measurements, expected to approach meters per second precision. Here, we describe the basic design and operational capabilities of GHOST, and proceed to derive and quantify the key aspects of its on-sky performance that are of most relevance to its science users. △ Less

Submitted 14 January, 2024; originally announced January 2024.

Comments: 37 pages, 27 figures. Accepted for publication in Publications of the Astronomical Society of the Pacific

Projected Stochastic Gradient Descent with Quantum Annealed Binary Gradients

Authors: Maximilian Krahn, Michelle Sasdelli, Fengyi Yang, Vladislav Golyanik, Juho Kannala, Tat-Jun Chin, Tolga Birdal

Abstract: We present, QP-SBGD, a novel layer-wise stochastic optimiser tailored towards training neural networks with binary weights, known as binary neural networks (BNNs), on quantum hardware. BNNs reduce the computational requirements and energy consumption of deep learning models with minimal loss in accuracy. However, training them in practice remains to be an open challenge. Most known BNN-optimisers… ▽ More We present, QP-SBGD, a novel layer-wise stochastic optimiser tailored towards training neural networks with binary weights, known as binary neural networks (BNNs), on quantum hardware. BNNs reduce the computational requirements and energy consumption of deep learning models with minimal loss in accuracy. However, training them in practice remains to be an open challenge. Most known BNN-optimisers either rely on projected updates or binarise weights post-training. Instead, QP-SBGD approximately maps the gradient onto binary variables, by solving a quadratic constrained binary optimisation. Under practically reasonable assumptions, we show that this update rule converges with a rate of $\mathcal{O}(1 / \sqrt{T})$. Moreover, we show how the $\mathcal{NP}$-hard projection can be effectively executed on an adiabatic quantum annealer, harnessing recent advancements in quantum computation. We also introduce a projected version of this update rule and prove that if a fixed point exists in the binary variable space, the modified updates will converge to it. Last but not least, our algorithm is implemented layer-wise, making it suitable to train larger networks on resource-limited quantum hardware. Through extensive evaluations, we show that QP-SBGD outperforms or is on par with competitive and well-established baselines such as BinaryConnect, signSGD and ProxQuant when optimising the Rosenbrock function, training BNNs as well as binary graph neural networks. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Commentary on Guyll et al. (2023): Misuse of Statistical Method Results in Highly Biased Interpretation of Forensic Evidence

Authors: Michael Rosenblum, Elizabeth T. Chin, Elizabeth L. Ogburn, Akihiko Nishimura, Daniel Westreich, Abhirup Datta, Susan Vanderplas, Maria Cuellar, William C. Thompson

Abstract: Since the National Academy of Sciences released their report outlining paths for improving reliability, standards, and policies in the forensic sciences NAS (2009), there has been heightened interest in evaluating and improving the scientific validity within forensic science disciplines. Guyll et al. (2023) seek to evaluate the validity of forensic cartridge-case comparisons. However, they make a… ▽ More Since the National Academy of Sciences released their report outlining paths for improving reliability, standards, and policies in the forensic sciences NAS (2009), there has been heightened interest in evaluating and improving the scientific validity within forensic science disciplines. Guyll et al. (2023) seek to evaluate the validity of forensic cartridge-case comparisons. However, they make a serious statistical error that leads to highly inflated claims about the probability that a cartridge case from a crime scene was fired from a reference gun, typically a gun found in the possession of a defendant. It is urgent to address this error since these claims, which are generally biased against defendants, are being presented by the prosecution in an ongoing homicide case where the defendant faces the possibility of a lengthy prison sentence (DC Superior Court, 2023). △ Less

Submitted 8 September, 2023; originally announced September 2023.

Domain Adaptation for Satellite-Borne Hyperspectral Cloud Detection

Authors: Andrew Du, Anh-Dzung Doan, Yee Wei Law, Tat-Jun Chin

Abstract: The advent of satellite-borne machine learning hardware accelerators has enabled the on-board processing of payload data using machine learning techniques such as convolutional neural networks (CNN). A notable example is using a CNN to detect the presence of clouds in hyperspectral data captured on Earth observation (EO) missions, whereby only clear sky data is downlinked to conserve bandwidth. Ho… ▽ More The advent of satellite-borne machine learning hardware accelerators has enabled the on-board processing of payload data using machine learning techniques such as convolutional neural networks (CNN). A notable example is using a CNN to detect the presence of clouds in hyperspectral data captured on Earth observation (EO) missions, whereby only clear sky data is downlinked to conserve bandwidth. However, prior to deployment, new missions that employ new sensors will not have enough representative datasets to train a CNN model, while a model trained solely on data from previous missions will underperform when deployed to process the data on the new missions. This underperformance stems from the domain gap, i.e., differences in the underlying distributions of the data generated by the different sensors in previous and future missions. In this paper, we address the domain gap problem in the context of on-board hyperspectral cloud detection. Our main contributions lie in formulating new domain adaptation tasks that are motivated by a concrete EO mission, developing a novel algorithm for bandwidth-efficient supervised domain adaptation, and demonstrating test-time adaptation algorithms on space deployable neural network accelerators. Our contributions enable minimal data transmission to be invoked (e.g., only 1% of the weights in ResNet50) to achieve domain adaptation, thereby allowing more sophisticated CNN models to be deployed and updated on satellites without being hampered by domain gap and bandwidth limitations. △ Less

Submitted 5 September, 2023; originally announced September 2023.

High Frequency, High Accuracy Pointing onboard Nanosats using Neuromorphic Event Sensing and Piezoelectric Actuation

Authors: Yasir Latif, Peter Anastasiou, Yonhon Ng, Zebb Prime, Tien-Fu Lu, Matthew Tetlow, Robert Mahony, Tat-Jun Chin

Abstract: As satellites become smaller, the ability to maintain stable pointing decreases as external forces acting on the satellite come into play. At the same time, reaction wheels used in the attitude determination and control system (ADCS) introduce high frequency jitter which can disrupt pointing stability. For space domain awareness (SDA) tasks that track objects tens of thousands of kilometres away,… ▽ More As satellites become smaller, the ability to maintain stable pointing decreases as external forces acting on the satellite come into play. At the same time, reaction wheels used in the attitude determination and control system (ADCS) introduce high frequency jitter which can disrupt pointing stability. For space domain awareness (SDA) tasks that track objects tens of thousands of kilometres away, the pointing accuracy offered by current nanosats, typically in the range of 10 to 100 arcseconds, is not sufficient. In this work, we develop a novel payload that utilises a neuromorphic event sensor (for high frequency and highly accurate relative attitude estimation) paired in a closed loop with a piezoelectric stage (for active attitude corrections) to provide highly stable sensor-specific pointing. Event sensors are especially suited for space applications due to their desirable characteristics of low power consumption, asynchronous operation, and high dynamic range. We use the event sensor to first estimate a reference background star field from which instantaneous relative attitude is estimated at high frequency. The piezoelectric stage works in a closed control loop with the event sensor to perform attitude corrections based on the discrepancy between the current and desired attitude. Results in a controlled setting show that we can achieve a pointing accuracy in the range of 1-5 arcseconds using our novel payload at an operating frequency of up to 50Hzへるつ using a prototype built from commercial-off-the-shelf components. Further details can be found at https://ylatif.github.io/ultrafinestabilisation △ Less

Submitted 10 September, 2023; v1 submitted 4 September, 2023; originally announced September 2023.

Direct initial orbit determination

Authors: Chee-Kheng Chng, Trent Jansen-Sturgeon, Timothy Payne, Tat-Jun Chin

Abstract: Initial orbit determination (IOD) is an important early step in the processing chain that makes sense of and reconciles the multiple optical observations of a resident space object. IOD methods generally operate on line-of-sight (LOS) vectors extracted from images of the object, hence the LOS vectors can be seen as discrete point samples of the raw optical measurements. Typically, the number of LO… ▽ More Initial orbit determination (IOD) is an important early step in the processing chain that makes sense of and reconciles the multiple optical observations of a resident space object. IOD methods generally operate on line-of-sight (LOS) vectors extracted from images of the object, hence the LOS vectors can be seen as discrete point samples of the raw optical measurements. Typically, the number of LOS vectors used by an IOD method is much smaller than the available measurements (\ie, the set of pixel intensity values), hence current IOD methods arguably under-utilize the rich information present in the data. In this paper, we propose a \emph{direct} IOD method called D-IOD that fits the orbital parameters directly on the observed streak images, without requiring LOS extraction. Since it does not utilize LOS vectors, D-IOD avoids potential inaccuracies or errors due to an imperfect LOS extraction step. Two innovations underpin our novel orbit-fitting paradigm: first, we introduce a novel non-linear least-squares objective function that computes the loss between the candidate-orbit-generated streak images and the observed streak images. Second, the objective function is minimized with a gradient descent approach that is embedded in our proposed optimization strategies designed for streak images. We demonstrate the effectiveness of D-IOD on a variety of simulated scenarios and challenging real streak images. △ Less

Submitted 28 August, 2023; originally announced August 2023.

Comments: 28 pages, 17 figures, Submitted to Advances in Space Research

Sensor Allocation and Online-Learning-based Path Planning for Maritime Situational Awareness Enhancement: A Multi-Agent Approach

Authors: Bach Long Nguyen, Anh-Dzung Doan, Tat-Jun Chin, Christophe Guettier, Surabhi Gupta, Estelle Parra, Ian Reid, Markus Wagner

Abstract: Countries with access to large bodies of water often aim to protect their maritime transport by employing maritime surveillance systems. However, the number of available sensors (e.g., cameras) is typically small compared to the to-be-monitored targets, and their Field of View (FOV) and range are often limited. This makes improving the situational awareness of maritime transports challenging. To t… ▽ More Countries with access to large bodies of water often aim to protect their maritime transport by employing maritime surveillance systems. However, the number of available sensors (e.g., cameras) is typically small compared to the to-be-monitored targets, and their Field of View (FOV) and range are often limited. This makes improving the situational awareness of maritime transports challenging. To this end, we propose a method that not only distributes multiple sensors but also plans paths for them to observe multiple targets, while minimizing the time needed to achieve situational awareness. In particular, we provide a formulation of this sensor allocation and path planning problem which considers the partial awareness of the targets' state, as well as the unawareness of the targets' trajectories. To solve the problem we present two algorithms: 1) a greedy algorithm for assigning sensors to targets, and 2) a distributed multi-agent path planning algorithm based on regret-matching learning. Because a quick convergence is a requirement for algorithms developed for high mobility environments, we employ a forgetting factor to quickly converge to correlated equilibrium solutions. Experimental results show that our combined approach achieves situational awareness more quickly than related work. △ Less

Submitted 26 November, 2023; v1 submitted 6 July, 2023; originally announced July 2023.

Semantic Segmentation on 3D Point Clouds with High Density Variations

Authors: Ryan Faulkner, Luke Haub, Simon Ratcliffe, Ian Reid, Tat-Jun Chin

Abstract: LiDAR scanning for surveying applications acquire measurements over wide areas and long distances, which produces large-scale 3D point clouds with significant local density variations. While existing 3D semantic segmentation models conduct downsampling and upsampling to build robustness against varying point densities, they are less effective under the large local density variations characteristic… ▽ More LiDAR scanning for surveying applications acquire measurements over wide areas and long distances, which produces large-scale 3D point clouds with significant local density variations. While existing 3D semantic segmentation models conduct downsampling and upsampling to build robustness against varying point densities, they are less effective under the large local density variations characteristic of point clouds from surveying applications. To alleviate this weakness, we propose a novel architecture called HDVNet that contains a nested set of encoder-decoder pathways, each handling a specific point density range. Limiting the interconnections between the feature maps enables HDVNet to gauge the reliability of each feature based on the density of a point, e.g., downweighting high density features not existing in low density objects. By effectively handling input density variations, HDVNet outperforms state-of-the-art models in segmentation accuracy on real point clouds with inconsistent density, using just over half the weights. △ Less

Submitted 4 July, 2023; originally announced July 2023.

ACM Class: I.4.6

GHOST Commissioning Science Results: Identifying a new chemically peculiar star in Reticulum II

Authors: Christian R. Hayes, Kim A. Venn, Fletcher Waller, Jaclyn Jensen, Alan W. McConnachie, John Pazder, Federico Sestito, Andre Anthony, Gabriella Baker, John Bassett, Joao Bento, Gregory Burley, Jurek Brzeski, Scott Case, Edward Chapin, Timothy Chin, Eric Chisholm, Vladimir Churilov, Adam Densmore, Ruben Diaz, Jennifer Dunn, Michael Edgar, Tony Farrell, Veronica Firpo, Joeleff Fitzsimmons , et al. (57 additional authors not shown)

Abstract: The Gemini High-resolution Optical SpecTrograph (GHOST) is the newest high resolution spectrograph to be developed for a large aperture telescope, recently deployed and commissioned at the Gemini-South telescope. In this paper, we present the first science results from the GHOST spectrograph taking during its commissioning runs. We have observed the bright metal-poor benchmark star HD 122563, alon… ▽ More The Gemini High-resolution Optical SpecTrograph (GHOST) is the newest high resolution spectrograph to be developed for a large aperture telescope, recently deployed and commissioned at the Gemini-South telescope. In this paper, we present the first science results from the GHOST spectrograph taking during its commissioning runs. We have observed the bright metal-poor benchmark star HD 122563, along with two stars in the ultra faint dwarf galaxy, Ret II, one of which was previously identified as a candidate member, but did not have a previous detailed chemical abundance analysis. This star (GDR3 0928) is found to be a bona fide member of Ret II, and from a spectral synthesis analysis, it is also revealed to be a CEMP-r star, with significant enhancements in the several light elements (C, N, O, Na, Mg, and Si), in addition to featuring an r-process enhancement like many other Ret II stars. The light-element enhancements in this star resemble the abundance patterns seen in the CEMP-no stars of other ultra faint dwarf galaxies, and are thought to have been produced by an independent source from the r-process. These unusual abundance patterns are thought to be produced by faint supernovae, which may be produced by some of the earliest generations of stars. △ Less

Submitted 7 June, 2023; originally announced June 2023.

Comments: 23 pages, 9 figures, 7 tables, submitted to the AAS Journals

Balancing Exploration and Exploitation: Disentangled $βべーた$-CVAE in De Novo Drug Design

Authors: Guang Jun Nicholas Ang, De Tao Irwin Chin, Bingquan Shen

Abstract: Deep generative models have recently emerged as a promising de novo drug design method. In this respect, deep generative conditional variational autoencoder (CVAE) models are a powerful approach for generating novel molecules with desired drug-like properties. However, molecular graph-based models with disentanglement and multivariate explicit latent conditioning have not been fully elucidated. To… ▽ More Deep generative models have recently emerged as a promising de novo drug design method. In this respect, deep generative conditional variational autoencoder (CVAE) models are a powerful approach for generating novel molecules with desired drug-like properties. However, molecular graph-based models with disentanglement and multivariate explicit latent conditioning have not been fully elucidated. To address this, we proposed a molecular-graph $βべーた$-CVAE model for de novo drug design. Here, we empirically tuned the value of disentanglement and assessed its ability to generate molecules with optimised univariate- or-multivariate properties. In particular, we optimised the octanol-water partition coefficient (ClogP), molar refractivity (CMR), quantitative estimate of drug-likeness (QED), and synthetic accessibility score (SAS). Results suggest that a lower $βべーた$ value increases the uniqueness of generated molecules (exploration). Univariate optimisation results showed our model generated molecular property averages of ClogP = 41.07% $\pm$ 0.01% and CMR 66.76% $\pm$ 0.01% by the Ghose filter. Multivariate property optimisation results showed that our model generated an average of 30.07% $\pm$ 0.01% molecules for both desired properties. Furthermore, our model improved the QED and SAS (exploitation) of molecules generated. Together, these results suggest that the $βべーた$-CVAE could balance exploration and exploitation through disentanglement and is a promising model for de novo drug design, thus providing a basis for future studies. △ Less

Submitted 17 August, 2023; v1 submitted 2 June, 2023; originally announced June 2023.

Federated Neural Radiance Fields

Authors: Lachlan Holden, Feras Dayoub, David Harvey, Tat-Jun Chin

Abstract: The ability of neural radiance fields or NeRFs to conduct accurate 3D modelling has motivated application of the technique to scene representation. Previous approaches have mainly followed a centralised learning paradigm, which assumes that all training images are available on one compute node for training. In this paper, we consider training NeRFs in a federated manner, whereby multiple compute n… ▽ More The ability of neural radiance fields or NeRFs to conduct accurate 3D modelling has motivated application of the technique to scene representation. Previous approaches have mainly followed a centralised learning paradigm, which assumes that all training images are available on one compute node for training. In this paper, we consider training NeRFs in a federated manner, whereby multiple compute nodes, each having acquired a distinct set of observations of the overall scene, learn a common NeRF in parallel. This supports the scenario of cooperatively modelling a scene using multiple agents. Our contribution is the first federated learning algorithm for NeRF, which splits the training effort across multiple compute nodes and obviates the need to pool the images at a central node. A technique based on low-rank decomposition of NeRF layers is introduced to reduce bandwidth consumption to transmit the model parameters for aggregation. Transferring compressed models instead of the raw data also contributes to the privacy of the data collecting agents. △ Less

Submitted 1 May, 2023; originally announced May 2023.

Comments: 10 pages, 7 figures

Potential for allocative harm in an environmental justice data tool

Authors: Benjamin Q. Huynh, Elizabeth T. Chin, Allison Koenecke, Derek Ouyang, Daniel E. Ho, Mathew V. Kiang, David H. Rehkopf

Abstract: Neighborhood-level screening algorithms are increasingly being deployed to inform policy decisions. We evaluate one such algorithm, CalEnviroScreen - designed to promote environmental justice and used to guide hundreds of millions of dollars in public funding annually - assessing its potential for allocative harm. We observe the model to be sensitive to subjective model decisions, with 16% of trac… ▽ More Neighborhood-level screening algorithms are increasingly being deployed to inform policy decisions. We evaluate one such algorithm, CalEnviroScreen - designed to promote environmental justice and used to guide hundreds of millions of dollars in public funding annually - assessing its potential for allocative harm. We observe the model to be sensitive to subjective model decisions, with 16% of tracts potentially changing designation, as well as financially consequential, estimating the effect of its positive designations as a 104% (62-145%) increase in funding, equivalent to \$2.08 billion (\$1.56-2.41 billion) over four years. We also observe allocative tradeoffs and susceptibility to manipulation, raising ethical concerns. We recommend incorporating sensitivity analyses to mitigate allocative harm and accountability mechanisms to prevent misuse. △ Less

Submitted 12 April, 2023; v1 submitted 12 April, 2023; originally announced April 2023.

Journal ref: Nat Mach Intell 6, 187-194 (2024)

Training Multilayer Perceptrons by Sampling with Quantum Annealers

Authors: Frances Fengyi Yang, Michele Sasdelli, Tat-Jun Chin

Abstract: A successful application of quantum annealing to machine learning is training restricted Boltzmann machines (RBM). However, many neural networks for vision applications are feedforward structures, such as multilayer perceptrons (MLP). Backpropagation is currently the most effective technique to train MLPs for supervised learning. This paper aims to be forward-looking by exploring the training of M… ▽ More A successful application of quantum annealing to machine learning is training restricted Boltzmann machines (RBM). However, many neural networks for vision applications are feedforward structures, such as multilayer perceptrons (MLP). Backpropagation is currently the most effective technique to train MLPs for supervised learning. This paper aims to be forward-looking by exploring the training of MLPs using quantum annealers. We exploit an equivalence between MLPs and energy-based models (EBM), which are a variation of RBMs with a maximum conditional likelihood objective. This leads to a strategy to train MLPs with quantum annealers as a sampling engine. We prove our setup for MLPs with sigmoid activation functions and one hidden layer, and demonstrated training of binary image classifiers on small subsets of the MNIST and Fashion-MNIST datasets using the D-Wave quantum annealer. Although problem sizes that are feasible on current annealers are limited, we obtained comprehensive results on feasible instances that validate our ideas. Our work establishes the potential of quantum computing for training MLPs. △ Less

Submitted 22 March, 2023; originally announced March 2023.

Comments: 22 pages, 15 figures

ACM Class: I.2.6

Assessing Domain Gap for Continual Domain Adaptation in Object Detection

Authors: Anh-Dzung Doan, Bach Long Nguyen, Surabhi Gupta, Ian Reid, Markus Wagner, Tat-Jun Chin

Abstract: To ensure reliable object detection in autonomous systems, the detector must be able to adapt to changes in appearance caused by environmental factors such as time of day, weather, and seasons. Continually adapting the detector to incorporate these changes is a promising solution, but it can be computationally costly. Our proposed approach is to selectively adapt the detector only when necessary,… ▽ More To ensure reliable object detection in autonomous systems, the detector must be able to adapt to changes in appearance caused by environmental factors such as time of day, weather, and seasons. Continually adapting the detector to incorporate these changes is a promising solution, but it can be computationally costly. Our proposed approach is to selectively adapt the detector only when necessary, using new data that does not have the same distribution as the current training data. To this end, we investigate three popular metrics for domain gap evaluation and find that there is a correlation between the domain gap and detection accuracy. Therefore, we apply the domain gap as a criterion to decide when to adapt the detector. Our experiments show that our approach has the potential to improve the efficiency of the detector's operation in real-world scenarios, where environmental conditions change in a cyclical manner, without sacrificing the overall performance of the detector. Our code is publicly available at https://github.com/dadung/DGE-CDA. △ Less

Submitted 21 November, 2023; v1 submitted 20 February, 2023; originally announced February 2023.

Comments: Accepted to CVIU

ROSIA: Rotation-Search-Based Star Identification Algorithm

Authors: Chee-Kheng Chng, Alvaro Parra Bustos, Benjamin McCarthy, Tat-Jun Chin

Abstract: This paper presents a rotation-search-based approach for addressing the star identification (Star-ID) problem. The proposed algorithm, ROSIA, is a heuristics-free algorithm that seeks the optimal rotation that maximally aligns the input and catalog stars in their respective coordinates. ROSIA searches the rotation space systematically with the Branch-and-Bound (BnB) method. Crucially affecting the… ▽ More This paper presents a rotation-search-based approach for addressing the star identification (Star-ID) problem. The proposed algorithm, ROSIA, is a heuristics-free algorithm that seeks the optimal rotation that maximally aligns the input and catalog stars in their respective coordinates. ROSIA searches the rotation space systematically with the Branch-and-Bound (BnB) method. Crucially affecting the runtime feasibility of ROSIA is the upper bound function that prioritizes the search space. In this paper, we make a theoretical contribution by proposing a tight (provable) upper bound function that enables a 400x speed-up compared to an existing formulation. Coupling the bounding function with an efficient evaluation scheme that leverages stereographic projection and the R-tree data structure, ROSIA achieves feasible operational speed on embedded processors with state-of-the-art performances under different sources of noise. The source code of ROSIA is available at https://github.com/ckchng/ROSIA. △ Less

Submitted 28 August, 2023; v1 submitted 2 October, 2022; originally announced October 2022.

Comments: 21 pages, 16 figures, Accepted to IEEE Transactions on Aerospace and Electronic Systems

Globally Optimal Event-Based Divergence Estimation for Ventral Landing

Authors: Sofia McLeod, Gabriele Meoni, Dario Izzo, Anne Mergy, Daqi Liu, Yasir Latif, Ian Reid, Tat-Jun Chin

Abstract: Event sensing is a major component in bio-inspired flight guidance and control systems. We explore the usage of event cameras for predicting time-to-contact (TTC) with the surface during ventral landing. This is achieved by estimating divergence (inverse TTC), which is the rate of radial optic flow, from the event stream generated during landing. Our core contributions are a novel contrast maximis… ▽ More Event sensing is a major component in bio-inspired flight guidance and control systems. We explore the usage of event cameras for predicting time-to-contact (TTC) with the surface during ventral landing. This is achieved by estimating divergence (inverse TTC), which is the rate of radial optic flow, from the event stream generated during landing. Our core contributions are a novel contrast maximisation formulation for event-based divergence estimation, and a branch-and-bound algorithm to exactly maximise contrast and find the optimal divergence value. GPU acceleration is conducted to speed up the global algorithm. Another contribution is a new dataset containing real event streams from ventral landing that was employed to test and benchmark our method. Owing to global optimisation, our algorithm is much more capable at recovering the true divergence, compared to other heuristic divergence estimators or event-based optic flow methods. With GPU acceleration, our method also achieves competitive runtimes. △ Less

Submitted 27 September, 2022; originally announced September 2022.

Comments: Accepted in the ECCV 2022 workshop on AI for Space, 18 pages, 6 figures

Towards Bridging the Space Domain Gap for Satellite Pose Estimation using Event Sensing

Authors: Mohsi Jawaid, Ethan Elms, Yasir Latif, Tat-Jun Chin

Abstract: Deep models trained using synthetic data require domain adaptation to bridge the gap between the simulation and target environments. State-of-the-art domain adaptation methods often demand sufficient amounts of (unlabelled) data from the target domain. However, this need is difficult to fulfil when the target domain is an extreme environment, such as space. In this paper, our target problem is clo… ▽ More Deep models trained using synthetic data require domain adaptation to bridge the gap between the simulation and target environments. State-of-the-art domain adaptation methods often demand sufficient amounts of (unlabelled) data from the target domain. However, this need is difficult to fulfil when the target domain is an extreme environment, such as space. In this paper, our target problem is close proximity satellite pose estimation, where it is costly to obtain images of satellites from actual rendezvous missions. We demonstrate that event sensing offers a promising solution to generalise from the simulation to the target domain under stark illumination differences. Our main contribution is an event-based satellite pose estimation technique, trained purely on synthetic event data with basic data augmentation to improve robustness against practical (noisy) event sensors. Underpinning our method is a novel dataset with carefully calibrated ground truth, comprising of real event data obtained by emulating satellite rendezvous scenarios in the lab under drastic lighting conditions. Results on the dataset showed that our event-based satellite pose estimation method, trained only on synthetic data without adaptation, could generalise to the target domain effectively. △ Less

Submitted 24 September, 2022; originally announced September 2022.

Comments: 8 pages. This work has been submitted to the IEEE (ICRA 2023) for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

QUIDAM: A Framework for Quantization-Aware DNN Accelerator and Model Co-Exploration

Authors: Ahmet Inci, Siri Garudanagiri Virupaksha, Aman Jain, Ting-Wu Chin, Venkata Vivek Thallam, Ruizhou Ding, Diana Marculescu

Abstract: As the machine learning and systems communities strive to achieve higher energy-efficiency through custom deep neural network (DNN) accelerators, varied precision or quantization levels, and model compression techniques, there is a need for design space exploration frameworks that incorporate quantization-aware processing elements into the accelerator design space while having accurate and fast po… ▽ More As the machine learning and systems communities strive to achieve higher energy-efficiency through custom deep neural network (DNN) accelerators, varied precision or quantization levels, and model compression techniques, there is a need for design space exploration frameworks that incorporate quantization-aware processing elements into the accelerator design space while having accurate and fast power, performance, and area models. In this work, we present QUIDAM, a highly parameterized quantization-aware DNN accelerator and model co-exploration framework. Our framework can facilitate future research on design space exploration of DNN accelerators for various design choices such as bit precision, processing element type, scratchpad sizes of processing elements, global buffer size, number of total processing elements, and DNN configurations. Our results show that different bit precisions and processing element types lead to significant differences in terms of performance per area and energy. Specifically, our framework identifies a wide range of design points where performance per area and energy varies more than 5x and 35x, respectively. With the proposed framework, we show that lightweight processing elements achieve on par accuracy results and up to 5.7x more performance per area and energy improvement when compared to the best INT16 based implementation. Finally, due to the efficiency of the pre-characterized power, performance, and area models, QUIDAM can speed up the design exploration process by 3-4 orders of magnitude as it removes the need for expensive synthesis and characterization of each design. △ Less

Submitted 30 June, 2022; originally announced June 2022.

Comments: 25 pages, 12 figures. arXiv admin note: substantial text overlap with arXiv:2205.13045, arXiv:2205.08648

Play It Cool: Dynamic Shifting Prevents Thermal Throttling

Authors: Yang Zhou, Feng Liang, Ting-wu Chin, Diana Marculescu

Abstract: Machine learning (ML) has entered the mobile era where an enormous number of ML models are deployed on edge devices. However, running common ML models on edge devices continuously may generate excessive heat from the computation, forcing the device to "slow down" to prevent overheating, a phenomenon called thermal throttling. This paper studies the impact of thermal throttling on mobile phones: wh… ▽ More Machine learning (ML) has entered the mobile era where an enormous number of ML models are deployed on edge devices. However, running common ML models on edge devices continuously may generate excessive heat from the computation, forcing the device to "slow down" to prevent overheating, a phenomenon called thermal throttling. This paper studies the impact of thermal throttling on mobile phones: when it occurs, the CPU clock frequency is reduced, and the model inference latency may increase dramatically. This unpleasant inconsistent behavior has a substantial negative effect on user experience, but it has been overlooked for a long time. To counter thermal throttling, we propose to utilize dynamic networks with shared weights and dynamically shift between large and small ML models seamlessly according to their thermal profile, i.e., shifting to a small model when the system is about to throttle. With the proposed dynamic shifting, the application runs consistently without experiencing CPU clock frequency degradation and latency increase. In addition, we also study the resulting accuracy when dynamic shifting is deployed and show that our approach provides a reasonable trade-off between model latency and model accuracy. △ Less

Submitted 8 July, 2022; v1 submitted 22 June, 2022; originally announced June 2022.

Comments: ICML DyNN Workshop 2022 Spotlight

Update Compression for Deep Neural Networks on the Edge

Authors: Bo Chen, Ali Bakhshi, Gustavo Batista, Brian Ng, Tat-Jun Chin

Abstract: An increasing number of artificial intelligence (AI) applications involve the execution of deep neural networks (DNNs) on edge devices. Many practical reasons motivate the need to update the DNN model on the edge device post-deployment, such as refining the model, concept drift, or outright change in the learning task. In this paper, we consider the scenario where retraining can be done on the ser… ▽ More An increasing number of artificial intelligence (AI) applications involve the execution of deep neural networks (DNNs) on edge devices. Many practical reasons motivate the need to update the DNN model on the edge device post-deployment, such as refining the model, concept drift, or outright change in the learning task. In this paper, we consider the scenario where retraining can be done on the server side based on a copy of the DNN model, with only the necessary data transmitted to the edge to update the deployed model. However, due to bandwidth constraints, we want to minimise the transmission required to achieve the update. We develop a simple approach based on matrix factorisation to compress the model update -- this differs from compressing the model itself. The key idea is to preserve existing knowledge in the current model and optimise only small additional parameters for the update which can be used to reconstitute the model on the edge. We compared our method to similar techniques used in federated learning; our method usually requires less than half of the update size of existing methods to achieve the same accuracy. △ Less

Submitted 21 April, 2022; v1 submitted 8 March, 2022; originally announced March 2022.

Comments: CVPR 2022 Mobile AI Workshop

Asynchronous Optimisation for Event-based Visual Odometry

Authors: Daqi Liu, Alvaro Parra, Yasir Latif, Bo Chen, Tat-Jun Chin, Ian Reid

Abstract: Event cameras open up new possibilities for robotic perception due to their low latency and high dynamic range. On the other hand, developing effective event-based vision algorithms that fully exploit the beneficial properties of event cameras remains work in progress. In this paper, we focus on event-based visual odometry (VO). While existing event-driven VO pipelines have adopted continuous-time… ▽ More Event cameras open up new possibilities for robotic perception due to their low latency and high dynamic range. On the other hand, developing effective event-based vision algorithms that fully exploit the beneficial properties of event cameras remains work in progress. In this paper, we focus on event-based visual odometry (VO). While existing event-driven VO pipelines have adopted continuous-time representations to asynchronously process event data, they either assume a known map, restrict the camera to planar trajectories, or integrate other sensors into the system. Towards map-free event-only monocular VO in SE(3), we propose an asynchronous structure-from-motion optimisation back-end. Our formulation is underpinned by a principled joint optimisation problem involving non-parametric Gaussian Process motion modelling and incremental maximum a posteriori inference. A high-performance incremental computation engine is employed to reason about the camera trajectory with every incoming event. We demonstrate the robustness of our asynchronous back-end in comparison to frame-based methods which depend on accurate temporal accumulation of measurements. △ Less

Submitted 2 March, 2022; originally announced March 2022.

Comments: 7 pages abd 5 figures, accepted to ICRA

A Hybrid Quantum-Classical Algorithm for Robust Fitting

Authors: Anh-Dzung Doan, Michele Sasdelli, David Suter, Tat-Jun Chin

Abstract: Fitting geometric models onto outlier contaminated data is provably intractable. Many computer vision systems rely on random sampling heuristics to solve robust fitting, which do not provide optimality guarantees and error bounds. It is therefore critical to develop novel approaches that can bridge the gap between exact solutions that are costly, and fast heuristics that offer no quality assurance… ▽ More Fitting geometric models onto outlier contaminated data is provably intractable. Many computer vision systems rely on random sampling heuristics to solve robust fitting, which do not provide optimality guarantees and error bounds. It is therefore critical to develop novel approaches that can bridge the gap between exact solutions that are costly, and fast heuristics that offer no quality assurances. In this paper, we propose a hybrid quantum-classical algorithm for robust fitting. Our core contribution is a novel robust fitting formulation that solves a sequence of integer programs and terminates with a global solution or an error bound. The combinatorial subproblems are amenable to a quantum annealer, which helps to tighten the bound efficiently. While our usage of quantum computing does not surmount the fundamental intractability of robust fitting, by providing error bounds our algorithm is a practical improvement over randomised heuristics. Moreover, our work represents a concrete application of quantum computing in computer vision. We present results obtained using an actual quantum computer (D-Wave Advantage) and via simulation. Source code: https://github.com/dadung/HQC-robust-fitting △ Less

Submitted 27 June, 2022; v1 submitted 25 January, 2022; originally announced January 2022.

Comments: IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) 2022

Direct reconstruction of tissue conductivity with deconvolution in magneto-acousto-electrical tomography (MAET): theory and numerical simulation

Authors: Tong Sun, Dingqian Deng, Linguo Yu, Yi Chen, Chien Ting Chin, Mian Chen, Chungi Chang, Siping Chen, Haoming Lin, Xin Chen

Abstract: Magneto-acousto-electrical tomography (MAET), a combination of ultrasound imaging and electrical impedance tomography (EIT), offers both high resolution (in comparison to EIT) and high contrast (in comparison to ultrasound imaging). It is used to map the internal conductivity distribution of an imaging object. However, conductivity reconstruction in MAET is a challenge, so conventional MAET is mai… ▽ More Magneto-acousto-electrical tomography (MAET), a combination of ultrasound imaging and electrical impedance tomography (EIT), offers both high resolution (in comparison to EIT) and high contrast (in comparison to ultrasound imaging). It is used to map the internal conductivity distribution of an imaging object. However, conductivity reconstruction in MAET is a challenge, so conventional MAET is mainly devoted to mapping the conductivity interface. This is primarily because integration byparts is used in the theory derivation, and the simplified measurement formula suggests the voltage is proportional to the conductivity gradient, which leads to an error in the measurement formula. In this study, the measurement signal is expressed as the convolution of acoustic velocity and conductivity distribution without using integration by parts, which retains the low-frequency term in the measurement signal. Based on the convolution formula, we subsequently propose a direct conductivity reconstruction scheme with deconvolution by utilizing the low-frequency component. We verify the proposed method based on two two-dimension models and quantify the L2 errors of reconstructed conductivity. Besides, we analyze factors influencing the reconstructed accuracy such as reconstructed regularization parameter ultrasound frequency, and noise. We also demonstrate that the spatial resolution is not influenced by the duration of excitation ultrasound. With the contributions of the proposed method, conductivity imaging appears to be feasible for application to the early diagnosis in the future. △ Less

Submitted 9 January, 2022; v1 submitted 5 January, 2022; originally announced January 2022.

Enabling equation-free modeling via diffusion maps

Authors: Tracy Chin, Jacob Ruth, Clayton Sanford, Rebecca Santorella, Paul Carter, Bjorn Sandstede

Abstract: Equation-free modeling aims at extracting low-dimensional macroscopic dynamics from complex high-dimensional systems that govern the evolution of microscopic states. This algorithm relies on lifting and restriction operators that map macroscopic states to microscopic states and vice versa. Combined with simulations of the microscopic state, this algorithm can be used to apply Newton solvers to the… ▽ More Equation-free modeling aims at extracting low-dimensional macroscopic dynamics from complex high-dimensional systems that govern the evolution of microscopic states. This algorithm relies on lifting and restriction operators that map macroscopic states to microscopic states and vice versa. Combined with simulations of the microscopic state, this algorithm can be used to apply Newton solvers to the implicitly defined low-dimensional macroscopic system or solve it more efficiently using direct numerical simulations. The key challenge is the construction of the lifting and restrictions operators that usually require a priori insight into the underlying application. In this paper, we design an application-independent algorithm that uses diffusion maps to construct these operators from simulation data. Code is available at https://doi.org/10.5281/zenodo.5793299. △ Less

Submitted 30 December, 2021; originally announced December 2021.

Comments: 19 pages, 16 figures, accepted for publication by Journal of Dynamics and Differential Equations

Adversarial Attacks against a Satellite-borne Multispectral Cloud Detector

Authors: Andrew Du, Yee Wei Law, Michele Sasdelli, Bo Chen, Ken Clarke, Michael Brown, Tat-Jun Chin

Abstract: Data collected by Earth-observing (EO) satellites are often afflicted by cloud cover. Detecting the presence of clouds -- which is increasingly done using deep learning -- is crucial preprocessing in EO applications. In fact, advanced EO satellites perform deep learning-based cloud detection on board the satellites and downlink only clear-sky data to save precious bandwidth. In this paper, we high… ▽ More Data collected by Earth-observing (EO) satellites are often afflicted by cloud cover. Detecting the presence of clouds -- which is increasingly done using deep learning -- is crucial preprocessing in EO applications. In fact, advanced EO satellites perform deep learning-based cloud detection on board the satellites and downlink only clear-sky data to save precious bandwidth. In this paper, we highlight the vulnerability of deep learning-based cloud detection towards adversarial attacks. By optimising an adversarial pattern and superimposing it into a cloudless scene, we bias the neural network into detecting clouds in the scene. Since the input spectra of cloud detectors include the non-visible bands, we generated our attacks in the multispectral domain. This opens up the potential of multi-objective attacks, specifically, adversarial biasing in the cloud-sensitive bands and visual camouflage in the visible bands. We also investigated mitigation strategies against the adversarial attacks. We hope our work further builds awareness of the potential of adversarial attacks in the EO community. △ Less

Submitted 3 December, 2021; originally announced December 2021.

Maximum Consensus by Weighted Influences of Monotone Boolean Functions

Authors: Erchuan Zhang, David Suter, Ruwan Tennakoon, Tat-Jun Chin, Alireza Bab-Hadiashar, Giang Truong, Syed Zulqarnain Gilani

Abstract: Robust model fitting is a fundamental problem in computer vision: used to pre-process raw data in the presence of outliers. Maximisation of Consensus (MaxCon) is one of the most popular robust criteria and widely used. Recently (Tennakoon et al. CVPR2021), a connection has been made between MaxCon and estimation of influences of a Monotone Boolean function. Equipping the Boolean cube with differen… ▽ More Robust model fitting is a fundamental problem in computer vision: used to pre-process raw data in the presence of outliers. Maximisation of Consensus (MaxCon) is one of the most popular robust criteria and widely used. Recently (Tennakoon et al. CVPR2021), a connection has been made between MaxCon and estimation of influences of a Monotone Boolean function. Equipping the Boolean cube with different measures and adopting different sampling strategies (two sides of the same coin) can have differing effects: which leads to the current study. This paper studies the concept of weighted influences for solving MaxCon. In particular, we study endowing the Boolean cube with the Bernoulli measure and performing biased (as opposed to uniform) sampling. Theoretically, we prove the weighted influences, under this measure, of points belonging to larger structures are smaller than those of points belonging to smaller structures in general. We also consider another "natural" family of sampling/weighting strategies, sampling with uniform measure concentrated on a particular (Hamming) level of the cube. Based on weighted sampling, we modify the algorithm of Tennakoon et al., and test on both synthetic and real datasets. This paper is not promoting a new approach per se, but rather studying the issue of weighted sampling. Accordingly, we are not claiming to have produced a superior algorithm: rather we show some modest gains of Bernoulli sampling, and we illuminate some of the interactions between structure in data and weighted sampling. △ Less

Submitted 6 March, 2022; v1 submitted 1 December, 2021; originally announced December 2021.

Occlusion-Robust Object Pose Estimation with Holistic Representation

Authors: Bo Chen, Tat-Jun Chin, Marius Klimavicius

Abstract: Practical object pose estimation demands robustness against occlusions to the target object. State-of-the-art (SOTA) object pose estimators take a two-stage approach, where the first stage predicts 2D landmarks using a deep network and the second stage solves for 6DOF pose from 2D-3D correspondences. Albeit widely adopted, such two-stage approaches could suffer from novel occlusions when generalis… ▽ More Practical object pose estimation demands robustness against occlusions to the target object. State-of-the-art (SOTA) object pose estimators take a two-stage approach, where the first stage predicts 2D landmarks using a deep network and the second stage solves for 6DOF pose from 2D-3D correspondences. Albeit widely adopted, such two-stage approaches could suffer from novel occlusions when generalising and weak landmark coherence due to disrupted features. To address these issues, we develop a novel occlude-and-blackout batch augmentation technique to learn occlusion-robust deep features, and a multi-precision supervision architecture to encourage holistic pose representation learning for accurate and coherent landmark predictions. We perform careful ablation tests to verify the impact of our innovations and compare our method to SOTA pose estimators. Without the need of any post-processing or refinement, our method exhibits superior performance on the LINEMOD dataset. On the YCB-Video dataset our method outperforms all non-refinement methods in terms of the ADD(-S) metric. We also demonstrate the high data-efficiency of our method. Our code is available at http://github.com/BoChenYS/ROPE △ Less

Submitted 22 October, 2021; originally announced October 2021.

Comments: WACV 2022

Autonomy and Perception for Space Mining

Authors: Ragav Sachdeva, Ravi Hammond, James Bockman, Alec Arthur, Brandon Smart, Dustin Craggs, Anh-Dzung Doan, Thomas Rowntree, Elijah Schutz, Adrian Orenstein, Andy Yu, Tat-Jun Chin, Ian Reid

Abstract: Future Moon bases will likely be constructed using resources mined from the surface of the Moon. The difficulty of maintaining a human workforce on the Moon and communications lag with Earth means that mining will need to be conducted using collaborative robots with a high degree of autonomy. In this paper, we describe our solution for Phase 2 of the NASA Space Robotics Challenge, which provided a… ▽ More Future Moon bases will likely be constructed using resources mined from the surface of the Moon. The difficulty of maintaining a human workforce on the Moon and communications lag with Earth means that mining will need to be conducted using collaborative robots with a high degree of autonomy. In this paper, we describe our solution for Phase 2 of the NASA Space Robotics Challenge, which provided a simulated lunar environment in which teams were tasked to develop software systems to achieve autonomous collaborative robots for mining on the Moon. Our 3rd place and innovation award winning solution shows how machine learning-enabled vision could alleviate major challenges posed by the lunar environment towards autonomous space mining, chiefly the lack of satellite positioning systems, hazardous terrain, and delicate robot interactions. A robust multi-robot coordinator was also developed to achieve long-term operation and effective collaboration between robots. △ Less

Submitted 13 April, 2022; v1 submitted 26 September, 2021; originally announced September 2021.

Comments: This paper describes our 3rd place and innovation award winning solution to the NASA Space Robotics Challenge Phase 2

Physical Adversarial Attacks on an Aerial Imagery Object Detector

Authors: Andrew Du, Bo Chen, Tat-Jun Chin, Yee Wei Law, Michele Sasdelli, Ramesh Rajasegaran, Dillon Campbell

Abstract: Deep neural networks (DNNs) have become essential for processing the vast amounts of aerial imagery collected using earth-observing satellite platforms. However, DNNs are vulnerable towards adversarial examples, and it is expected that this weakness also plagues DNNs for aerial imagery. In this work, we demonstrate one of the first efforts on physical adversarial attacks on aerial imagery, whereby… ▽ More Deep neural networks (DNNs) have become essential for processing the vast amounts of aerial imagery collected using earth-observing satellite platforms. However, DNNs are vulnerable towards adversarial examples, and it is expected that this weakness also plagues DNNs for aerial imagery. In this work, we demonstrate one of the first efforts on physical adversarial attacks on aerial imagery, whereby adversarial patches were optimised, fabricated and installed on or near target objects (cars) to significantly reduce the efficacy of an object detector applied on overhead images. Physical adversarial attacks on aerial images, particularly those captured from satellite platforms, are challenged by atmospheric factors (lighting, weather, seasons) and the distance between the observer and target. To investigate the effects of these challenges, we devised novel experiments and metrics to evaluate the efficacy of physical adversarial attacks against object detectors in aerial scenes. Our results indicate the palpable threat posed by physical adversarial attacks towards DNNs for processing satellite imagery. △ Less

Submitted 20 October, 2021; v1 submitted 26 August, 2021; originally announced August 2021.

Quantum Annealing Formulation for Binary Neural Networks

Authors: Michele Sasdelli, Tat-Jun Chin

Abstract: Quantum annealing is a promising paradigm for building practical quantum computers. Compared to other approaches, quantum annealing technology has been scaled up to a larger number of qubits. On the other hand, deep learning has been profoundly successful in pushing the boundaries of AI. It is thus natural to investigate potentially game changing technologies such as quantum annealers to augment t… ▽ More Quantum annealing is a promising paradigm for building practical quantum computers. Compared to other approaches, quantum annealing technology has been scaled up to a larger number of qubits. On the other hand, deep learning has been profoundly successful in pushing the boundaries of AI. It is thus natural to investigate potentially game changing technologies such as quantum annealers to augment the capabilities of deep learning. In this work, we explore binary neural networks, which are lightweight yet powerful models typically intended for resource constrained devices. Departing from current training regimes for binary networks that smooth/approximate the activation functions to make the network differentiable, we devise a quadratic unconstrained binary optimization formulation for the training problem. While the problem is intractable, i.e., the cost to estimate the binary weights scales exponentially with network size, we show how the problem can be optimized directly on a quantum annealer, thereby opening up to the potential gains of quantum computing. We experimentally validated our formulation via simulation and testing on an actual quantum annealer (D-Wave Advantage), the latter to the extent allowable by the capacity of current technology. △ Less

Submitted 4 July, 2021; originally announced July 2021.

Comments: 13 pages, 4 figures

A Spacecraft Dataset for Detection, Segmentation and Parts Recognition

Authors: Dung Anh Hoang, Bo Chen, Tat-Jun Chin

Abstract: Virtually all aspects of modern life depend on space technology. Thanks to the great advancement of computer vision in general and deep learning-based techniques in particular, over the decades, the world witnessed the growing use of deep learning in solving problems for space applications, such as self-driving robot, tracers, insect-like robot on cosmos and health monitoring of spacecraft. These… ▽ More Virtually all aspects of modern life depend on space technology. Thanks to the great advancement of computer vision in general and deep learning-based techniques in particular, over the decades, the world witnessed the growing use of deep learning in solving problems for space applications, such as self-driving robot, tracers, insect-like robot on cosmos and health monitoring of spacecraft. These are just some prominent examples that has advanced space industry with the help of deep learning. However, the success of deep learning models requires a lot of training data in order to have decent performance, while on the other hand, there are very limited amount of publicly available space datasets for the training of deep learning models. Currently, there is no public datasets for space-based object detection or instance segmentation, partly because manually annotating object segmentation masks is very time consuming as they require pixel-level labelling, not to mention the challenge of obtaining images from space. In this paper, we aim to fill this gap by releasing a dataset for spacecraft detection, instance segmentation and part recognition. The main contribution of this work is the development of the dataset using images of space stations and satellites, with rich annotations including bounding boxes of spacecrafts and masks to the level of object parts, which are obtained with a mixture of automatic processes and manual efforts. We also provide evaluations with state-of-the-art methods in object detection and instance segmentation as a benchmark for the dataset. The link for downloading the proposed dataset can be found on https://github.com/Yurushia1998/SatelliteDataset. △ Less

Submitted 15 June, 2021; originally announced June 2021.

Learning to Predict Repeatability of Interest Points

Authors: Anh-Dzung Doan, Daniyar Turmukhambetov, Yasir Latif, Tat-Jun Chin, Soohyun Bae

Abstract: Many robotics applications require interest points that are highly repeatable under varying viewpoints and lighting conditions. However, this requirement is very challenging as the environment changes continuously and indefinitely, leading to appearance changes of interest points with respect to time. This paper proposes to predict the repeatability of an interest point as a function of time, whic… ▽ More Many robotics applications require interest points that are highly repeatable under varying viewpoints and lighting conditions. However, this requirement is very challenging as the environment changes continuously and indefinitely, leading to appearance changes of interest points with respect to time. This paper proposes to predict the repeatability of an interest point as a function of time, which can tell us the lifespan of the interest point considering daily or seasonal variation. The repeatability predictor (RP) is formulated as a regressor trained on repeated interest points from multiple viewpoints over a long period of time. Through comprehensive experiments, we demonstrate that our RP can estimate when a new interest point is repeated, and also highlight an insightful analysis about this problem. For further comparison, we apply our RP to the map summarization under visual localization framework, which builds a compact representation of the full context map given the query time. The experimental result shows a careful selection of potentially repeatable interest points predicted by our RP can significantly mitigate the degeneration of localization accuracy from map summarization. △ Less

Submitted 7 May, 2021; originally announced May 2021.

Comments: Accepted at IEEE International Conference on Robotics and Automation (ICRA) 2021

Width Transfer: On the (In)variance of Width Optimization

Authors: Ting-Wu Chin, Diana Marculescu, Ari S. Morcos

Abstract: Optimizing the channel counts for different layers of a CNN has shown great promise in improving the efficiency of CNNs at test-time. However, these methods often introduce large computational overhead (e.g., an additional 2x FLOPs of standard training). Minimizing this overhead could therefore significantly speed up training. In this work, we propose width transfer, a technique that harnesses the… ▽ More Optimizing the channel counts for different layers of a CNN has shown great promise in improving the efficiency of CNNs at test-time. However, these methods often introduce large computational overhead (e.g., an additional 2x FLOPs of standard training). Minimizing this overhead could therefore significantly speed up training. In this work, we propose width transfer, a technique that harnesses the assumptions that the optimized widths (or channel counts) are regular across sizes and depths. We show that width transfer works well across various width optimization algorithms and networks. Specifically, we can achieve up to 320x reduction in width optimization overhead without compromising the top-1 accuracy on ImageNet, making the additional cost of width optimization negligible relative to initial training. Our findings not only suggest an efficient way to conduct width optimization but also highlight that the widths that lead to better accuracy are invariant to various aspects of network architectures and training data. △ Less

Submitted 24 April, 2021; originally announced April 2021.

Comments: Full paper accepted at CVPR Workshops 2021; a 4-page abridged version is accepted at ICLR 2021 NAS Workshop

Rotation Coordinate Descent for Fast Globally Optimal Rotation Averaging

Authors: Álvaro Parra, Shin-Fang Chng, Tat-Jun Chin, Anders Eriksson, Ian Reid

Abstract: Under mild conditions on the noise level of the measurements, rotation averaging satisfies strong duality, which enables global solutions to be obtained via semidefinite programming (SDP) relaxation. However, generic solvers for SDP are rather slow in practice, even on rotation averaging instances of moderate size, thus developing specialised algorithms is vital. In this paper, we present a fast a… ▽ More Under mild conditions on the noise level of the measurements, rotation averaging satisfies strong duality, which enables global solutions to be obtained via semidefinite programming (SDP) relaxation. However, generic solvers for SDP are rather slow in practice, even on rotation averaging instances of moderate size, thus developing specialised algorithms is vital. In this paper, we present a fast algorithm that achieves global optimality called rotation coordinate descent (RCD). Unlike block coordinate descent (BCD) which solves SDP by updating the semidefinite matrix in a row-by-row fashion, RCD directly maintains and updates all valid rotations throughout the iterations. This obviates the need to store a large dense semidefinite matrix. We mathematically prove the convergence of our algorithm and empirically show its superior efficiency over state-of-the-art global methods on a variety of problem configurations. Maintaining valid rotations also facilitates incorporating local optimisation routines for further speed-ups. Moreover, our algorithm is simple to implement; see supplementary material for a demonstration program. △ Less

Submitted 15 March, 2021; v1 submitted 15 March, 2021; originally announced March 2021.

Comments: Accepted to CVPR 2021 as an oral presentation

Spatiotemporal Registration for Event-based Visual Odometry

Authors: Daqi Liu, Alvaro Parra, Tat-Jun Chin

Abstract: A useful application of event sensing is visual odometry, especially in settings that require high-temporal resolution. The state-of-the-art method of contrast maximisation recovers the motion from a batch of events by maximising the contrast of the image of warped events. However, the cost scales with image resolution and the temporal resolution can be limited by the need for large batch sizes to… ▽ More A useful application of event sensing is visual odometry, especially in settings that require high-temporal resolution. The state-of-the-art method of contrast maximisation recovers the motion from a batch of events by maximising the contrast of the image of warped events. However, the cost scales with image resolution and the temporal resolution can be limited by the need for large batch sizes to yield sufficient structure in the contrast image. In this work, we propose spatiotemporal registration as a compelling technique for event-based rotational motion estimation. We theoretcally justify the approach and establish its fundamental and practical advantages over contrast maximisation. In particular, spatiotemporal registration also produces feature tracks as a by-product, which directly supports an efficient visual odometry pipeline with graph-based optimisation for motion averaging. The simplicity of our visual odometry pipeline allows it to process more than 1 M events/second. We also contribute a new event dataset for visual odometry, where motion sequences with large velocity variations were acquired using a high-precision robot arm. △ Less

Submitted 18 March, 2021; v1 submitted 10 March, 2021; originally announced March 2021.

Comments: 10 pages

Consensus Maximisation Using Influences of Monotone Boolean Functions

Authors: Ruwan Tennakoon, David Suter, Erchuan Zhang, Tat-Jun Chin, Alireza Bab-Hadiashar

Abstract: Consensus maximisation (MaxCon), which is widely used for robust fitting in computer vision, aims to find the largest subset of data that fits the model within some tolerance level. In this paper, we outline the connection between MaxCon problem and the abstract problem of finding the maximum upper zero of a Monotone Boolean Function (MBF) defined over the Boolean Cube. Then, we link the concept o… ▽ More Consensus maximisation (MaxCon), which is widely used for robust fitting in computer vision, aims to find the largest subset of data that fits the model within some tolerance level. In this paper, we outline the connection between MaxCon problem and the abstract problem of finding the maximum upper zero of a Monotone Boolean Function (MBF) defined over the Boolean Cube. Then, we link the concept of influences (in a MBF) to the concept of outlier (in MaxCon) and show that influences of points belonging to the largest structure in data would generally be smaller under certain conditions. Based on this observation, we present an iterative algorithm to perform consensus maximisation. Results for both synthetic and real visual data experiments show that the MBF based algorithm is capable of generating a near optimal solution relatively quickly. This is particularly important where there are large number of outliers (gross or pseudo) in the observed data. △ Less

Submitted 6 March, 2021; originally announced March 2021.

Comments: To appear in CVPR 2021 as an ORAL paper. arXiv admin note: text overlap with arXiv:2005.05490

Semantics for Robotic Mapping, Perception and Interaction: A Survey

Authors: Sourav Garg, Niko Sünderhauf, Feras Dayoub, Douglas Morrison, Akansel Cosgun, Gustavo Carneiro, Qi Wu, Tat-Jun Chin, Ian Reid, Stephen Gould, Peter Corke, Michael Milford

Abstract: For robots to navigate and interact more richly with the world around them, they will likely require a deeper understanding of the world in which they operate. In robotics and related research fields, the study of understanding is often referred to as semantics, which dictates what does the world "mean" to a robot, and is strongly tied to the question of how to represent that meaning. With humans… ▽ More For robots to navigate and interact more richly with the world around them, they will likely require a deeper understanding of the world in which they operate. In robotics and related research fields, the study of understanding is often referred to as semantics, which dictates what does the world "mean" to a robot, and is strongly tied to the question of how to represent that meaning. With humans and robots increasingly operating in the same world, the prospects of human-robot interaction also bring semantics and ontology of natural language into the picture. Driven by need, as well as by enablers like increasing availability of training data and computational resources, semantics is a rapidly growing research area in robotics. The field has received significant attention in the research literature to date, but most reviews and surveys have focused on particular aspects of the topic: the technical research issues regarding its use in specific robotic topics like mapping or segmentation, or its relevance to one particular application domain like autonomous driving. A new treatment is therefore required, and is also timely because so much relevant research has occurred since many of the key surveys were published. This survey therefore provides an overarching snapshot of where semantics in robotics stands today. We establish a taxonomy for semantics research in or relevant to robotics, split into four broad categories of activity, in which semantics are extracted, used, or both. Within these broad categories we survey dozens of major topics including fundamentals from the computer vision field and key robotics research areas utilizing semantics, including mapping, navigation and interaction with the world. The survey also covers key practical considerations, including enablers like increased data availability and improved computational hardware, and major application areas where... △ Less

Submitted 2 January, 2021; originally announced January 2021.

Comments: 81 pages, 1 figure, published in Foundations and Trends in Robotics, 2020

Journal ref: Foundations and Trends in Robotics: Vol. 8: No. 1-2, pp 1-224 (2020)

HM4: Hidden Markov Model with Memory Management for Visual Place Recognition

Authors: Anh-Dzung Doan, Yasir Latif, Tat-Jun Chin, Ian Reid

Abstract: Visual place recognition needs to be robust against appearance variability due to natural and man-made causes. Training data collection should thus be an ongoing process to allow continuous appearance changes to be recorded. However, this creates an unboundedly-growing database that poses time and memory scalability challenges for place recognition methods. To tackle the scalability issue for visu… ▽ More Visual place recognition needs to be robust against appearance variability due to natural and man-made causes. Training data collection should thus be an ongoing process to allow continuous appearance changes to be recorded. However, this creates an unboundedly-growing database that poses time and memory scalability challenges for place recognition methods. To tackle the scalability issue for visual place recognition in autonomous driving, we develop a Hidden Markov Model approach with a two-tiered memory management. Our algorithm, dubbed HM$^4$, exploits temporal look-ahead to transfer promising candidate images between passive storage and active memory when needed. The inference process takes into account both promising images and a coarse representations of the full database. We show that this allows constant time and space inference for a fixed coverage area. The coarse representations can also be updated incrementally to absorb new data. To further reduce the memory requirements, we derive a compact image representation inspired by Locality Sensitive Hashing (LSH). Through experiments on real world data, we demonstrate the excellent scalability and accuracy of the approach under appearance changes and provide comparisons against state-of-the-art techniques. △ Less

Submitted 1 November, 2020; originally announced November 2020.

Comments: Accepted for publication by IEEE Robotics and Automation Letters

Monocular Rotational Odometry with Incremental Rotation Averaging and Loop Closure

Authors: Chee-Kheng Chng, Alvaro Parra, Tat-Jun Chin, Yasir Latif

Abstract: Estimating absolute camera orientations is essential for attitude estimation tasks. An established approach is to first carry out visual odometry (VO) or visual SLAM (V-SLAM), and retrieve the camera orientations (3 DOF) from the camera poses (6 DOF) estimated by VO or V-SLAM. One drawback of this approach, besides the redundancy in estimating full 6 DOF camera poses, is the dependency on estimati… ▽ More Estimating absolute camera orientations is essential for attitude estimation tasks. An established approach is to first carry out visual odometry (VO) or visual SLAM (V-SLAM), and retrieve the camera orientations (3 DOF) from the camera poses (6 DOF) estimated by VO or V-SLAM. One drawback of this approach, besides the redundancy in estimating full 6 DOF camera poses, is the dependency on estimating a map (3D scene points) jointly with the 6 DOF poses due to the basic constraint on structure-and-motion. To simplify the task of absolute orientation estimation, we formulate the monocular rotational odometry problem and devise a fast algorithm to accurately estimate camera orientations with 2D-2D feature matches alone. Underpinning our system is a new incremental rotation averaging method for fast and constant time iterative updating. Furthermore, our system maintains a view-graph that 1) allows solving loop closure to remove camera orientation drift, and 2) can be used to warm start a V-SLAM system. We conduct extensive quantitative experiments on real-world datasets to demonstrate the accuracy of our incremental camera orientation solver. Finally, we showcase the benefit of our algorithm to V-SLAM: 1) solving the known rotation problem to estimate the trajectory of the camera and the surrounding map, and 2)enabling V-SLAM systems to track pure rotational motions. △ Less

Submitted 5 October, 2020; originally announced October 2020.

Comments: Accepted to DICTA 2020