Showing 1–8 of 8 results for author: Pearl, N

Estimating Galaxy Parameters with Self-Organizing Maps and the Effect of Missing Data

Authors: Valentina La Torre, Anna Sajina, Andy D. Goulding, Danilo Marchesini, Rachel Bezanson, Alan N. Pearl, Laerte Sodré Jr

Abstract: The current and upcoming large data volume galaxy surveys require the use of machine learning techniques to maximize their scientific return. This study explores the use of Self-Organizing Maps (SOMs) to estimate galaxy parameters with a focus on handling cases of missing data and providing realistic probability distribution functions for the parameters. We train a SOM with a simulated mass-limite… ▽ More The current and upcoming large data volume galaxy surveys require the use of machine learning techniques to maximize their scientific return. This study explores the use of Self-Organizing Maps (SOMs) to estimate galaxy parameters with a focus on handling cases of missing data and providing realistic probability distribution functions for the parameters. We train a SOM with a simulated mass-limited lightcone assuming a ugrizYJHKs+IRAC dataset, mimicking the Hyper Suprime-Cam (HSC) Deep joint dataset. For parameter estimation, we derive SOM likelihood surfaces considering photometric errors to derive total (statistical and systematic) uncertainties. We explore the effects of missing data including which bands are particular critical to the accuracy of the derived parameters. We demonstrate that the parameter recovery is significantly better when the missing bands are "filled-in" rather than if they are completely omitted. We propose a practical method for such recovery of missing data. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Comments: 22 pages, 12 figures, 1 table, accepted for publication in The Astronomical Journal

The DESI One-Percent Survey: Evidence for Assembly Bias from Low-Redshift Counts-in-Cylinders Measurements

Authors: Alan N. Pearl, Andrew R. Zentner, Jeffrey A. Newman, Rachel Bezanson, Kuan Wang, John Moustakas, Jessica N. Aguilar, Steven Ahlen, David Brooks, Todd Claybaugh, Shaun Cole, Kyle Dawson, Axel de la Macorra, Peter Doel, Jamie E. Forero-Romero, Satya Gontcho A Gontcho, Klaus Honscheid, Martin Landriau, Marc Manera, Paul Martini Aaron Meisner, Ramon Miquel, Jundan Nie, Will Percival, Francisco Prada, Mehdi Rezaie , et al. (6 additional authors not shown)

Abstract: We explore the galaxy-halo connection information that is available in low-redshift samples from the early data release of the Dark Energy Spectroscopic Instrument (DESI). We model the halo occupation distribution (HOD) from z=0.1-0.3 using Survey Validation 3 (SV3; a.k.a., the One-Percent Survey) data of the DESI Bright Galaxy Survey (BGS). In addition to more commonly used metrics, we incorporat… ▽ More We explore the galaxy-halo connection information that is available in low-redshift samples from the early data release of the Dark Energy Spectroscopic Instrument (DESI). We model the halo occupation distribution (HOD) from z=0.1-0.3 using Survey Validation 3 (SV3; a.k.a., the One-Percent Survey) data of the DESI Bright Galaxy Survey (BGS). In addition to more commonly used metrics, we incorporate counts-in-cylinders (CiC) measurements, which drastically tighten HOD constraints. Our analysis is aided by the Python package, galtab, which enables the rapid, precise prediction of CiC for any HOD model available in halotools. This methodology allows our Markov chains to converge with much fewer trial points, and enables even more drastic speedups due to its GPU portability. Our HOD fits constrain characteristic halo masses tightly and provide statistical evidence for assembly bias, especially at lower luminosity thresholds: the HOD of central galaxies in $z\sim0.15$ samples with limiting absolute magnitude $M_r < -20.0$ and $M_r < -20.5$ samples is positively correlated with halo concentration with a significance of 99.9% and 99.5%, respectively. Our models also favor positive central assembly bias for the brighter $M_r < -21.0$ sample at $z\sim0.25$ (94.8% significance), but there is no significant evidence for assembly bias with the same luminosity threshold at $z\sim0.15$. We provide our constraints for each threshold sample's characteristic halo masses, assembly bias, and other HOD parameters. These constraints are expected to be significantly tightened with future DESI data, which will span an area 100 times larger than that of SV3. △ Less

Submitted 15 September, 2023; originally announced September 2023.

SVNR: Spatially-variant Noise Removal with Denoising Diffusion

Authors: Naama Pearl, Yaron Brodsky, Dana Berman, Assaf Zomet, Alex Rav Acha, Daniel Cohen-Or, Dani Lischinski

Abstract: Denoising diffusion models have recently shown impressive results in generative tasks. By learning powerful priors from huge collections of training images, such models are able to gradually modify complete noise to a clean natural image via a sequence of small denoising steps, seemingly making them well-suited for single image denoising. However, effectively applying denoising diffusion models to… ▽ More Denoising diffusion models have recently shown impressive results in generative tasks. By learning powerful priors from huge collections of training images, such models are able to gradually modify complete noise to a clean natural image via a sequence of small denoising steps, seemingly making them well-suited for single image denoising. However, effectively applying denoising diffusion models to removal of realistic noise is more challenging than it may seem, since their formulation is based on additive white Gaussian noise, unlike noise in real-world images. In this work, we present SVNR, a novel formulation of denoising diffusion that assumes a more realistic, spatially-variant noise model. SVNR enables using the noisy input image as the starting point for the denoising diffusion process, in addition to conditioning the process on it. To this end, we adapt the diffusion process to allow each pixel to have its own time embedding, and propose training and inference schemes that support spatially-varying time maps. Our formulation also accounts for the correlation that exists between the condition image and the samples along the modified diffusion process. In our experiments we demonstrate the advantages of our approach over a strong diffusion model baseline, as well as over a state-of-the-art single image denoising method. △ Less

Submitted 28 June, 2023; originally announced June 2023.

SeaThru-NeRF: Neural Radiance Fields in Scattering Media

Authors: Deborah Levy, Amit Peleg, Naama Pearl, Dan Rosenbaum, Derya Akkaynak, Simon Korman, Tali Treibitz

Abstract: Research on neural radiance fields (NeRFs) for novel view generation is exploding with new models and extensions. However, a question that remains unanswered is what happens in underwater or foggy scenes where the medium strongly influences the appearance of objects. Thus far, NeRF and its variants have ignored these cases. However, since the NeRF framework is based on volumetric rendering, it has… ▽ More Research on neural radiance fields (NeRFs) for novel view generation is exploding with new models and extensions. However, a question that remains unanswered is what happens in underwater or foggy scenes where the medium strongly influences the appearance of objects. Thus far, NeRF and its variants have ignored these cases. However, since the NeRF framework is based on volumetric rendering, it has inherent capability to account for the medium's effects, once modeled appropriately. We develop a new rendering model for NeRFs in scattering media, which is based on the SeaThru image formation model, and suggest a suitable architecture for learning both scene information and medium parameters. We demonstrate the strength of our method using simulated and real-world scenes, correctly rendering novel photorealistic views underwater. Even more excitingly, we can render clear views of these scenes, removing the medium between the camera and the scene and reconstructing the appearance and depth of far objects, which are severely occluded by the medium. Our code and unique datasets are available on the project's website. △ Less

Submitted 16 April, 2023; originally announced April 2023.

NAN: Noise-Aware NeRFs for Burst-Denoising

Authors: Naama Pearl, Tali Treibitz, Simon Korman

Abstract: Burst denoising is now more relevant than ever, as computational photography helps overcome sensitivity issues inherent in mobile phones and small cameras. A major challenge in burst-denoising is in coping with pixel misalignment, which was so far handled with rather simplistic assumptions of simple motion, or the ability to align in pre-processing. Such assumptions are not realistic in the presen… ▽ More Burst denoising is now more relevant than ever, as computational photography helps overcome sensitivity issues inherent in mobile phones and small cameras. A major challenge in burst-denoising is in coping with pixel misalignment, which was so far handled with rather simplistic assumptions of simple motion, or the ability to align in pre-processing. Such assumptions are not realistic in the presence of large motion and high levels of noise. We show that Neural Radiance Fields (NeRFs), originally suggested for physics-based novel-view rendering, can serve as a powerful framework for burst denoising. NeRFs have an inherent capability of handling noise as they integrate information from multiple images, but they are limited in doing so, mainly since they build on pixel-wise operations which are suitable to ideal imaging conditions. Our approach, termed NAN, leverages inter-view and spatial information in NeRFs to better deal with noise. It achieves state-of-the-art results in burst denoising and is especially successful in coping with large movement and occlusions, under very high levels of noise. With the rapid advances in accelerating NeRFs, it could provide a powerful platform for denoising in challenging environments. △ Less

Submitted 12 April, 2022; v1 submitted 10 April, 2022; originally announced April 2022.

Comments: to appear at CVPR 2022

CLIMBER: Galaxy-Halo Connection Constraints from Next-Generation Surveys

Authors: Alan N. Pearl, Rachel Bezanson, Andrew R. Zentner, Jeffrey A. Newman, Andy D. Goulding, Katherine E. Whitaker, Sean D. Johnson, Jenny E. Greene

Abstract: In the coming decade, a new generation of massively multiplexed spectroscopic surveys, such as PFS, WAVES, and MOONS, will probe galaxies in the distant universe in vastly greater numbers than was previously possible. In this work, we generate mock catalogs for each of these three planned surveys to help quantify and optimize their scientific output. To assign photometry into the UniverseMachine e… ▽ More In the coming decade, a new generation of massively multiplexed spectroscopic surveys, such as PFS, WAVES, and MOONS, will probe galaxies in the distant universe in vastly greater numbers than was previously possible. In this work, we generate mock catalogs for each of these three planned surveys to help quantify and optimize their scientific output. To assign photometry into the UniverseMachine empirical model, we develop the Calibrating Light: Illuminating Mocks By Empirical Relations (CLIMBER) procedure using UltraVISTA photometry. Using the published empirical selection functions for each aforementioned survey, we quantify the mass completeness of each survey. We compare different targeting strategies by varying the area and targeting completeness, and quantify how these survey parameters affect the uncertainty of the two-point correlation function. We demonstrate that the PFS and MOONS measurements will be primarily dominated by cosmic variance, not shot noise, motivating the need for increasingly large survey areas. On the other hand, the WAVES survey, which covers a much larger area, will strike a good balance between cosmic variance and shot noise. For a fixed number of targets, a 5% increased survey area (and $\sim$5% decreased completeness) would decrease the uncertainty of the correlation function at intermediate scales by 0.15%, 1.2%, and 1.1% for our WAVES, PFS, and MOONS samples, respectively. Meanwhile, for a fixed survey area, 5% increased targeting completeness improves the same constraints by 0.7%, 0.25%, and 0.1%. All of the utilities used to construct our mock catalogs and many of the catalogs themselves are publicly available. △ Less

Submitted 30 November, 2021; originally announced December 2021.

Metalearning Linear Bandits by Prior Update

Authors: Amit Peleg, Naama Pearl, Ron Meir

Abstract: Fully Bayesian approaches to sequential decision-making assume that problem parameters are generated from a known prior. In practice, such information is often lacking. This problem is exacerbated in setups with partial information, where a misspecified prior may lead to poor exploration and performance. In this work we prove, in the context of stochastic linear bandits and Gaussian priors, that a… ▽ More Fully Bayesian approaches to sequential decision-making assume that problem parameters are generated from a known prior. In practice, such information is often lacking. This problem is exacerbated in setups with partial information, where a misspecified prior may lead to poor exploration and performance. In this work we prove, in the context of stochastic linear bandits and Gaussian priors, that as long as the prior is sufficiently close to the true prior, the performance of the applied algorithm is close to that of the algorithm that uses the true prior. Furthermore, we address the task of learning the prior through metalearning, where a learner updates her estimate of the prior across multiple task instances in order to improve performance on future tasks. We provide an algorithm and regret bounds, demonstrate its effectiveness in comparison to an algorithm that knows the correct prior, and support our theoretical results empirically. Our theoretical results hold for a broad class of algorithms, including Thompson Sampling and Information Directed Sampling. △ Less

Submitted 2 March, 2022; v1 submitted 12 July, 2021; originally announced July 2021.

Journal ref: Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (AISTATS), 2022

A Map of the Local Velocity Substructure in the Milky Way Disk

Authors: Alan N. Pearl, Heidi Jo Newberg, Jeffrey L. Carlin, R. Fiona Smith

Abstract: We confirm, quantify, and provide a table of the coherent velocity substructure of the Milky Way disk within 2 kpc of the Sun towards the Galactic anticenter, with 0.2 kpc resolution. We use the radial velocities of ~340,000 F-type stars obtained with the Guoshoujing Telescope (also known as the Large Sky Area Multi-Object Fiber Spectroscopic Telescope, LAMOST), and proper motions derived from the… ▽ More We confirm, quantify, and provide a table of the coherent velocity substructure of the Milky Way disk within 2 kpc of the Sun towards the Galactic anticenter, with 0.2 kpc resolution. We use the radial velocities of ~340,000 F-type stars obtained with the Guoshoujing Telescope (also known as the Large Sky Area Multi-Object Fiber Spectroscopic Telescope, LAMOST), and proper motions derived from the PPMXL catalog. The PPMXL proper motions have been corrected to remove systematic errors by subtracting the average proper motions of galaxies and QSOs that have been confirmed in the LAMOST spectroscopic survey, and that are within 2.5 degrees of the star's position. We provide the resulting table of systematic offsets derived from the PPMXL proper motion measurements of extragalactic objects identified in the LAMOST spectroscopic survey. Using the corrected phase- space stellar sample, we find statistically significant deviations in the bulk disk velocity of 20 km/s or more in the three dimensional velocities of Galactic disk stars. The bulk velocity varies significantly over length scales of half a kpc or less. The rotation velocity of the disk increases by 20 km/s from the Sun's position to 1.5 kpc outside the solar circle. Disk stars in the second quadrant, within 1 kpc of the Sun, are moving radially towards the Galactic center and vertically towards a point a few tenths of a kpc above the Galactic plane; looking down on the disk, the stars appear to move in a circular streaming motion with a radius of order 1 kpc. △ Less

Submitted 15 August, 2017; v1 submitted 11 August, 2017; originally announced August 2017.

Comments: 20 pages, 13 figures, accepted by ApJ. v2 - Fixed typo in title and added two references