no code implementations • 12 Dec 2023 • Rohan Deb, Yikun Ban, Shiliang Zuo, Jingrui He, Arindam Banerjee
Based on such a perturbed prediction, we show a ${\mathcal{O}}(\log T)$ regret for online regression with both squared loss and KL loss, and subsequently convert these respectively to $\tilde{\mathcal{O}}(\sqrt{KT})$ and $\tilde{\mathcal{O}}(\sqrt{KL^*} + K)$ regret for NeuCB, where $L^*$ is the loss of the best policy.
no code implementations • 9 Sep 2023 • Varun A. Kelkar, Rucha Deshpande, Arindam Banerjee, Mark A. Anastasio
In applications such as computed imaging, it is often difficult to acquire such data due to requirements such as long acquisition time or high radiation dose, while acquiring noisy or partially observed measurements of these objects is more feasible.
2 code implementations • NeurIPS 2023 • Adam J. Stewart, Nils Lehmann, Isaac A. Corley, Yi Wang, Yi-Chia Chang, Nassim Ait Ali Braham, Shradha Sehgal, Caleb Robinson, Arindam Banerjee
The Landsat program is the longest-running Earth observation program in history, with 50+ years of data acquisition by 8 satellites.
1 code implementation • 5 May 2023 • Yikun Ban, Yuchen Yan, Arindam Banerjee, Jingrui He
In recent literature, a series of neural bandit algorithms have been proposed to adapt to the non-linear reward function, combined with TS or UCB strategies for exploration.
1 code implementation • 17 Mar 2023 • Mathew Mithra Noel, Arindam Banerjee, Geraldine Bessie Amali D, Venkataraman Muthiah-Nakarajan
Two new classification loss functions that significantly improve performance on a wide variety of benchmark tasks are proposed.
1 code implementation • 2 Oct 2022 • Yikun Ban, Yuheng Zhang, Hanghang Tong, Arindam Banerjee, Jingrui He
We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting.
no code implementations • 29 Sep 2022 • Arindam Banerjee, Pedro Cisneros-Velarde, Libin Zhu, Mikhail Belkin
Second, we introduce a new analysis of optimization based on Restricted Strong Convexity (RSC) which holds as long as the squared norm of the average gradient of predictors is $\Omega(\frac{\text{poly}(L)}{\sqrt{m}})$ for the square loss.
no code implementations • 9 Jan 2022 • Arindam Banerjee, Tiancong Chen, Xinyan Li, Yingxue Zhou
Recent years have seen advances in generalization bounds for noisy stochastic algorithms, especially stochastic gradient Langevin dynamics (SGLD) based on stability (Mou et al., 2018; Li et al., 2020) and information theoretic approaches (Xu and Raginsky, 2017; Negrea et al., 2019; Steinke and Zakynthinou, 2020).
1 code implementation • 17 Nov 2021 • Adam J. Stewart, Caleb Robinson, Isaac A. Corley, Anthony Ortiz, Juan M. Lavista Ferres, Arindam Banerjee
Deep learning methods are particularly promising for modeling many remote sensing tasks given the success of deep neural networks in similar computer vision tasks and the sheer volume of remotely sensed imagery available.
1 code implementation • ICLR 2022 • Yikun Ban, Yuchen Yan, Arindam Banerjee, Jingrui He
To overcome this challenge, a series of neural bandit algorithms have been proposed, where a neural network is used to learn the underlying reward function and TS or UCB are adapted for exploration.
1 code implementation • 29 Sep 2021 • Sijie He, Xinyan Li, Laurie Trenary, Benjamin A Cash, Timothy DelSole, Arindam Banerjee
The SSF dataset constructed for the work, dynamical model predictions, and code for the ML models are released along with the paper for the benefit of the broader machine learning community.
no code implementations • 26 Feb 2021 • Xinyan Li, Arindam Banerjee
Inspired by this, we investigate probabilistic LWS-SGD, which mostly updates the top layers and occasionally updates the full network.
no code implementations • 26 Feb 2021 • Yingxue Zhou, Xinyan Li, Arindam Banerjee
Our experiments on a variety of benchmark datasets (MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100) with various networks (VGG and ResNet) validate the theoretical properties of NT-SGD, i. e., NT-SGD matches the speed and accuracy of vanilla SGD while effectively working with sparse gradients, and can successfully escape poor local minima.
no code implementations • ICLR 2021 • Yingxue Zhou, Zhiwei Steven Wu, Arindam Banerjee
Existing lower bounds on private ERM show that such dependence on $p$ is inevitable in the worst case.
no code implementations • 24 Jun 2020 • Yingxue Zhou, Xiangyi Chen, Mingyi Hong, Zhiwei Steven Wu, Arindam Banerjee
We obtain this rate by providing the first analyses on a collection of private gradient-based methods, including adaptive algorithms DP RMSProp and DP Adam.
no code implementations • 14 Jun 2020 • Sijie He, Xinyan Li, Timothy DelSole, Pradeep Ravikumar, Arindam Banerjee
Sub-seasonal climate forecasting (SSF) focuses on predicting key climate variables such as temperature and precipitation in the 2-week to 2-month time scales.
1 code implementation • NeurIPS 2020 • Robert Giaquinto, Arindam Banerjee
We propose an alternative: Gradient Boosted Normalizing Flows (GBNF) model a density by successively adding new NF components with gradient boosting.
no code implementations • ICML 2020 • Vidyashankar Sivakumar, Zhiwei Steven Wu, Arindam Banerjee
Bandit learning algorithms typically involve the balance of exploration and exploitation.
no code implementations • 23 Feb 2020 • Arindam Banerjee, Tiancong Chen, Yingxue Zhou
Existing approaches for deterministic non-smooth deep nets typically need to bound the Lipschitz constant of such deep nets but such bounds are quite large, may even increase with the training set size yielding vacuous generalization bounds.
no code implementations • NeurIPS 2019 • Arindam Banerjee, Qilong Gu, Vidyashankar Sivakumar, Zhiwei Steven Wu
We also discuss stochastic process based forms of J-L, RIP, and sketching, to illustrate the generality of the results.
no code implementations • 24 Jul 2019 • Xinyan Li, Qilong Gu, Yingxue Zhou, Tiancong Chen, Arindam Banerjee
(2) how can we characterize the stochastic optimization dynamics of SGD with fixed and adaptive step sizes and diagonal pre-conditioning based on the first and second moments of SGs?
no code implementations • 10 Jul 2019 • Andre Goncalves, Xiaoli Liu, Arindam Banerjee
Alternating Direction Method of Multipliers (ADMM) has become a widely used optimization method for convex problems, particularly in the context of data mining in which large optimization problems are often encountered.
no code implementations • NeurIPS 2018 • Sheng Chen, Arindam Banerjee
To find the coefficient vector, estimators with a joint approximation of the noise covariance are often preferred than the simple linear regression in view of their superior empirical performance, which can be generally solved by alternating-minimization type procedures.
1 code implementation • 3 Nov 2018 • Robert Giaquinto, Arindam Banerjee
Extracting common narratives from multi-author dynamic text corpora requires complex models, such as the Dynamic Author Persona (DAP) topic model.
no code implementations • 21 Sep 2018 • Konstantina Christakopoulou, Arindam Banerjee
We propose a framework for generating fake user profiles which, when incorporated in the training of a recommendation system, can achieve an adversarial intent, while remaining indistinguishable from real user profiles.
no code implementations • 16 Jul 2018 • Amir Asiaee, Hardik Goel, Shalini Ghosh, Vinod Yegneswaran, Arindam Banerjee
Stream deinterleaving is an important problem with various applications in the cybersecurity domain.
no code implementations • 11 Jun 2018 • Amir Asiaee, Samet Oymak, Kevin R. Coombes, Arindam Banerjee
We consider the problem of multi-task learning in the high dimensional setting.
1 code implementation • 15 Jan 2018 • Robert Giaquinto, Arindam Banerjee
Topic modeling enables exploration and compact representation of a corpus.
no code implementations • 16 Oct 2017 • Sheng Chen, Arindam Banerjee
In machine learning and data mining, linear models have been widely used to model the response as parametric linear functions of the predictors.
no code implementations • 12 Sep 2017 • Jamal Golmohammadi, Imme Ebert-Uphoff, Sijie He, Yi Deng, Arindam Banerjee
We compare ACLIME-ADMM with baselines on both synthetic data simulated by partial differential equations (PDEs) that model advection-diffusion processes, and real data (50 years) of daily global geopotential heights to study information flow in the atmosphere.
no code implementations • 10 Sep 2017 • Hardik Goel, Igor Melnyk, Arindam Banerjee
In many multivariate time series modeling problems, there is usually a significant linear dependency component, for which VARs are suitable, and a nonlinear component, for which RNNs are suitable.
no code implementations • ICML 2017 • Sheng Chen, Arindam Banerjee
In this paper, we investigate general single-index models (SIMs) in high dimensions.
no code implementations • ICML 2017 • Vidyashankar Sivakumar, Arindam Banerjee
In this work we consider the problem of linear quantile regression in high dimensions where the number of predictor variables is much higher than the number of samples available for parameter estimation.
no code implementations • NeurIPS 2016 • Qilong Gu, Arindam Banerjee
High dimensional superposition models characterize observations using parameters which can be written as a sum of multiple component parameters, each with its own structure, e. g., sum of low rank and sparse matrices, sum of sparse and rotated sparse vectors, etc.
no code implementations • 30 Jan 2017 • André R. Gonçalves, Arindam Banerjee, Fernando J. Von Zuben
While IPCC has traditionally used a simple model output average, recent work has illustrated potential advantages of using a multitask learning (MTL) framework for projections of individual climate variables.
no code implementations • 18 Jan 2017 • Konstantina Christakopoulou, Jaya Kawale, Arindam Banerjee
In this paper, we investigate the common scenario where every candidate item for recommendation is characterized by a maximum capacity, i. e., number of seats in a Point-of-Interest (POI) or size of an item's inventory.
no code implementations • 27 Dec 2016 • Anuj Karpatne, Gowtham Atluri, James Faghmous, Michael Steinbach, Arindam Banerjee, Auroop Ganguly, Shashi Shekhar, Nagiza Samatova, Vipin Kumar
Theory-guided data science (TGDS) is an emerging paradigm that aims to leverage the wealth of scientific knowledge for improving the effectiveness of data science models in enabling scientific discovery.
no code implementations • NeurIPS 2017 • Sheng Chen, Arindam Banerjee
We consider learning high-dimensional multi-response linear models with structured parameters.
no code implementations • 17 Jun 2016 • Nicholas Johnson, Vidyashankar Sivakumar, Arindam Banerjee
The goal in such a problem is to minimize the (pseudo) regret which is the difference between the total expected loss of the algorithm and the total expected loss of the best fixed vector in hindsight.
no code implementations • 16 Jun 2016 • Farideh Fazayeli, Arindam Banerjee
We consider the problem of estimating change in the dependency structure between two $p$-dimensional Ising models, based on respectively $n_1$ and $n_2$ samples drawn from the models.
no code implementations • NeurIPS 2016 • Sheng Chen, Arindam Banerjee
In recent years, structured matrix recovery problems have gained considerable attention for its real world applications, such as recommender systems and computer vision.
no code implementations • 12 Apr 2016 • Farideh Fazayeli, Arindam Banerjee
Based on the property, we propose an importance sampling method for the $\mathcal{MGIG}$ where the mode of the proposal distribution matches that of the target.
no code implementations • NeurIPS 2015 • Suriya Gunasekar, Arindam Banerjee, Joydeep Ghosh
In this paper, we present a unified analysis of matrix completion under general low-dimensional structural constraints induced by {\em any} norm regularization.
no code implementations • 21 Feb 2016 • Igor Melnyk, Arindam Banerjee, Bryan Matthews, Nikunj Oza
In this context the goal is to detect anomalous flight segments, due to mechanical, environmental, or human factors in order to identifying operationally significant events and provide insights into the flight operations and highlight otherwise unavailable potential safety risks and precursors to accidents.
no code implementations • 21 Feb 2016 • Igor Melnyk, Arindam Banerjee
While considerable advances have been made in estimating high-dimensional structured models from independent data using Lasso-type models, limited progress has been made for settings when the samples are dependent.
no code implementations • NeurIPS 2015 • Vidyashankar Sivakumar, Arindam Banerjee, Pradeep K. Ravikumar
In contrast, for the sub-exponential setting, we show that the sample complexity and the estimation error will depend on the exponential width of the corresponding sets, and the analysis holds for any norm.
no code implementations • NeurIPS 2015 • Sheng Chen, Arindam Banerjee
For structured estimation problems with atomic norms, recent advances in the literature express sample complexity and estimation error bounds in terms of certain geometric measures, in particular Gaussian width of the unit norm ball, Gaussian width of a spherical cap induced by a tangent cone, and a restricted norm compatibility constant.
no code implementations • NeurIPS 2014 • Arindam Banerjee, Sheng Chen, Farideh Fazayeli, Vidyashankar Sivakumar
Analysis of non-asymptotic estimation error and structured statistical recovery based on norm regularized regression, such as Lasso, needs to consider four aspects: the norm, the loss function, the design matrix, and the noise model.
no code implementations • NeurIPS 2014 • Huahua Wang, Arindam Banerjee, Zhi-Quan Luo
In this paper, we propose a parallel randomized block coordinate method named Parallel Direction Method of Multipliers (PDMM) to solve the optimization problems with multi-block linear constraints.
no code implementations • 1 Sep 2014 • Andre R. Goncalves, Puja Das, Soumyadeep Chatterjee, Vidyashankar Sivakumar, Fernando J. Von Zuben, Arindam Banerjee
We illustrate the effectiveness of the proposed model on a variety of synthetic and benchmark datasets for regression and classification.
no code implementations • 12 Jul 2014 • Igor Melnyk, Arindam Banerjee
Hidden semi-Markov models (HSMMs) are latent variable models which allow latent state persistence and can be viewed as a generalization of the popular hidden Markov models (HMMs).
no code implementations • 1 Jul 2014 • Huahua Wang, Arindam Banerjee
One is online or stochastic gradient descent (OGD/SGD), and the other is randomzied coordinate descent (RBCD).
no code implementations • NeurIPS 2014 • Soumyadeep Chatterjee, Sheng Chen, Arindam Banerjee
For statistical analysis, we provide upper bounds for the Gaussian widths needed in the GDS analysis, yielding the first statistical recovery guarantee for estimation with the $k$-support norm.
no code implementations • NeurIPS 2013 • Huahua Wang, Arindam Banerjee, Cho-Jui Hsieh, Pradeep K. Ravikumar, Inderjit S. Dhillon
We consider the problem of sparse precision matrix estimation in high dimensions using the CLIME estimator, which has several desirable theoretical properties.
no code implementations • 26 Sep 2013 • Qiang Fu, Huahua Wang, Arindam Banerjee
We present a parallel MAP inference algorithm called Bethe-ADMM based on two ideas: tree-decomposition of the graph and the alternating direction method of multipliers (ADMM).
no code implementations • 17 Jun 2013 • Huahua Wang, Arindam Banerjee
Online optimization has emerged as powerful tool in large scale optimization.
no code implementations • NeurIPS 2014 • Huahua Wang, Arindam Banerjee
The mirror descent algorithm (MDA) generalizes gradient descent by using a Bregman divergence to replace squared Euclidean distance.
no code implementations • NeurIPS 2012 • Cho-Jui Hsieh, Arindam Banerjee, Inderjit S. Dhillon, Pradeep K. Ravikumar
We derive a bound on the distance of the approximate solution to the true solution.
no code implementations • NeurIPS 2012 • Shiva P. Kasiviswanathan, Huahua Wang, Arindam Banerjee, Prem Melville
Given their pervasive use, social media, such as Twitter, have become a leading source of breaking news.