no code implementations • 28 Oct 2023 • Shibal Ibrahim, Kayhan Behdin, Rahul Mazumder
Skinny Trees lead to superior feature selection than many existing toolkits e. g., in terms of AUC performance for $25\%$ feature budget, Skinny Trees outperforms LightGBM by $10. 2\%$ (up to $37. 7\%$), and Random Forests by $3\%$ (up to $12. 5\%$).
no code implementations • 5 Sep 2023 • Kayhan Behdin, Ayan Acharya, Aman Gupta, Qingquan Song, Siyu Zhu, Sathiya Keerthi, Rahul Mazumder
Particularly noteworthy is our outlier-aware algorithm's capability to achieve near or sub-3-bit quantization of LLMs with an acceptable drop in accuracy, obviating the need for non-uniform quantization or grouping techniques, improving upon methods such as SpQR by up to two times in terms of perplexity.
no code implementations • 18 Jul 2023 • Kayhan Behdin, Wenyu Chen, Rahul Mazumder
To solve the MIP, we propose a custom nonlinear branch-and-bound (BnB) framework that solves node relaxations with tailored first-order methods.
no code implementations • 23 Feb 2023 • Kayhan Behdin, Rahul Mazumder
As SAM has been numerically successful, recent papers have studied the theoretical aspects of the framework and have shown SAM solutions are indeed flat.
no code implementations • 19 Feb 2023 • Kayhan Behdin, Qingquan Song, Aman Gupta, Sathiya Keerthi, Ayan Acharya, Borja Ocejo, Gregory Dexter, Rajiv Khanna, David Durfee, Rahul Mazumder
Modern deep learning models are over-parameterized, where different optima can result in widely varying generalization performance.
1 code implementation • 16 Dec 2022 • Gabriel Loewinger, Kayhan Behdin, Kenneth T. Kishida, Giovanni Parmigiani, Rahul Mazumder
Allowing the regression coefficients of tasks to have different sparsity patterns (i. e., different supports), we propose a modeling framework for MTL that encourages models to share information across tasks, for a given covariate, through separately 1) shrinking the coefficient supports together, and/or 2) shrinking the coefficient values together.
no code implementations • 7 Dec 2022 • Kayhan Behdin, Qingquan Song, Aman Gupta, David Durfee, Ayan Acharya, Sathiya Keerthi, Rahul Mazumder
To that end, this paper presents a thorough empirical evaluation of mSAM on various tasks and datasets.
1 code implementation • 8 Apr 2021 • Kayhan Behdin, Rahul Mazumder
We consider the problem of sparse nonnegative matrix factorization (NMF) using archetypal regularization.
no code implementations • 7 Oct 2018 • Ashkan Esmaeili, Kayhan Behdin, Sina Al-E-Mohammad, Farokh Marvasti
In this paper, we propose a novel approach in order to recover a quantized matrix with missing information.
no code implementations • 19 May 2018 • Ashkan Esmaeili, Kayhan Behdin, Mohammad Amin Fakharian, Farokh Marvasti
In this paper, we propose two new algorithms for transduction with Matrix Completion (MC) problem.
1 code implementation • 7 Apr 2017 • Ali Mottaghi, Kayhan Behdin, Ashkan Esmaeili, Mohammadreza Heydari, Farokh Marvasti
In this paper, we design a system in order to perform the real-time beat tracking for an audio signal.