Search Results for author: Qingquan Song

Found 22 papers, 4 papers with code

QuantEase: Optimization-based Quantization for Language Models

no code implementations5 Sep 2023 Kayhan Behdin, Ayan Acharya, Aman Gupta, Qingquan Song, Siyu Zhu, Sathiya Keerthi, Rahul Mazumder

Particularly noteworthy is our outlier-aware algorithm's capability to achieve near or sub-3-bit quantization of LLMs with an acceptable drop in accuracy, obviating the need for non-uniform quantization or grouping techniques, improving upon methods such as SpQR by up to two times in terms of perplexity.

Quantization

mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization

no code implementations19 Feb 2023 Kayhan Behdin, Qingquan Song, Aman Gupta, Sathiya Keerthi, Ayan Acharya, Borja Ocejo, Gregory Dexter, Rajiv Khanna, David Durfee, Rahul Mazumder

Modern deep learning models are over-parameterized, where different optima can result in widely varying generalization performance.

Image Classification

Geometric Graph Representation Learning via Maximizing Rate Reduction

no code implementations13 Feb 2022 Xiaotian Han, Zhimeng Jiang, Ninghao Liu, Qingquan Song, Jundong Li, Xia Hu

Learning discriminative node representations benefits various downstream tasks in graph analysis such as community detection and node classification.

Community Detection Contrastive Learning +2

Efficient Differentiable Neural Architecture Search with Model Parallelism

no code implementations1 Jan 2021 Yi-Wei Chen, Qingquan Song, Xia Hu

Differentiable NAS with supernets that encompass all potential architectures in a large graph cuts down search overhead to few GPU days or less.

Neural Architecture Search

Detecting Interactions from Neural Networks via Topological Analysis

no code implementations NeurIPS 2020 Zirui Liu, Qingquan Song, Kaixiong Zhou, Ting-Hsiang Wang, Ying Shan, Xia Hu

Motivated by the observation, in this paper, we propose to investigate the interaction detection problem from a novel topological perspective by analyzing the connectivity in neural networks.

Towards Interaction Detection Using Topological Analysis on Neural Networks

no code implementations25 Oct 2020 Zirui Liu, Qingquan Song, Kaixiong Zhou, Ting Hsiang Wang, Ying Shan, Xia Hu

Detecting statistical interactions between input features is a crucial and challenging task.

Towards Automated Neural Interaction Discovery for Click-Through Rate Prediction

no code implementations29 Jun 2020 Qingquan Song, Dehua Cheng, Hanning Zhou, Jiyan Yang, Yuandong Tian, Xia Hu

Click-Through Rate (CTR) prediction is one of the most important machine learning tasks in recommender systems, driving personalized experience for billions of consumers.

Click-Through Rate Prediction Learning-To-Rank +2

AutoRec: An Automated Recommender System

1 code implementation26 Jun 2020 Ting-Hsiang Wang, Qingquan Song, Xiaotian Han, Zirui Liu, Haifeng Jin, Xia Hu

To address the need, we present AutoRec, an open-source automated machine learning (AutoML) platform extended from the TensorFlow ecosystem and, to our knowledge, the first framework to leverage AutoML for model search and hyperparameter tuning in deep recommendation models.

AutoML Click-Through Rate Prediction +1

Multi-Channel Graph Convolutional Networks

no code implementations17 Dec 2019 Kaixiong Zhou, Qingquan Song, Xiao Huang, Daochen Zha, Na Zou, Xia Hu

To further improve the graph representation learning ability, hierarchical GNN has been explored.

Clustering Graph Classification +1

Sub-Architecture Ensemble Pruning in Neural Architecture Search

1 code implementation1 Oct 2019 Yijun Bian, Qingquan Song, Mengnan Du, Jun Yao, Huanhuan Chen, Xia Hu

Neural architecture search (NAS) is gaining more and more attention in recent years due to its flexibility and remarkable capability to reduce the burden of neural network design.

Ensemble Learning Ensemble Pruning +1

Techniques for Automated Machine Learning

no code implementations21 Jul 2019 Yi-Wei Chen, Qingquan Song, Xia Hu

Automated machine learning (AutoML) aims to find optimal machine learning solutions automatically given a machine learning problem.

Automated Feature Engineering Bayesian Optimization +4

Coupled Variational Recurrent Collaborative Filtering

1 code implementation11 Jun 2019 Qingquan Song, Shiyu Chang, Xia Hu

To bridge the gap, in this paper, we propose a Coupled Variational Recurrent Collaborative Filtering (CVRCF) framework based on the idea of Deep Bayesian Learning to handle the streaming recommendation problem.

Collaborative Filtering Recommendation Systems +1

Multi-Label Adversarial Perturbations

no code implementations2 Jan 2019 Qingquan Song, Haifeng Jin, Xiao Huang, Xia Hu

Experiments on real-world multi-label image classification and ranking problems demonstrate the effectiveness of our proposed frameworks and provide insights of the vulnerability of multi-label deep learning models under diverse targeted attacking strategies.

General Classification Multi-class Classification +3

Auto-Keras: An Efficient Neural Architecture Search System

14 code implementations27 Jun 2018 Haifeng Jin, Qingquan Song, Xia Hu

In this paper, we propose a novel framework enabling Bayesian optimization to guide the network morphism for efficient neural architecture search.

Bayesian Optimization Neural Architecture Search

Towards Explanation of DNN-based Prediction with Guided Feature Inversion

no code implementations19 Mar 2018 Mengnan Du, Ninghao Liu, Qingquan Song, Xia Hu

While deep neural networks (DNN) have become an effective computational tool, the prediction results are often criticized by the lack of interpretability, which is essential in many real-world applications such as health informatics.

Decision Making

Tensor Completion Algorithms in Big Data Analytics

no code implementations28 Nov 2017 Qingquan Song, Hancheng Ge, James Caverlee, Xia Hu

Tensor completion is a problem of filling the missing or unobserved entries of partially observed tensors.

Cannot find the paper you are looking for? You can Submit a new open access paper.