Search Results for author:
Found 257 papers, 92 papers with code
Network Pruning by Greedy Subnetwork Selection
Theoretically, we show that the small networks pruned using our method achieve provably lower loss than small networks trained from scratch with the same size.
Accountable Off-Policy Evaluation via a Kernelized Bellman Statistics
In this work, we investigate the statistical properties of the kernel loss, which allows us to find a feasible set that contains the true value function with high probability.
Harmless Transfer Learning for Item Embeddings
On the definition side, we reduce the bias in transfer loss by focusing on the items to which information from high-frequency items can be efficiently transferred.
A Chance-Constrained Generative Framework for Sequence Optimization
Deep generative modeling has achieved many successes for continuous data generation, such as producing realistic images and controlling their properties (e. g., styles).
Communication Efficient Distributed Training with Distributed Lion
The Lion optimizer has been a promising competitor with the AdamW for training large AI models, with advantages on memory, computation, and sample efficiency.
Out-of-distribution Rumor Detection via Test-Time Adaptation
This method models the propagation of news in the form of a propagation graph, and builds propagation graph test-time adaptation framework, enhancing the model's adaptability and robustness when facing OOD problems.
Towards Unified Modeling for Positive and Negative Preferences in Sign-Aware Recommendation
Recently, sign-aware graph recommendation has drawn much attention as it will learn users' negative preferences besides positive ones from both positive and negative interactions (i. e., links in a graph) with items.
KEBench: A Benchmark on Knowledge Editing for Large Vision-Language Models
We conducted experiments of different editing methods on five LVLMs, and thoroughly analyze how these methods impact the models.
TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document
We present TextMonkey, a large multimodal model (LMM) tailored for text-centric tasks.
Bridging Text and Molecule: A Survey on Multimodal Frameworks for Molecule
In this paper, we present the first systematic survey on multimodal frameworks for molecules research.
Evolving to the Future: Unseen Event Adaptive Fake News Detection on Social Media
With the rapid development of social media, the wide dissemination of fake news on social media is increasingly threatening both individuals and society.
DecompDiff: Diffusion Models with Decomposed Priors for Structure-Based Drug Design
Designing 3D ligands within a target binding site is a fundamental task in drug discovery.
Enhancing Temporal Knowledge Graph Forecasting with Large Language Models via Chain-of-History Reasoning
Temporal Knowledge Graph (TKG) forecasting aims to predict future facts based on given histories.
Knowledge Graph Enhanced Large Language Model Editing
Large language models (LLMs) are pivotal in advancing natural language processing (NLP) tasks, yet their efficacy is hampered by inaccuracies and outdated knowledge.
Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables
To address this, we propose a novel word-level Heterogeneous-graph-based model for Fact Checking over unstructured and structured information, namely HeterFC.
Text-Guided Molecule Generation with Diffusion Language Model
In this work, we propose the Text-Guided Molecule Generation with Diffusion Language Model (TGM-DLM), a novel approach that leverages diffusion models to address the limitations of autoregressive methods.
Ranked #6 on
Text-based de novo Molecule Generation
on ChEBI-20
Language Modelling
Text-based de novo Molecule Generation
+1
Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models
In this work, we adopt the intuition that the LVLM tends to respond logically consistently for existent objects but inconsistently for hallucinated objects.
Stealthy Attack on Large Language Model based Recommendation
Recently, the powerful large language models (LLMs) have been instrumental in propelling the progress of recommender systems (RS).
Rethinking Graph Masked Autoencoders through Alignment and Uniformity
Based on our theoretical analysis, we further identify the limitations of the GraphMAE from the perspectives of alignment and uniformity, which have been considered as two key properties of high-quality representations in GCL.
AdaFlow: Imitation Learning with Variance-Adaptive Flow-Based Policies
To address this challenge, we propose AdaFlow, an imitation learning framework based on flow-based generative modeling.
Can Large Language Models Detect Rumors on Social Media?
In this work, we investigate to use Large Language Models (LLMs) for rumor detection on social media.
Fast and Scalable Network Slicing by Integrating Deep Learning with Lagrangian Methods
Based on the proposed framework, we design a new neural-assisted algorithm to allocate radio resources to slices to maximize the network utility under inter-slice resource constraints.
SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity
In this paper, we reveal that the gradient estimation in score distillation is inherent to high variance.
Taming Mode Collapse in Score Distillation for Text-to-3D Generation
In this paper, we reveal that the existing score distillation-based text-to-3D generation frameworks degenerate to maximal likelihood seeking on each view independently and thus suffer from the mode collapse problem, manifesting as the Janus artifact in practice.
Uncertainty-aware Surrogate Models for Airfoil Flow Simulations with Denoising Diffusion Probabilistic Models
Leveraging neural networks as surrogate models for turbulence simulation is a topic of growing interest.
ControlRec: Bridging the Semantic Gap between Language Model and Personalized Recommendation
The successful integration of large language models (LLMs) into recommendation systems has proven to be a major breakthrough in recent studies, paving the way for more generic and transferable recommendations.
Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models
Additionally, experiments on 18 datasets further demonstrate that Monkey surpasses existing LMMs in many tasks like Image Captioning and various Visual Question Answering formats.
ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation
In this paper, we revisit the value of ID embeddings for multimodal recommendation and conduct a thorough study regarding its semantics, which we recognize as subtle features of content and structures.
A Computational Framework for Solving Wasserstein Lagrangian Flows
The dynamical formulation of the optimal transport can be extended through various choices of the underlying geometry ($\textit{kinetic energy}$), and the regularization of density paths ($\textit{potential energy}$).
EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification
Each instance is accompanied by a veracity label and an explanation that outlines the reasoning path supporting the veracity classification.
Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts
As we can expect from the results of a random search program, Lion incorporates elements from several existing algorithms, including signed momentum, decoupled weight decay, Polak, and Nesterov momentum, but does not fit into any existing category of theoretically grounded optimizers.
CvFormer: Cross-view transFormers with Pre-training for fMRI Analysis of Human Brain
In recent years, functional magnetic resonance imaging (fMRI) has been widely utilized to diagnose neurological disease, by exploiting the region of interest (RoI) nodes as well as their connectivities in human brain.
TiBGL: Template-induced Brain Graph Learning for Functional Neuroimaging Analysis
Motivated by the related medical findings on functional connectivites, TiBGL proposes template-induced brain graph learning to extract template brain graphs for all groups.
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation
Leveraging our new pipeline, we create, to the best of our knowledge, the first one-step diffusion-based text-to-image generator with SD-level image quality, achieving an FID (Frechet Inception Distance) of $23. 3$ on MS COCO 2017-5k, surpassing the previous state-of-the-art technique, progressive distillation, by a significant margin ($37. 2$ $\rightarrow$ $23. 3$ in FID).
Imitation with Spatial-Temporal Heatmap: 2nd Place Solution for NuPlan Challenge
This paper presents our 2nd place solution for the NuPlan Challenge 2023.
Mining Stable Preferences: Adaptive Modality Decorrelation for Multimedia Recommendation
Even worse, the strong statistical correlation might mislead models to learn the spurious preference towards inconsequential modalities.
Inter-Cell Network Slicing With Transfer Learning Empowered Multi-Agent Deep Reinforcement Learning
Network slicing enables operators to efficiently support diverse applications on a common physical infrastructure.
FAMO: Fast Adaptive Multitask Optimization
One of the grand enduring goals of AI is to create generalist agents that can learn multiple different tasks from diverse data via multitask learning (MTL).
DUBLIN -- Document Understanding By Language-Image Network
Visual document understanding is a complex task that involves analyzing both the text and the visual elements in document images.
Ranked #1 on
Visual Question Answering (VQA)
on DeepForm
MolDiff: Addressing the Atom-Bond Inconsistency Problem in 3D Molecule Diffusion Generation
Deep generative models have recently achieved superior performance in 3D molecule generation.
Out-of-distribution Evidence-aware Fake News Detection via Dual Adversarial Debiasing
Then, DAL reversely optimizes news-aspect and evidence-aspect debiasing discriminators to mitigate the impact of news and evidence content biases.
LLM+P: Empowering Large Language Models with Optimal Planning Proficiency
LLM+P takes in a natural language description of a planning problem, then returns a correct (or optimal) plan for solving that problem in natural language.
Deep Stable Multi-Interest Learning for Out-of-distribution Sequential Recommendation
However, none of existing multi-interest recommendation models consider the Out-Of-Distribution (OOD) generalization problem, in which interest distribution may change.
Uncertainty Calibration for Counterfactual Propensity Estimation in Recommendation
In recommendation systems, a large portion of the ratings are missing due to the selection biases, which is known as Missing Not At Random.
Efficient Transformer-based 3D Object Detection with Dynamic Token Halting
Although halting a token is a non-differentiable operation, our method allows for differentiable end-to-end learning by leveraging an equivalent differentiable forward-pass.
Language Is Not All You Need: Aligning Perception with Language Models
A big convergence of language, multimodal perception, action, and world modeling is a key step toward artificial general intelligence.
Friend Ranking in Online Games via Pre-training Edge Transformers
Friend recall is an important way to improve Daily Active Users (DAU) in online games.
MetaTKG: Learning Evolutionary Meta-Knowledge for Temporal Knowledge Graph Reasoning
Most existing works focus on exploring evolutionary information in history to obtain effective temporal embeddings for entities and relations, but they ignore the variation in evolution patterns of facts, which makes them struggle to adapt to future data with different evolution patterns.
Metamobility: Connecting Future Mobility with Metaverse
A Metaverse is a perpetual, immersive, and shared digital universe that is linked to but beyond the physical reality, and this emerging technology is attracting enormous attention from different industries.
Network Slicing via Transfer Learning aided Distributed Deep Reinforcement Learning
In this paper, we propose a novel transfer learning (TL) aided multi-agent deep reinforcement learning (MADRL) approach with inter-agent similarity analysis for inter-cell inter-slice resource partitioning.
FlowGrad: Controlling the Output of Generative ODEs With Gradients
To further accelerate the computation of the back-propagation, we propose to use a non-uniform discretization to approximate the ODE trajectory, where we measure how straight the trajectory is and gather the straight parts into one discretization step.
Image as a Foreign Language: BEiT Pretraining for Vision and Vision-Language Tasks
A big convergence of language, vision, and multimodal pretraining is emerging.
Sparsely Annotated Semantic Segmentation With Adaptive Gaussian Mixtures
Our AGMM can effectively endow reliable supervision for unlabeled pixels based on the distributions of labeled and unlabeled pixels.
Planning-oriented Autonomous Driving
Oriented at this, we revisit the key components within perception and prediction, and prioritize the tasks such that all these tasks contribute to planning.
PathFusion: Path-consistent Lidar-Camera Deep Feature Fusion
Fusing 3D LiDAR features with 2D camera features is a promising technique for enhancing the accuracy of 3D detection, thanks to their complementary physical properties.
Fast Point Cloud Generation with Straight Flows
We perform evaluations on multiple 3D tasks and find that our PSF performs comparably to the standard diffusion model, outperforming other efficient 3D point cloud generation methods.
Distributed Node Covering Optimization for Large Scale Networks and Its Application on Social Advertising
Combinatorial optimizations are usually complex and inefficient, which limits their applications in large-scale networks with billions of links.
Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report
While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints.
Atlas: Automate Online Service Configuration in Network Slicing
First, we design a learning-based simulator to reduce the sim-to-real discrepancy, which is accomplished by a new parameter searching method based on Bayesian optimization.
Sampling with Mollified Interaction Energy Descent
These energies rely on mollifier functions -- smooth approximations of the Dirac delta originated from PDE theory.
The Devil is in the Conflict: Disentangled Information Graph Neural Networks for Fraud Detection
In this paper, we argue that the performance degradation is mainly attributed to the inconsistency between topology and attribute.
Sampling in Constrained Domains with Orthogonal-Space Variational Gradient Descent
Sampling methods, as important inference and learning techniques, are typically designed for unconstrained domains.
Adversarial Contrastive Learning for Evidence-aware Fake News Detection with Graph Neural Networks
Comprehensive experiments have demonstrated the superiority of GETRAL over the state-of-the-arts and validated the efficacy of semantic mining with graph structure and contrastive learning.
Neural Volumetric Mesh Generator
We first propose a diffusion-based generative model to tackle this problem by generating voxelized shapes with close-to-reality outlines and structures.
Improving Molecular Pretraining with Complementary Featurizations
Molecular pretraining, which learns molecular representations over massive unlabeled data, has become a prominent paradigm to solve a variety of tasks in computational chemistry and drug discovery.
Rectified Flow: A Marginal Preserving Approach to Optimal Transport
We present a flow-based approach to the optimal transport (OT) problem between two continuous distributions $\pi_0,\pi_1$ on $\mathbb{R}^d$, of minimizing a transport cost $\mathbb{E}[c(X_1-X_0)]$ in the set of couplings $(X_0, X_1)$ whose marginal distributions on $X_0, X_1$ equals $\pi_0,\pi_1$, respectively, where $c$ is a cost function.
BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach
Bilevel optimization (BO) is useful for solving a variety of important machine learning problems including but not limited to hyperparameter optimization, meta-learning, continual learning, and reinforcement learning.
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
The idea of rectified flow is to learn the ODE to follow the straight paths connecting the points drawn from \pi_0 and \pi_1 as much as possible.
Deep Stable Representation Learning on Electronic Health Records
The unstable correlation between procedures and diagnoses existed in the training distribution can cause spurious correlation between historical EHR and future diagnosis.
Diffusion-based Molecule Generation with Informative Prior Bridges
AI-based molecule generation provides a promising approach to a large area of biomedical sciences and engineering, such as antibody design, hydrolase engineering, or vaccine development.
First Hitting Diffusion Models for Generating Manifold, Graph and Categorical Data
We propose a family of First Hitting Diffusion Models (FHDM), deep generative models that generate data with a diffusion process that terminates at a random first hitting time.
Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems
One of the key challenges of learning an online recommendation model is the temporal domain shift, which causes the mismatch between the training and testing data distribution and hence domain generalization error.
Let us Build Bridges: Understanding and Extending Diffusion Generative Models
Diffusion-based generative models have achieved promising results recently, but raise an array of open questions in terms of conceptual understanding, theoretical analysis, algorithm improvement and extensions to discrete, structured, non-Euclidean domains.
Second-Order Global Attention Networks for Graph Classification and Regression
Graph Neural Networks (GNNs) are powerful to learn representation of graph-structured data, which fuse both attributive and topological information.
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks
A big convergence of language, vision, and multimodal pretraining is emerging.
Ranked #1 on
Visual Reasoning
on NLVR2 Test
Metric Residual Networks for Sample Efficient Goal-Conditioned Reinforcement Learning
Furthermore, we introduce the metric residual network (MRN) that deliberately decomposes the action-value function Q(s, a, g) into the negated summation of a metric plus a residual asymmetric component.
Improving Multi-Interest Network with Stable Learning
Modeling users' dynamic preferences from historical behaviors lies at the core of modern recommender systems.
Network Pruning via Feature Shift Minimization
In this paper, we propose a novel Feature Shift Minimization (FSM) method to compress CNN models, which evaluates the feature shift by converging the information of both features and filters.
Split Localized Conformal Prediction
The modified score inherits the spirit of split conformal methods, which is simple and efficient and can scale to high dimensional settings.
HOPE: Hierarchical Spatial-temporal Network for Occupancy Flow Prediction
In this report, we introduce our solution to the Occupancy and Flow Prediction challenge in the Waymo Open Dataset Challenges at CVPR 2022, which ranks 1st on the leaderboard.
A Langevin-like Sampler for Discrete Distributions
We propose discrete Langevin proposal (DLP), a simple and scalable gradient-based proposal for sampling complex high-dimensional discrete distributions.
CAINNFlow: Convolutional block Attention modules and Invertible Neural Networks Flow for anomaly detection and localization tasks
Detection of object anomalies is crucial in industrial processes, but unsupervised anomaly detection and localization is particularly important due to the difficulty of obtaining a large number of defective samples and the unpredictable types of anomalies in real life.
RMT-Net: Reject-aware Multi-Task Network for Modeling Missing-not-at-random Data in Financial Credit Scoring
Accordingly, we for the first time propose to model the biased credit scoring data with Multi-Task Learning (MTL).
Estimating spot volatility under infinite variation jumps with dependent market microstructure noise
Jumps and market microstructure noise are stylized features of high-frequency financial data.
Continual Learning and Private Unlearning
As intelligent agents become autonomous over longer periods of time, they may eventually become lifelong counterparts to specific people.
WCL-BBCD: A Contrastive Learning and Knowledge Graph Approach to Named Entity Recognition
The model first trains the sentence pairs in the text, calculate similarity between sentence pairs, and fine-tunes BERT used for the named entity recognition task according to the similarity, so as to alleviate word ambiguity.
A Survey on Deep Graph Generation: Methods and Applications
In this paper, we conduct a comprehensive review on the existing literature of deep graph generation from a variety of emerging methods to its wide application areas.
A Dual Neighborhood Hypergraph Neural Network for Change Detection in VHR Remote Sensing Images
To address this limitation, a dual neighborhood hypergraph neural network is proposed in this article, which combines the multiscale superpixel segmentation and hypergraph convolution to model and exploit the complex relationships.
How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity
Although traditional optimization methods focus on finding a single optimal solution, most objective functions in modern machine learning problems, especially those in deep learning, often have multiple or infinite numbers of optima.
EdgeMap: CrowdSourcing High Definition Map in Automotive Edge Computing
High definition (HD) map needs to be updated frequently to capture road changes, which is constrained by limited specialized collection vehicles.
Evidence-aware Fake News Detection with Graph Neural Networks
In this paper, we focus on the evidence-based fake news detection, where several evidences are utilized to probe the veracity of news (i. e., a claim).
Operator Deep Q-Learning: Zero-Shot Reward Transferring in Reinforcement Learning
The benefit of learning the operator is that we can incorporate any new reward function as input and attain its corresponding value function in a zero-shot manner.
Are we really making much progress? Revisiting, benchmarking, and refining heterogeneous graph neural networks
Heterogeneous graph neural networks (HGNNs) have been blossoming in recent years, but the unique data processing and evaluation setups used by each work obstruct a full understanding of their advancements.
AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds
From this observation, we have devised a single-stage anchor-free network that can fulfill these requirements.
Network Compression via Central Filter
In this paper, by exploring the similarities between feature maps, we propose a novel filter pruning method, Central Filter (CF), which suggests that a filter is approximately equal to a set of other filters after appropriate adjustments.
FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimization
We approach text-to-image generation by combining the power of the retrained CLIP representation with an off-the-shelf image generator (GANs), optimizing in the latent space of GAN to find images that achieve maximum CLIP score with the given input text.
Ranked #48 on
Text-to-Image Generation
on MS COCO
Automatic and Harmless Regularization with Constrained and Lexicographic Optimization: A Dynamic Barrier Approach
In this work, we consider constrained optimization as a more principled approach for trading off two losses, with a special emphasis on lexicographic optimization, a degenerated limit of constrained optimization which optimizes a secondary loss inside the optimal set of the main loss.
Sampling with Trusthworthy Constraints: A Variational Gradient Framework
In this work, we propose a family of constrained sampling algorithms which generalize Langevin Dynamics (LD) and Stein Variational Gradient Descent (SVGD) to incorporate a moment constraint specified by a general nonlinear function.
argmax centroid
We propose a general method to construct centroid approximation for the distribution of maximum points of a random function (a. k. a.
Profiling Pareto Front With Multi-Objective Stein Variational Gradient Descent
Finding diverse and representative Pareto solutions from the Pareto front is a key challenge in multi-objective optimization (MOO).
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts
We present a unified Vision-Language pretrained Model (VLMo) that jointly learns a dual encoder and a fusion encoder with a modular Transformer network.
Ranked #2 on
Image Retrieval
on PhotoChat
OnSlicing: Online End-to-End Network Slicing with Reinforcement Learning
As online learning is converged, OnSlicing reduces 12. 5% usage without any violations as compared to the state-of-the-art online DRL solution.
Latent Structure Mining with Contrastive Modality Fusion for Multimedia Recommendation
Although having access to multiple modalities might allow us to capture rich information, we argue that the simple coarse-grained fusion by linear combination or concatenation in previous work is insufficient to fully understand content information and item relationships. To this end, we propose a latent structure MIning with ContRastive mOdality fusion method (MICRO for brevity).
Conflict-Averse Gradient Descent for Multi-task Learning
The goal of multi-task learning is to enable more efficient learning than single task learning by sharing model structures for a diverse set of tasks.
Centroid Approximation for Bootstrap: Improving Particle Quality at Inference
In this work, we propose an efficient method to explicitly \emph{optimize} a small set of high quality ``centroid'' points to better approximate the ideal bootstrap distribution.
Pareto Navigation Gradient Descent: a First-Order Algorithm for Optimization in Pareto Set
The notion of the Pareto set allows us to focus on the set of (often infinite number of) models that cannot be strictly improved.
Exciting-Inhibition Network for Person Reidentification in Internet of Things
Index Terms—Channel-spatial attention block (CSAB), exciting-inhibition network (EINet), Internet of Things (IoT), person reidentification (re-ID), soft batch dropblock.
Ranked #55 on
Person Re-Identification
on Market-1501
Relation-aware Heterogeneous Graph for User Profiling
User profiling has long been an important problem that investigates user interests in many real applications.
Speeding up Deep Model Training by Sharing Weights and Then Unsharing
Our approach exploits the special structure of BERT that contains a stack of repeated modules (i. e., transformer encoders).
Neural Energy Minimization for Molecular Conformation Optimization
Assuming different forms of the underlying potential energy function, we can not only reinterpret and unify many of the existing models but also derive new variants of SE(3)-equivariant neural networks in a principled manner.
NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training
In this work, we observe that the poor performance is due to a gradient conflict issue: the gradients of different sub-networks conflict with that of the supernet more severely in ViTs than CNNs, which leads to early saturation in training and inferior convergence.
Ranked #7 on
Neural Architecture Search
on ImageNet
Pareto Navigation Gradient Descent: a First Order Algorithm for Optimization in Pareto Set
The notion of the Pareto set allows us to focus on the set of (often infinite number of) models that cannot be strictly improved.
An Empirical Study of Graph Contrastive Learning
We envision this work to provide useful empirical evidence of effective GCL algorithms and offer several insights for future research.
Structure-Aware Hard Negative Mining for Heterogeneous Graph Contrastive Learning
Recently, heterogeneous Graph Neural Networks (GNNs) have become a de facto model for analyzing HGs, while most of them rely on a relative large number of labeled data.
Deep Contrastive Multiview Network Embedding
In our work, different views can be obtained based on the various relations among nodes.
Deep Active Learning for Text Classification with Diverse Interpretations
To train high-performing models with the minimal annotation cost, active learning is proposed to select and label the most informative samples, yet it is still challenging to measure informativeness of samples used in DNNs.
Real-Time Anchor-Free Single-Stage 3D Detection with IoU-Awareness
Extended from our last year's award-winning model AFDet, we have made a handful of modifications to the base model, to improve the accuracy and at the same time to greatly reduce the latency.
PNet -- A Deep Learning Based Photometry and Astrometry Bayesian Framework
Time domain astronomy has emerged as a vibrant research field in recent years, focusing on celestial objects that exhibit variable magnitudes or positions.
MaxUp: Lightweight Adversarial Training With Data Augmentation Improves Neural Network Training
The idea is to generate a set of augmented data with some random perturbations or transforms, and minimize the maximum, or worst case loss over the augmented data.
Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)
Partial differential equations (PDEs) are concise and understandable representations of domain knowledge, which are essential for deepening our understanding of physical processes and predicting future responses.
Physics-Guided Discovery of Highly Nonlinear Parametric Partial Differential Equations
Especially, the discovery of PDEs with highly nonlinear coefficients from low-quality data remains largely under-addressed.
Sampling with Trusthworthy Constraints: A Variational Gradient Framework
In this work, we propose a family of constrained sampling algorithms which generalize Langevin Dynamics (LD) and Stein Variational Gradient Descent (SVGD) to incorporate a moment constraint specified by a general nonlinear function.
Automatic and Harmless Regularization with Constrained and Lexicographic Optimization: A Dynamic Barrier Approach
In this work, we consider constrained optimization as a more principled approach for trading off two losses, with a special emphasis on lexicographic optimization, a degenerated limit of constrained optimization which optimizes a secondary loss inside the optimal set of the main loss.
Profiling Pareto Front With Multi-Objective Stein Variational Gradient Descent
Finding diverse and representative Pareto solutions from the Pareto front is a key challenge in multi-objective optimization (MOO).
Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition
Specifically, we 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
Vision Transformers with Patch Diversification
To alleviate this problem, in this work, we introduce novel loss functions in vision transformer training to explicitly encourage diversity across patch representations for more discriminative feature extraction.
Ranked #19 on
Semantic Segmentation
on Cityscapes val
Mining Latent Structures for Multimedia Recommendation
To be specific, in the proposed LATTICE model, we devise a novel modality-aware structure learning layer, which learns item-item structures for each modality and aggregates multiple modalities to obtain latent item graphs.
Dynamic Graph Neural Networks for Sequential Recommendation
We propose a new method named Dynamic Graph Neural Network for Sequential Recommendation (DGSR), which connects different user sequences through a dynamic graph structure, exploring the interactive behavior of users and items with time and order information.
DyGCN: Dynamic Graph Embedding with Graph Convolutional Network
We naturally generalizes the embedding propagation scheme of GCN to dynamic setting in an efficient manner, which is to propagate the change along the graph to update node embeddings.
GraphDIVE: Graph Classification by Mixture of Diverse Experts
Graph classification is a challenging research problem in many applications across a broad range of domains.
VCNet and Functional Targeted Regularization For Learning Causal Effects of Continuous Treatments
Motivated by the rising abundance of observational data with continuous treatments, we investigate the problem of estimating the average dose-response curve (ADRF).
Non-asymptotic Confidence Intervals of Off-policy Evaluation: Primal and Dual Bounds
Off-policy evaluation (OPE) is the task of estimating the expected reward of a given policy based on offline data previously collected under different policies.
A Survey on Graph Structure Learning: Progress and Opportunities
Specifically, we first formulate a general pipeline of GSL and review state-of-the-art methods classified by the way of modeling graph structures, followed by applications of GSL across domains.
DNN2LR: Automatic Feature Crossing for Credit Scoring
Accordingly, we can design an automatic feature crossing method to find feature interactions in DNN, and use them as cross features in LR.
Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks
We propose firefly neural architecture descent, a general framework for progressively and dynamically growing neural networks to jointly optimize the networks' parameters and architectures.
Centroid Transformers: Learning to Abstract with Attention
Self-attention, as the key block of transformers, is a powerful mechanism for extracting features from the inputs.
Ranked #618 on
Image Classification
on ImageNet
AlphaNet: Improved Training of Supernets with Alpha-Divergence
Weight-sharing NAS builds a supernet that assembles all the architectures as its sub-networks and jointly trains the supernet with the sub-networks.
Ranked #12 on
Neural Architecture Search
on ImageNet
STAN: Spatio-Temporal Attention Network for Next Location Recommendation
The next location recommendation is at the core of various location-based applications.
Ranked #1 on
point of interests
on Gowalla
Coherent ray-wave structured light based on (helical) Ince-Gaussian modes
The topological evolution of classic eigenmodes including Hermite-Laguerre-Gaussian and (helical) InceGaussian modes is exploited to construct coherent state modes, which unifies the representations of travelingwave (TW) and standing-wave (SW) ray-wave structured light for the first time and realizes the TW-SW unified ray-wave geometric beam with topology of raytrajectories splitting effect, breaking the boundary of TW and SW structured light.
Optics
Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction
To better model complex feature interaction, in this paper we propose a novel DisentanglEd Self-atTentIve NEtwork (DESTINE) framework for CTR prediction that explicitly decouples the computation of unary feature importance from pairwise interaction.
Fast Training of Contrastive Learning with Intermediate Contrastive Loss
We apply our method to recently-proposed MOCO, SimCLR, SwAV and notice that we can reduce the computational cost with little loss on the performance of ImageNet linear classification and other downstream tasks.
Varying Coefficient Neural Network with Functional Targeted Regularization for Estimating Continuous Treatment Effects
With the rising abundance of observational data with continuous treatments, we investigate the problem of estimating average dose-response curve (ADRF).
Speeding up Deep Learning Training by Sharing Weights and Then Unsharing
It has been widely observed that increasing deep learning model sizes often leads to significant performance improvements on a variety of natural language processing and computer vision tasks.
A Coach-Player Framework for Dynamic Team Composition
The performance of our method is comparable or even better than the setting where all players have a full view of the environment, but no coach.
LiveMap: Real-Time Dynamic Map in Automotive Edge Computing
In this paper, we propose LiveMap, a real-time dynamic map, that detects, matches, and tracks objects on the road with crowdsourcing data from connected vehicles in sub-second.
Eudoxus: Characterizing and Accelerating Localization in Autonomous Machines
We develop and commercialize autonomous machines, such as logistic robots and self-driving cars, around the globe.
Self-Driving Cars
Hardware Architecture
KeepAugment: A Simple Information-Preserving Data Augmentation Approach
Data augmentation (DA) is an essential technique for training state-of-the-art deep learning systems.
AlphaMatch: Improving Consistency for Semi-supervised Learning with Alpha-divergence
Semi-supervised learning (SSL) is a key approach toward more data-efficient machine learning by jointly leverage both labeled and unlabeled data.
Point Spread Function Estimation for Wide Field Small Aperture Telescopes with Deep Neural Networks and Calibration Data
Then we use these data to train a DNN (Tel--Net).
Certified Monotonic Neural Networks
In this work, we propose to certify the monotonicity of the general piece-wise linear neural networks by solving a mixed integer linear programming problem. This provides a new general approach for learning monotonic neural networks with arbitrary model structures.
Smart obervation method with wide field small aperture telescopes for real time transient detection
The ARGUS uses a deep learning based astronomical detection algorithm implemented in embedded devices in each WFSATs to detect astronomical targets.
Debiasing Convolutional Neural Networks via Meta Orthogonalization
While deep learning models often achieve strong task performance, their successes are hampered by their inability to disentangle spurious correlations from causative factors, such as when they use protected attributes (e. g., race, gender, etc.)
When Contrastive Learning Meets Active Learning: A Novel Graph Active Learning Paradigm with Self-Supervision
To this end, we present a minimax selection scheme that explicitly harnesses neighborhood information and discover homophilous subgraphs to facilitate active selection.
Greedy Optimization Provably Wins the Lottery: Logarithmic Number of Winning Tickets is Enough
Despite the great success of deep learning, recent works show that large deep neural networks are often highly redundant and can be significantly reduced in size.
Off-Policy Interval Estimation with Lipschitz Value Iteration
Off-policy evaluation provides an essential tool for evaluating the effects of different policies or treatments using only observed data.
Graph Contrastive Learning with Adaptive Augmentation
On the node attribute level, we corrupt node features by adding more noise to unimportant node features, to enforce the model to recognize underlying semantic information.
Population Gradients improve performance across data-sets and architectures in object classification
The most successful methods such as ReLU transfer functions, batch normalization, Xavier initialization, dropout, learning rate decay, or dynamic optimizers, have become standards in the field due, particularly, to their ability to increase the performance of Neural Networks (NNs) significantly and in almost all situations.
An efficient representation of chronological events in medical texts
In this work we addressed the problem of capturing sequential information contained in longitudinal electronic health records (EHRs).
Adaptive Dense-to-Sparse Paradigm for Pruning Online Recommendation System with Non-Stationary Data
To the best of our knowledge, this is the first work to provide in-depth analysis and discussion of applying pruning to online recommendation systems with non-stationary data distribution.
Modeling Dyadic Conversations for Personality Inference
We adjust the formulation of each layer of a conventional GRU with sequence to sequence learning and personal information of both sides of the conversation.
DNN2LR: Interpretation-inspired Feature Crossing for Real-world Tabular Data
Simple classifiers, e. g., Logistic Regression (LR), are globally interpretable, but not powerful enough to model complex nonlinear interactions among features in tabular data.
DeepSlicing: Deep Reinforcement Learning Assisted Resource Allocation for Network Slicing
Network slicing enables multiple virtual networks run on the same physical infrastructure to support various use cases in 5G and beyond.
Disentangled Item Representation for Recommender Systems
In this way, the items are represented at the attribute level, which can provide fine-grained information of items in recommendation.
Accountable Off-Policy Evaluation With Kernel Bellman Statistics
We consider off-policy evaluation (OPE), which evaluates the performance of a new policy from observed data collected from previous experiments, without requiring the execution of the new policy.
Deep Active Learning by Model Interpretability
In this paper, inspired by piece-wise linear interpretability in DNN, we introduce the linearly separable regions of samples to the problem of active learning, and propose a novel Deep Active learning approach by Model Interpretability (DAMI).
Simplification of Graph Convolutional Networks: A Matrix Factorization-based Perspective
Moreover, we have also conducted experiments on a typical task of graph embedding, i. e., community detection, and the proposed UCMF model outperforms several representative graph embedding models.
Go Wide, Then Narrow: Efficient Training of Deep Thin Networks
This is achieved by layerwise imitation, that is, forcing the thin network to mimic the intermediate outputs of the wide network from layer to layer.
TFNet: Multi-Semantic Feature Interaction for CTR Prediction
The CTR (Click-Through Rate) prediction plays a central role in the domain of computational advertising and recommender systems.
Ranked #31 on
Click-Through Rate Prediction
on Criteo
Deep Graph Contrastive Representation Learning
Moreover, our unsupervised method even surpasses its supervised counterparts on transductive tasks, demonstrating its great potential in real-world applications.
Ranked #1 on
Node Classification
on DBLP
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
For security reasons, it is of critical importance to develop models with certified robustness that can provably guarantee that the prediction is can not be altered by any possible synonymous word substitution.
TAGNN: Target Attentive Graph Neural Networks for Session-based Recommendation
However, these methods compress a session into one fixed representation vector without considering the target items to be predicted.
Ranked #3 on
Session-Based Recommendations
on yoochoose1
An Empirical Study on Feature Discretization
When dealing with continuous numeric features, we usually adopt feature discretization.
Dimension Independent Generalization Error by Stochastic Gradient Descent
One classical canon of statistics is that large models are prone to overfitting, and model selection procedures are necessary for high dimensional data.
Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning
Off-policy estimation for long-horizon problems is important in many real-life applications such as healthcare and robotics, where high-fidelity simulators may not be available and on-policy evaluation is expensive or impossible.
Steepest Descent Neural Architecture Optimization: Escaping Local Optimum with Signed Neural Splitting
Recently, Liu et al.[19] proposed a splitting steepest descent (S2D) method that jointly optimizes the neural parameters and architectures based on progressively growing network structures by splitting neurons into multiple copies in a steepest descent fashion.
Med7: a transferable clinical natural language processing model for electronic health records
In this work we introduced a named-entity recognition model for clinical natural language processing.
Medical Named Entity Recognition
named-entity-recognition
+4
Good Subnetworks Provably Exist: Pruning via Greedy Forward Selection
This differs from the existing methods based on backward elimination, which remove redundant neurons from the large network.
Stein Variational Inference for Discrete Distributions
In addition, such transform can be straightforwardly employed in gradient-free kernelized Stein discrepancy to perform goodness-of-fit (GOF) test on discrete distributions.
Statistical Adaptive Stochastic Gradient Methods
We propose a statistical adaptive procedure called SALSA for automatically scheduling the learning rate (step size) in stochastic gradient methods.
Black-Box Certification with Randomized Smoothing: A Functional Optimization Based Framework
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning.
Detection and Classification of Astronomical Targets with Deep Neural Networks in Wide Field Small Aperture Telescopes
To increase the generalization ability of our framework, we use both simulated and real observation images to train the neural network.
Stein Self-Repulsive Dynamics: Benefits From Past Samples
Our idea is to introduce Stein variational gradient as a repulsive force to push the samples of Langevin dynamics away from the past trajectories.
Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision
We propose multipoint quantization, a quantization method that approximates a full-precision weight vector using a linear combination of multiple vectors of low-bit numbers; this is in contrast to typical quantization methods that approximate each weight using a single low precision number.
MaxUp: A Simple Way to Improve Generalization of Neural Network Training
The idea is to generate a set of augmented data with some random perturbations or transforms and minimize the maximum, or worst case loss over the augmented data.
Ranked #187 on
Image Classification
on ImageNet
Attributed Multi-Relational Attention Network for Fact-checking URL Recommendation
To combat fake news, researchers mostly focused on detecting fake news and journalists built and maintained fact-checking sites (e. g., Snopes. com and Politifact. com).
Deep Interaction Machine: A Simple but Effective Model for High-order Feature Interactions
IM is an efficient and exact implementation of high-order FM, whose time complexity linearly grows with the order of interactions and the number of feature fields.
Deep Interaction Machine: A Simple but Effective Model for High-order Feature Interactions
IM is an efficient and exact implementation of high-order FM, whose time complexity linearly grows with the order of interactions and the number of feature fields.
Unifying Graph Convolutional Networks as Matrix Factorization
In recent years, substantial progress has been made on graph convolutional networks (GCN).
The Black-Scholes-Merton dual equation
We derive the Black-Scholes-Merton dual equation, which has exactly the same form as the Black-Scholes-Merton equation.
Implicit Regularization and Convergence for Weight Normalization
For certain stepsizes of g and w , we show that they can converge close to the minimum norm solution.
Learning Preferences and Demands in Visual Recommendation
For modeling users' demands on different categories of items, the problem can be formulated as recommendation with contextual and sequential information.
Stein Variational Gradient Descent With Matrix-Valued Kernels
Stein variational gradient descent (SVGD) is a particle-based inference algorithm that leverages gradient information for efficient approximate inference.
Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation
Our method is doubly robust in that the bias vanishes when either the density ratio or the value function estimation is perfect.
Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent
Designing energy-efficient networks is of critical importance for enabling state-of-the-art deep learning in mobile and edge settings where the computation and energy budgets are highly limited.
Splitting Steepest Descent for Growing Neural Architectures
We develop a progressive training approach for neural networks which adaptively grows the network structure by splitting existing neurons to multiple off-springs.
Automatically Learning Feature Crossing from Model Interpretation for Tabular Data
Automatically feature generation is a major topic of automated machine learning.
Statistical Adaptive Stochastic Optimization
We investigate statistical methods for automatically scheduling the learning rate (step size) in stochastic optimization.
Filling the Soap Bubbles: Efficient Black-Box Adversarial Certification with Non-Gaussian Smoothing
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning.
Training Robust Deep Neural Networks via Adversarial Noise Propagation
Various adversarial defense methods have accordingly been developed to improve adversarial robustness for deep models.
