Search Results for author:
Found 38 papers, 11 papers with code
Simulating Weighted Automata over Sequences and Trees with Transformers
Recent work has shown that these models can compactly simulate the sequential reasoning abilities of deterministic finite automata (DFAs).
Generative Learning of Continuous Data by Tensor Networks
Beyond their origin in modeling many-body quantum systems, tensor networks have emerged as a promising class of models for solving machine learning problems, notably in unsupervised generative learning.
Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets
Recently, pre-trained foundation models have enabled significant advancements in multiple fields.
Temporal Graph Benchmark for Machine Learning on Temporal Graphs
We present the Temporal Graph Benchmark (TGB), a collection of challenging and diverse benchmark datasets for realistic, reproducible, and robust evaluation of machine learning models on temporal graphs.
Fast and Attributed Change Detection on Dynamic Graphs with Density of States
Current solutions do not scale well to large real-world graphs, lack robustness to large amounts of node additions/deletions, and overlook changes in node attributes.
Laplacian Change Point Detection for Single and Multi-view Dynamic Graphs
how to capture temporal dependencies, and iii).
Spectral Regularization: an Inductive Bias for Sequence Modeling
Various forms of regularization in learning tasks strive for different notions of simplicity.
Sequential Density Estimation via Nonlinear Continuous Weighted Finite Automata
Although WFAs have been extended to deal with continuous input data, namely continuous WFAs (CWFAs), it is still unclear how to approximate density functions over sequences of continuous random variables using WFA-based models, due to the limitation on the expressiveness of the model as well as the tractability of approximating density functions via CWFAs.
High-Order Pooling for Graph Neural Networks with Tensor Decomposition
Graph Neural Networks (GNNs) are attracting growing attention due to their effectiveness and flexibility in modeling a variety of graph-structured data.
Lower and Upper Bounds on the Pseudo-Dimension of Tensor Network Models
These results are used to derive a generalization bound which can be applied to classification with low rank matrices as well as linear classifiers based on any of the commonly used tensor decomposition models.
Rademacher Random Projections with Tensor Networks
Random projection (RP) have recently emerged as popular techniques in the machine learning community for their ability in reducing the dimension of very high-dimensional tensors.
Lower and Upper Bounds on the VC-Dimension of Tensor Network Models
These results are used to derive a generalization bound which can be applied to classification with low rank matrices as well as linear classifiers based on any of the commonly used tensor decomposition models.
Extracting Weighted Automata for Approximate Minimization in Language Modelling
The objective is to obtain a weighted finite automaton (WFA) that fits within a given size constraint and which mimics the behaviour of the original model while minimizing some notion of distance between the black box and the extracted WFA.
Assessing the Impact: Does an Improvement to a Revenue Management System Lead to an Improved Revenue?
This is a challenging problem as it corresponds to the difference between the generated value and the value that would have been generated keeping the system as before.
Quantum Tensor Networks, Stochastic Processes, and Weighted Automata
We address this gap by showing how stationary or uniform versions of popular quantum tensor network models have equivalent representations in the stochastic processes and weighted automata literature, in the limit of infinitely long sequences.
Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning
In this paper, we present connections between three models used in different research fields: weighted finite automata~(WFA) from formal languages and linguistics, recurrent neural networks used in machine learning, and tensor networks which encompasses a set of optimization techniques for high-order tensors used in quantum physics and numerical analysis.
A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix
Continual learning (CL) is a setting in which an agent has to learn from an incoming stream of data during its entire lifetime.
Adaptive Learning of Tensor Network Structures
However, identifying the best tensor network structure from data for a given task is challenging.
Laplacian Change Point Detection for Dynamic Graphs
To solve the above challenges, we propose Laplacian Anomaly Detection (LAD) which uses the spectrum of the Laplacian matrix of the graph structure at each snapshot to obtain low dimensional embeddings.
Tensorized Random Projections
We introduce a novel random projection technique for efficiently reducing the dimension of very high-dimensional tensors.
RandomNet: Towards Fully Automatic Neural Architecture Design for Multimodal Learning
Almost all neural architecture search methods are evaluated in terms of performance (i. e. test accuracy) of the model structures that it finds.
Tensor Networks for Probabilistic Sequence Modeling
Tensor networks are a powerful modeling framework developed for computational many-body physics, which have only recently been applied within machine learning.
Representation of Reinforcement Learning Policies in Reproducing Kernel Hilbert Spaces
We propose a general framework for policy representation for reinforcement learning tasks.
Efficient Planning under Partial Observability with Unnormalized Q Functions and Spectral Learning
Learning and planning in partially-observable domains is one of the most difficult problems in reinforcement learning.
Neural Architecture Search for Class-incremental Learning
To understand how to expand a continual learner, we focus on the neural architecture design problem in the context of class-incremental learning: at each time step, the learner must optimize its performance on all classes observed so far by selecting the most competitive neural architecture.
Clustering-Oriented Representation Learning with Attractive-Repulsive Loss
The standard loss function used to train neural network classifiers, categorical cross-entropy (CCE), seeks to maximize accuracy on the training data; building useful representations is not a necessary byproduct of this objective.
Hierarchical Methods of Moments
Spectral methods of moments provide a powerful tool for learning the parameters of latent variable models.
Sequential Coordination of Deep Models for Learning Visual Arithmetic
Achieving machine intelligence requires a smooth integration of perception and reasoning, yet models developed to date tend to specialize in one or the other; sophisticated manipulation of symbols acquired from rich perceptual spaces has so far proved elusive.
Connecting Weighted Automata and Recurrent Neural Networks through Spectral Learning
In this paper, we unravel a fundamental connection between weighted finite automata~(WFAs) and second-order recurrent neural networks~(2-RNNs): in the case of sequences of discrete symbols, WFAs and 2-RNNs with linear activation functions are expressively equivalent.
Learning Graph Weighted Models on Pictures
Graph Weighted Models (GWMs) have recently been proposed as a natural generalization of weighted automata over strings and trees to arbitrary families of labeled graphs (and hypergraphs).
Tensor Regression Networks with various Low-Rank Tensor Approximations
We evaluate the compressive and regularization performances of the proposed model with both deep and shallow convolutional neural networks.
Multitask Spectral Learning of Weighted Automata
We first present a natural notion of relatedness between WFAs by considering to which extent several WFAs can share a common underlying representation.
On overfitting and asymptotic bias in batch reinforcement learning with partial observability
This paper provides an analysis of the tradeoff between asymptotic bias (suboptimality with unlimited data) and overfitting (additional suboptimality due to limited data) in the context of reinforcement learning with partial observability.
Neural Network Based Nonlinear Weighted Finite Automata
Weighted finite automata (WFA) can expressively model functions defined over strings but are inherently linear models.
Low-Rank Regression with Tensor Responses
This paper proposes an efficient algorithm (HOLRR) to handle regression tasks where the outputs have a tensor structure.
Higher-Order Low-Rank Regression
This paper proposes an efficient algorithm (HOLRR) to handle regression tasks where the outputs have a tensor structure.
Low-Rank Approximation of Weighted Tree Automata
We describe a technique to minimize weighted tree automata (WTA), a powerful formalisms that subsumes probabilistic context-free grammars (PCFGs) and latent-variable PCFGs.
Learning Negative Mixture Models by Tensor Decompositions
Building upon a recent paper on tensor decompositions for learning latent variable models, we extend this work to the broader setting of tensors having a symmetric decomposition with positive and negative weights.
