Search Results for author:
Found 6 papers, 2 papers with code
TextTN: Probabilistic Encoding of Language on Tensor Network
As a novel model that bridges machine learning and quantum theory, tensor network (TN) has recently gained increasing attention and successful applications for processing natural images.
TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling
Transformer has been widely-used in many Natural Language Processing (NLP) tasks and the scaled dot-product attention between tokens is a core module of Transformer.
Interpretable Network Structure for Modeling Contextual Dependency
Then, we show that the lower bound of such a separation rank can reveal the quantitative relation between the network structure (e. g. depth/width) and the modeling ability for the contextual dependency.
Leveraging Entanglement Entropy for Deep Understanding of Attention Matrix in Text Matching
We prove that by a mapping (via the trace operator) on the high-dimensional matching matrix, a low-dimensional attention matrix can be derived.
A Tensorized Transformer for Language Modeling
In this paper, based on the ideas of tensor decomposition and parameters sharing, we propose a novel self-attention model (namely Multi-linear attention) with Block-Term Tensor Decomposition (BTD).
A Generalized Language Model in Tensor Space
Theoretically, we prove that such tensor representation is a generalization of the n-gram language model.
