no code implementations • 19 Jan 2024 • Chaitanya Patel, Shaojie Bai, Te-Li Wang, Jason Saragih, Shih-En Wei
In this work, we first show that the domain gap between the avatar and headset-camera images is one of the primary sources of difficulty, where a transformer-based architecture achieves high accuracy on domain-consistent data, but degrades when the domain-gap is re-introduced.
1 code implementation • 3 Jan 2024 • Evonne Ng, Javier Romero, Timur Bagautdinov, Shaojie Bai, Trevor Darrell, Angjoo Kanazawa, Alexander Richard
We present a framework for generating full-bodied photorealistic avatars that gesture according to the conversational dynamics of a dyadic interaction.
no code implementations • 18 Nov 2022 • Cem Anil, Ashwini Pokle, Kaiqu Liang, Johannes Treutlein, Yuhuai Wu, Shaojie Bai, Zico Kolter, Roger Grosse
Designing networks capable of attaining better performance with an increased inference budget is important to facilitate generalization to harder problem instances.
no code implementations • 13 Jul 2022 • Shaojie Bai, Dongxia Wang, Tim Muller, Peng Cheng, Jiming Chen
To formally analyse the uncertainty to the decision process, we introduce and analyse two important properties of such unbiased trust values: stability of correctness and stability of optimality.
1 code implementation • CVPR 2022 • Shaojie Bai, Zhengyang Geng, Yash Savani, J. Zico Kolter
Many recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms by encouraging iterative refinements toward a stable flow estimation.
Ranked #1 on Optical Flow Estimation on KITTI 2015 (train)
no code implementations • NeurIPS 2021 • Zhichun Huang, Shaojie Bai, J. Zico Kolter
Recent research in deep learning has investigated two very different forms of ''implicitness'': implicit representations model high-frequency data such as images or 3D shapes directly via a low-dimensional neural network (often using e. g., sinusoidal bases or nonlinearities); implicit layers, in contrast, refer to techniques where the forward pass of a network is computed via non-linear dynamical systems, such as fixed-point or differential equation solutions, with the backward pass computed via the implicit function theorem.
1 code implementation • NeurIPS 2021 • Swaminathan Gurumurthy, Shaojie Bai, Zachary Manchester, J. Zico Kolter
Many tasks in deep learning involve optimizing over the \emph{inputs} to a network to minimize or maximize some objective; examples include optimization over latent spaces in a generative model to match a target image, or adversarially perturbing an input to worsen classifier performance.
1 code implementation • NeurIPS 2021 • Zhengyang Geng, Xin-Yu Zhang, Shaojie Bai, Yisen Wang, Zhouchen Lin
This paper focuses on training implicit models of infinite layers.
no code implementations • ICLR 2022 • Shaojie Bai, Vladlen Koltun, J Zico Kolter
A deep equilibrium (DEQ) model abandons traditional depth by solving for the fixed point of a single nonlinear layer $f_\theta$.
1 code implementation • 28 Jun 2021 • Shaojie Bai, Vladlen Koltun, J. Zico Kolter
Deep equilibrium networks (DEQs) are a new class of models that eschews traditional depth in favor of finding the fixed point of a single nonlinear layer.
2 code implementations • ICLR 2022 • Zaccharie Ramzi, Florian Mannel, Shaojie Bai, Jean-Luc Starck, Philippe Ciuciu, Thomas Moreau
In Deep Equilibrium Models (DEQs), the training is performed as a bi-level problem, and its computational complexity is partially driven by the iterative inversion of a huge Jacobian matrix.
2 code implementations • 28 Apr 2021 • Yao-Hung Hubert Tsai, Shaojie Bai, Louis-Philippe Morency, Ruslan Salakhutdinov
In this report, we relate the algorithmic design of Barlow Twins' method to the Hilbert-Schmidt Independence Criterion (HSIC), thus establishing it as a contrastive learning approach that is free of negative samples.
1 code implementation • 13 Aug 2020 • Lars A. Bratholm, Will Gerrard, Brandon Anderson, Shaojie Bai, Sunghwan Choi, Lam Dang, Pavel Hanchar, Addison Howard, Guillaume Huard, Sanghoon Kim, Zico Kolter, Risi Kondor, Mordechai Kornbluth, Youhan Lee, Youngsoo Lee, Jonathan P. Mailoa, Thanh Tu Nguyen, Milos Popovic, Goran Rakocevic, Walter Reade, Wonho Song, Luka Stojanovic, Erik H. Thiede, Nebojsa Tijanic, Andres Torrubia, Devin Willmott, Craig P. Butts, David R. Glowacki, Kaggle participants
The rise of machine learning (ML) has created an explosion in the potential strategies for using data to make scientific predictions.
Ranked #1 on NMR J-coupling on QM9
4 code implementations • NeurIPS 2020 • Shaojie Bai, Vladlen Koltun, J. Zico Kolter
These simultaneously-learned multi-resolution features allow us to train a single model on a diverse set of tasks and loss functions, such as using a single MDEQ to perform both image classification and semantic segmentation.
Ranked #48 on Semantic Segmentation on Cityscapes val
no code implementations • IJCNLP 2019 • Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, Ruslan Salakhutdinov
This new formulation gives us a better way to understand individual components of the Transformer{'}s attention, such as the better way to integrate the positional embedding.
9 code implementations • NeurIPS 2019 • Shaojie Bai, J. Zico Kolter, Vladlen Koltun
We present a new approach to modeling sequential data: the deep equilibrium model (DEQ).
Ranked #29 on Language Modelling on Penn Treebank (Word Level)
1 code implementation • EMNLP 2019 • Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, Ruslan Salakhutdinov
This new formulation gives us a better way to understand individual components of the Transformer's attention, such as the better way to integrate the positional embedding.
4 code implementations • ACL 2019 • Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J. Zico Kolter, Louis-Philippe Morency, Ruslan Salakhutdinov
Human language is often multimodal, which comprehends a mixture of natural language, facial gestures, and acoustic behaviors.
Ranked #5 on Multimodal Sentiment Analysis on MOSI
1 code implementation • ICLR 2019 • Shaojie Bai, J. Zico Kolter, Vladlen Koltun
On the other hand, we show that truncated recurrent networks are equivalent to trellis networks with special sparsity structure in their weight matrices.
33 code implementations • 4 Mar 2018 • Shaojie Bai, J. Zico Kolter, Vladlen Koltun
Our results indicate that a simple convolutional architecture outperforms canonical recurrent networks such as LSTMs across a diverse range of tasks and datasets, while demonstrating longer effective memory.
Ranked #4 on Music Modeling on Nottingham
no code implementations • ICLR 2018 • Shaojie Bai, J. Zico Kolter, Vladlen Koltun
This paper revisits the problem of sequence modeling using convolutional architectures.
Ranked #84 on Language Modelling on WikiText-103