REBUS: A Robust Evaluation Benchmark of Understanding Symbols

1 code implementation • 11 Jan 2024 • Andrew Gritsevskiy, Arjun Panickssery, Aaron Kirtland, Derik Kauffman, Hans Gundlach, Irina Gritsevskaya, Joe Cavanagh, Jonathan Chiang, Lydia La Roux, Michelle Hung

We propose a new benchmark evaluating the performance of multimodal large language models on rebus puzzles.

Inverse Scaling: When Bigger Isn't Better

no code implementations • 15 Jun 2023 • Ian R. McKenzie, Alexander Lyzhov, Michael Pieler, Alicia Parrish, Aaron Mueller, Ameya Prabhu, Euan McLean, Aaron Kirtland, Alexis Ross, Alisa Liu, Andrew Gritsevskiy, Daniel Wurgaft, Derik Kauffman, Gabriel Recchia, Jiacheng Liu, Joe Cavanagh, Max Weiss, Sicong Huang, The Floating Droid, Tom Tseng, Tomasz Korbak, Xudong Shen, Yuhui Zhang, Zhengping Zhou, Najoung Kim, Samuel R. Bowman, Ethan Perez

Here, we present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale, e. g., due to flaws in the training objective and data.

Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge network

2 code implementations • 23 Sep 2022 • Mario Krenn, Lorenzo Buffoni, Bruno Coutinho, Sagi Eppel, Jacob Gates Foster, Andrew Gritsevskiy, Harlin Lee, Yichao Lu, Joao P. Moutinho, Nima Sanjabi, Rishi Sonthalia, Ngoc Mai Tran, Francisco Valente, Yangxinyu Xie, Rose Yu, Michael Kopp

For that, we use more than 100, 000 research papers and build up a knowledge network with more than 64, 000 concept nodes.

Link Prediction

Capsule networks for low-data transfer learning

no code implementations • 26 Apr 2018 • Andrew Gritsevskiy, Maksym Korablyov

We propose a capsule network-based architecture for generalizing learning to new data with few examples.

Transfer Learning