RWKV: Reinventing RNNs for the Transformer Era

5 code implementations • 22 May 2023 • Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Jiaju Lin, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Bolun Wang, Johan S. Wind, Stanislaw Wozniak, Ruichong Zhang, Zhenyuan Zhang, Qihang Zhao, Peng Zhou, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu

This work presents a significant step towards reconciling trade-offs between computational efficiency and model performance in sequence processing tasks.

Computational Efficiency Natural Language Inference

Neuromorphic Optical Flow and Real-time Implementation with Event Cameras

no code implementations • 14 Apr 2023 • Yannick Schnider, Stanislaw Wozniak, Mathias Gehrig, Jules Lecomte, Axel von Arnim, Luca Benini, Davide Scaramuzza, Angeliki Pantazi

Optical flow provides information on relative motion that is an important component in many computer vision pipelines.

Event-based vision Optical Flow Estimation

Dynamic Event-based Optical Identification and Communication

no code implementations • 13 Mar 2023 • Axel von Arnim, Jules Lecomte, Naima Elosegui Borras, Stanislaw Wozniak, Angeliki Pantazi

Temporal pattern recognition, depending on the technology, involves a trade-off between communication frequency, range and accurate tracking.

Event-based Optical Flow Optical Flow Estimation

Learning in Deep Neural Networks Using a Biologically Inspired Optimizer

no code implementations • 23 Apr 2021 • Giorgia Dellaferrera, Stanislaw Wozniak, Giacomo Indiveri, Angeliki Pantazi, Evangelos Eleftheriou

Here, we propose a novel biologically inspired optimizer for artificial (ANNs) and spiking neural networks (SNNs) that incorporates key principles of synaptic integration observed in dendrites of cortical neurons: GRAPES (Group Responsibility for Adjusting the Propagation of Error Signals).