Bugs in Large Language Models Generated Code: An Empirical Study

1 code implementation • 13 Mar 2024 • Florian Tambon, Arghavan Moradi Dakhel, Amin Nikanjam, Foutse khomh, Michel C. Desmarais, Giuliano Antoniol

The bug patterns are presented in the form of a taxonomy.

Attribute Code Generation

Deep Learning Model Reuse in the HuggingFace Community: Challenges, Benefit and Trends

1 code implementation • 24 Jan 2024 • Mina Taraghi, Gianolli Dorcelus, Armstrong Foundjem, Florian Tambon, Foutse khomh

Based on our qualitative analysis, we present a taxonomy of the challenges and benefits associated with PTM reuse within this community.

GIST: Generated Inputs Sets Transferability in Deep Learning

1 code implementation • 1 Nov 2023 • Florian Tambon, Foutse khomh, Giuliano Antoniol

As the demand for verifiability and testability of neural networks continues to rise, an increasing number of methods for generating test sets are being developed.

Bug Characterization in Machine Learning-based Systems

1 code implementation • 26 Jul 2023 • Mohammad Mehdi Morovati, Amin Nikanjam, Florian Tambon, Foutse khomh, Zhen Ming, Jiang

Based on our results, fixing ML bugs are more costly and ML components are more error-prone, compared to non-ML bugs and non-ML components respectively.

Bug fixing

Mutation Testing of Deep Reinforcement Learning Based on Real Faults

1 code implementation • 13 Jan 2023 • Florian Tambon, Vahid Majdinasab, Amin Nikanjam, Foutse khomh, Giuliano Antonio

This allows us to compare different mutation killing definitions based on existing approaches, as well as to analyze the behavior of the obtained mutation operators and their potential combinations called Higher Order Mutation(s) (HOM).

reinforcement-learning Reinforcement Learning (RL)

A Probabilistic Framework for Mutation Testing in Deep Neural Networks

1 code implementation • 11 Aug 2022 • Florian Tambon, Foutse khomh, Giuliano Antoniol

Methods: In this work, we propose a Probabilistic Mutation Testing (PMT) approach that alleviates the inconsistency problem and allows for a more consistent decision on whether a mutant is killed or not.

Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlow

1 code implementation • 26 Dec 2021 • Florian Tambon, Amin Nikanjam, Le An, Foutse khomh, Giuliano Antoniol

This paper presents the first empirical study of Keras and TensorFlow silent bugs, and their impact on users' programs.

How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review

1 code implementation • 26 Jul 2021 • Florian Tambon, Gabriel Laberge, Le An, Amin Nikanjam, Paulina Stevia Nouwou Mindom, Yann Pequignot, Foutse khomh, Giulio Antoniol, Ettore Merlo, François Laviolette

Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems.

BIG-bench Machine Learning Safe Reinforcement Learning

HOMRS: High Order Metamorphic Relations Selector for Deep Neural Networks

1 code implementation • 10 Jul 2021 • Florian Tambon, Giulio Antoniol, Foutse khomh

Deep Neural Networks (DNN) applications are increasingly becoming a part of our everyday life, from medical applications to autonomous cars.

Uncertainty Quantification valid +1