Search Results for author:
Found 22 papers, 17 papers with code
Introducing v0.5 of the AI Safety Benchmark from MLCommons
We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.
Bugs in Large Language Models Generated Code: An Empirical Study
The bug patterns are presented in the form of a taxonomy.
Trained Without My Consent: Detecting Code Inclusion In Language Models Trained on Code
Therefore, auditing code developed using LLMs is challenging, as it is difficult to reliably assert if an LLM used during development has been trained on specific copyrighted codes, given that we do not have access to the training datasets of these models.
Data Cleaning and Machine Learning: A Systematic Literature Review
First, it aims to summarize the latest approaches for data cleaning for ML and ML for data cleaning.
Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challenges
In this paper, we propose an empirical study on Stack Overflow (SO), the most popular Q&A forum for developers, to uncover and understand the challenges practitioners faced when deploying DRL systems.
Bug Characterization in Machine Learning-based Systems
Based on our results, fixing ML bugs are more costly and ML components are more error-prone, compared to non-ML bugs and non-ML components respectively.
Quality Issues in Machine Learning Software Systems
We validate the identified quality issues via a survey with ML practitioners.
Detecting Concept Drift for the reliability prediction of Software Defects using Instance Interpretation
We then compared the output of the proposed methods with baseline methods based on performance monitoring of threshold-dependent and threshold-independent criteria using well-known performance measures in CD detection methods, such as accuracy, MDR, MTD, MTFA, and MTR.
Mutation Testing of Deep Reinforcement Learning Based on Real Faults
This allows us to compare different mutation killing definitions based on existing approaches, as well as to analyze the behavior of the obtained mutation operators and their potential combinations called Higher Order Mutation(s) (HOM).
A Comparison of Reinforcement Learning Frameworks for Software Testing Tasks
In this paper, we empirically investigate the applications of carefully selected DRL algorithms on two important software testing tasks: test case prioritization in the context of Continuous Integration (CI) and game testing.
Quality issues in Machine Learning Software Systems
This empirical study aims to identify a catalog of bad-practices related to poor quality in MLSSs.
GitHub Copilot AI pair programmer: Asset or Liability?
In this paper, we study the capabilities of Copilot in two different programming tasks: (i) generating (and reproducing) correct and efficient solutions for fundamental algorithmic problems, and (ii) comparing Copilot's proposed solutions with those of human programmers on a set of programming tasks.
An Empirical Study of Challenges in Converting Deep Learning Models
Usually DL models are developed and trained using DL frameworks that have their own internal mechanisms/formats to represent and train DL models, and usually those formats cannot be recognized by other frameworks.
Bugs in Machine Learning-based Systems: A Faultload Benchmark
Although most of these tools use bugs' lifecycle, there is no standard benchmark of bugs to assess their performance, compare them and discuss their advantages and weaknesses.
Novel Metric based on Walsh Coefficients for measuring problem difficulty in Estimation of Distribution Algorithms
This information is represented as a probabilistic model and the effectiveness of these algorithms is dependent on the quality of these models.
Silent Bugs in Deep Learning Frameworks: An Empirical Study of Keras and TensorFlow
This paper presents the first empirical study of Keras and TensorFlow silent bugs, and their impact on users' programs.
On Assessing The Safety of Reinforcement Learning algorithms Using Formal Methods
The increasing adoption of Reinforcement Learning in safety-critical systems domains such as autonomous vehicles, health, and aviation raises the need for ensuring their safety.
The challenge of reproducible ML: an empirical study on the impact of bugs
This study attempts to quantify the impact that the occurrence of bugs in a popular ML framework, PyTorch, has on the performance of trained models.
A Stochastic Variance-Reduced Coordinate Descent Algorithm for Learning Sparse Bayesian Network from Discrete High-Dimensional Data
Compared to continuous Bayesian networks, learning a discrete Bayesian network is a challenging problem due to the large parameter space.
How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
Method: We conduct a Systematic Literature Review (SLR) of research papers published between 2015 to 2020, covering topics related to the certification of ML systems.
Improved Reinforcement Learning in Cooperative Multi-agent Environments Using Knowledge Transfer
In the proposed communication framework, agents learn to cooperate effectively and also by introduction of a new state calculation method the size of state space will decline considerably.
Multi-agent Reinforcement Learning
reinforcement-learning
+2
Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approach
To allow for the automatic detection of faults in DRL programs, we have defined a meta-model of DRL programs and developed DRLinter, a model-based fault detection approach that leverages static analysis and graph transformations.
