Holdout Set
11 papers with code • 1 benchmarks • 1 datasets
Benchmarks
These leaderboards are used to track progress in Holdout Set
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
xView3-SAR
Most implemented papers
Distribution-Free, Risk-Controlling Prediction Sets
While improving prediction accuracy has been the focus of machine learning in recent years, this alone does not suffice for reliable decision-making.
Generalization in Adaptive Data Analysis and Holdout Reuse
We also formalize and address the general problem of data reuse in adaptive data analysis.
A shared latent space matrix factorisation method for recommending new trial evidence for systematic review updates
dsurian/matfac
•
•
Our aim was to evaluate a new method that could partially automate the identification of trial registrations that may be relevant for systematic review updates.
Template-Based Automatic Search of Compact Semantic Segmentation Architectures
Automatic search of neural architectures for various vision and natural language tasks is becoming a prominent tool as it allows to discover high-performing structures on any dataset of interest.
Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation
The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of...
Generalization of Reinforcement Learners with Working and Episodic Memory
In this paper, we aim to develop a comprehensive methodology to test different kinds of memory in an agent and assess how well the agent can apply what it learns in training to a holdout set that differs from the training set along dimensions that we suggest are relevant for evaluating memory-specific generalization.
RATT: Leveraging Unlabeled Data to Guarantee Generalization
acmi-lab/ratt_generalization_bound
•
•
To assess generalization, machine learning scientists typically either (i) bound the generalization gap and then (after training) plug in the empirical risk to obtain a bound on the true risk; or (ii) validate empirically on holdout data.
Persistent Homology Captures the Generalization of Neural Networks Without A Validation Set
asier-gutierrez/nn-evolution
•
•
NeurIPS 2021
The training of neural networks is usually monitored with a validation (holdout) set to estimate the generalization of the model.
xView3-SAR: Detecting Dark Fishing Activity Using Synthetic Aperture Radar Imagery
diux-xview/xview3-reference
•
•
Unsustainable fishing practices worldwide pose a major threat to marine resources and ecosystems.
Testing for Overfitting
High complexity models are notorious in machine learning for overfitting, a phenomenon in which models well represent data but fail to generalize an underlying data generating process.
