Mean Bias Error (MBE) measures the average bias in a model’s predictions by computing the mean difference between predicted and actual values.
\r\n\r\nIt indicates whether a model tends to systematically overestimate or underestimate the target variable.
\r\n\r\nMean Bias Error is calculated as the average of the prediction errors.
\r\n\r\n\r\n Mean Bias Error = (1 / N) × Σ ( ŷᵢ − yᵢ )\r\n
Where:
\r\n\r\nA positive MBE indicates overestimation, while a negative MBE indicates underestimation.
\r\n\r\nIf the average prediction error across all samples is −1.5, then:
\r\n\r\n\r\n Mean Bias Error = −1.5\r\n
hi....
","industry":"Fin","flops_per_user":1000000000000000,"owner":{"id":311,"first_name":"waqas","last_name":"khan","profile_image":"https://s3.eu-central-1.amazonaws.com/tracebloc-develop-s3-bucket/xrayimagedata/media/user_profile_photos/user-b023b5000cdf44d49bfbfcd743d497bd.jpeg","is_active":true},"announced_date":null,"start_date":"2026-03-25T11:17:45Z","current_date":"2026-04-03","team_merger_date":null,"final_submission_date":null,"end_date":"2026-05-12T11:16:00Z","created_date":"2026-03-25T11:17:43.373239Z","updated_date":"2026-04-03T14:55:48.338941Z","competition_image":null,"competition_thumbnail":null,"og_image":null,"seo_description_page_title":"","seo_description_page_description":"","seo_leaderboard_page_title":"","seo_leaderboard_page_description":"","seo_eda_page_title":"","seo_eda_page_description":"","keywords":[],"total_participants":0,"high_score":0.0,"test_data_percentage":100,"final_data_percentage":0,"progress_status":"Active","is_sustainable":false,"competition_rules_template":"","score_formula_display":"mean bias error (mbe)","score_formula_description":"Mean Bias Error (MBE) measures the average bias in a model’s predictions by computing the mean difference between predicted and actual values.
\r\n\r\nIt indicates whether a model tends to systematically overestimate or underestimate the target variable.
\r\n\r\nMean Bias Error is calculated as the average of the prediction errors.
\r\n\r\n\r\n Mean Bias Error = (1 / N) × Σ ( ŷᵢ − yᵢ )\r\n
Where:
\r\n\r\nA positive MBE indicates overestimation, while a negative MBE indicates underestimation.
\r\n\r\nIf the average prediction error across all samples is −1.5, then:
\r\n\r\n\r\n Mean Bias Error = −1.5\r\n
About this use case: This page shows how biomarker research teams evaluate candidate multi-omic panels across independent patient cohorts — and how tracebloc makes it possible to run that validation on real patient data without centralising a single record. The scenario and numbers reflect patterns observed across rare disease research environments. The dataset is augmented to represent the statistical structure of real-world rare disease cohorts without containing any identifiable patient data. Explore the data, submit your own model, and see how your panel holds up.
Published biomarker panels rarely survive first contact with an independent cohort. Biomarker validation is the discipline's persistent failure: a panel trained and reported on one institution's patients classifies a second institution's patients significantly worse — often enough to invalidate the clinical claim. For rare disease research teams working across Cystic Fibrosis, Duchenne Muscular Dystrophy, and Spinal Muscular Atrophy, where patient numbers are small and cohorts are hard to assemble, failed replication does not just delay publication. It delays regulatory submission and, ultimately, patient access to treatment.
Dr. Sophie Hartmann, Head of Translational Biomarker Research at a rare disease foundation, deploys a tracebloc workspace loaded with 320 anonymised patient records spanning three rare genetic diseases. Biomarker research teams — internal and external — submit their candidate panel models to the workspace. Inside tracebloc's containerised training environment, each model trains on the 256-record patient cohort — fine-tuning to the specific multi-omic feature distributions, disease subtypes, and clinical covariate patterns present in this dataset — without any patient data leaving Sophie's infrastructure. tracebloc handles orchestration, scores each adapted model against the holdout cohort, and publishes results to a live leaderboard automatically. This is a federated learning application of reproducible benchmarking: contributors test their panels on real patient data, and the data never moves.
In this example evaluation, the best-performing model held up across all three disease subgroups after fine-tuning — but the gap between the top contributor and the second-best was larger on the Spinal Muscular Atrophy cohort than on Cystic Fibrosis, a finding invisible in any single-institution benchmark. The leaderboard captures those subgroup differences persistently. The workspace stays active so that as new candidate panels are developed, they face the same holdout cohort and the same evaluation conditions.
Sophie's team has spent three years assembling a rare disease cohort — 320 patients across Cystic Fibrosis, Duchenne Muscular Dystrophy, and Spinal Muscular Atrophy, each with longitudinal clinical assessments, CFTR mutation profiling, and a suite of functional measures including six-minute walk distance, FEV1 percent predicted, motor function score, and sweat chloride levels. The dataset took two ethics committee cycles, four clinical site agreements, and significant effort to harmonise across measurement protocols. It is, for the field, genuinely valuable.
The problem is that every biomarker panel her team has evaluated — whether developed internally or submitted by academic collaborators — has been trained and tested on the same patient records it was developed on. When those panels are published, other groups attempt to replicate them. They often cannot. Not because the methods are wrong, but because the panels were never tested on an independent cohort before publication.
This is not a niche critique. The biomarker replication crisis is well documented. In rare disease specifically, where each disease subtype may have fewer than 100 patients available globally, overfitting to the available cohort is almost inevitable without systematic independent validation. A six-minute walk distance that predicts progression in Sophie's Cystic Fibrosis patients may perform differently in another centre's cohort where patient age distribution or treatment history differs. A CFTR mutation combination that separates disease classes in one country's registry may not separate them in another's.
The structural problem Sophie faces: the specialists who could validate her panels are at academic centres across Europe. They have their own patient cohorts, their own biomarker expertise, and every incentive to collaborate. But they cannot access her patient records. Ethics approval does not transfer across institutions. GDPR does not permit unilateral data sharing. And even setting aside the legal constraints, centralising sensitive rare disease patient data into a shared research database is a governance commitment that takes months to establish and creates ongoing risk.
She needs a mechanism that lets external research groups test their models on her cohort — calibrating to her specific disease population — without those researchers ever seeing a single patient record.
The evaluation dataset contains 320 anonymised rare disease patient records spanning three disease groups. Full dataset statistics, feature distributions, and class-level analysis are available in the Exploratory Data Analysis tab.
This dataset is augmented. It was constructed to reflect the statistical structure of real-world rare disease cohorts — the disease class distribution, CFTR mutation prevalence patterns, clinical measurement distributions, and feature correlation structure — without containing any identifiable patient information.
| Property | Value |
|---|---|
| Total records | 320 |
| Training cohort | 256 records |
| Holdout cohort | 64 records |
| Features | 254 (250 numerical, 4 categorical) |
| Disease classes | 3 |
| Missing values | None |
| Duplicate records | None |
| Class imbalance ratio | 2.5× |
| Evaluation metric | Multi-class F1 score |
Disease class distribution (full dataset):
| Disease | Patients | Share |
|---|---|---|
| Cystic Fibrosis | 165 | 51.6% |
| Duchenne Muscular Dystrophy | 89 | 27.8% |
| Spinal Muscular Atrophy | 66 | 20.6% |
A note on the features: The 254 features span three categories. Mutation markers (CFTR_c_1 through CFTR_c_100+) are binary variables encoding specific CFTR gene variants — present or absent in each patient. Clinical measures include six-minute walk distance (metres), FEV1 percent predicted, motor function score, sweat chloride concentration (mmol/L), steroid use, and cardiac ejection fraction. SMN1 variant markers are included for the SMA subgroup. No features require imputation — the dataset is complete. The class imbalance (Spinal Muscular Atrophy at 20.6% versus Cystic Fibrosis at 51.6%) is preserved as observed in real cohort distributions. A model that classifies every patient as Cystic Fibrosis achieves 51.6% accuracy — which is why per-class recall and macro-F1 are the metrics that matter here.
Each contributor submitted their candidate biomarker panel model to the tracebloc workspace. The evaluation ran in two phases.
Phase 1 — Out-of-the-box performance. Each model was scored as submitted, with no adaptation to Sophie's patient cohort. This establishes the true baseline: what the published or internally-validated panel actually delivers when applied to an independent patient population without retraining.
Phase 2 — Fine-tuning. Contributors were given access to the training environment inside the tracebloc workspace. Each contributor transferred their model into tracebloc and ran training on the 256-record cohort. This process fine-tuned the model to the specific covariate distributions, mutation prevalence patterns, and disease subtype characteristics in Sophie's patient population — adapting from a generalised biomarker classifier to one calibrated for this specific rare disease cohort. After training, the adapted model was evaluated automatically against the 64-record holdout. Contributors received only their own results back; no contributor had visibility into another's performance before the leaderboard published.
→ View the full model leaderboard — complete contributor rankings, per-class recall, and biomarker feature importance across all submitted panels.
| Contributor | Claimed F1 | Out-of-the-Box | After Fine-tuning | SMA Recall |
|---|---|---|---|---|
| Contributor A | 0.84 | 0.71 | 0.79 | 0.61 |
| Contributor B ✅ | 0.81 | 0.74 | 0.86 | 0.78 |
| Contributor C ⚠️ | 0.88 | 0.77 | 0.83 | 0.52 |
What the numbers reveal:
Contributor B achieved the strongest macro-F1 after fine-tuning at 0.86, and — more critically — held the highest recall on the Spinal Muscular Atrophy subgroup at 0.78. Starting at 0.74 out-of-the-box, the model improved meaningfully after training on 256 real-distribution patient records inside the tracebloc workspace, with its SMA recall rising from 0.63 to 0.78. This is the replication result: not just that the panel works on a new cohort, but that it can be calibrated to perform robustly on the minority subgroup where clinical decisions are hardest.
Contributor C had the highest claimed F1 at 0.88 and the strongest out-of-the-box baseline at 0.77. After fine-tuning it reached 0.83 — a meaningful result, but with SMA recall at 0.52, it misclassifies roughly one in two SMA patients. In a disease where the patient population is small and each misclassification delays or misdirects clinical decision-making, a 52% recall on the rarest subgroup is not a publishable panel.
Contributor A's claimed F1 of 0.84 dropped to 0.71 on the independent cohort before fine-tuning — the largest out-of-the-box degradation in the evaluation. After adaptation it recovered to 0.79, still below both other contributors. The claimed performance was measured on a training distribution that does not match Sophie's patient population.
Illustrative assumptions: Rare disease pipeline with 3 candidate biomarker panels / Phase II endpoint selection decision worth $80M+ in downstream trial investment / Cost of failed replication at regulatory review: 12–18 months delay, estimated €3–6M in additional study costs per episode
| Strategy | SMA Recall | Replication Risk | Regulatory Readiness | Est. Cost of Late Failure |
|---|---|---|---|---|
| Unvalidated (internal only) | Unknown | High | Low — single-cohort evidence | €3–6M per episode |
| Contributor A | 0.61 | Moderate | Partial — SMA subgroup underperforms | €1.5–3M exposure |
| Contributor B ✅ | 0.78 | Low | Strong — reproducible across subtypes | Minimised |
| Contributor C | 0.52 | Moderate-High | Partial — SMA recall insufficient | €2–4M exposure |
The primary value of this evaluation is not the cost saved on one decision — it is the ability to generate reproducible, multi-cohort biomarker evidence without a data sharing agreement, without centralising patient records, and without the 12–18 month governance cycle those agreements require. Contributor B delivers that evidence. The alternative is publishing a panel that has never been independently validated, and discovering at EMA interaction that the evidence base is insufficient.
Sophie selects Contributor B's panel for advancement, with the validation run against her cohort constituting the first independent replication evidence for the panel. The feature importance output from the fine-tuning run identifies which CFTR variant combinations and clinical measures — particularly sweat chloride and motor function score — are driving performance in each disease subgroup, giving her team a mechanistic basis for the panel's design that supports regulatory submission narratives.
The tracebloc workspace stays active after the initial evaluation. New candidate panels — whether developed by Sophie's team or submitted by academic collaborators — face the same holdout cohort under identical evaluation conditions. The leaderboard becomes a persistent record of which biomarker approaches replicate and which do not, turning a one-off validation exercise into ongoing reproducibility infrastructure.
Explore this use case further:
Related use cases: See how the same federated evaluation approach applies to prognostic transcriptomics in neuromuscular disease and combination multi-omics therapy response prediction. For proteomics-based biomarker discovery, see the liquid biopsy rare disease use case. For a broader view of what federated learning applications look like across life sciences, see our guide.
Deploy your workspace or schedule a call.
","industry":"Fin","flops_per_user":1000000000000000,"owner":{"id":311,"first_name":"waqas","last_name":"khan","profile_image":"https://s3.eu-central-1.amazonaws.com/tracebloc-develop-s3-bucket/xrayimagedata/media/user_profile_photos/user-b023b5000cdf44d49bfbfcd743d497bd.jpeg","is_active":true},"announced_date":null,"start_date":"2026-03-25T10:53:18Z","current_date":"2026-04-03","team_merger_date":null,"final_submission_date":null,"end_date":"2026-05-12T10:52:00Z","created_date":"2026-03-25T10:53:16.740970Z","updated_date":"2026-03-31T07:15:36.508589Z","competition_image":null,"competition_thumbnail":null,"og_image":null,"seo_description_page_title":"","seo_description_page_description":"","seo_leaderboard_page_title":"","seo_leaderboard_page_description":"","seo_eda_page_title":"","seo_eda_page_description":"","keywords":[],"total_participants":2,"high_score":0.0,"test_data_percentage":100,"final_data_percentage":0,"progress_status":"Active","is_sustainable":false,"competition_rules_template":"","score_formula_display":"mean bias error (mbe)","score_formula_description":"Disclaimer: The dataset used in this use case is augmented — constructed to reflect the statistical structure of real-world rare disease cohorts, including disease class distribution, CFTR mutation prevalence, clinical measurement distributions, and feature correlation patterns, without containing any identifiable patient information. The persona, contributor names, claimed performance figures, and scenario are illustrative and based on patterns observed across rare disease biomarker research environments. They do not represent any specific institution, research group, or regulatory submission.
Mean Bias Error (MBE) measures the average bias in a model’s predictions by computing the mean difference between predicted and actual values.
\r\n\r\nIt indicates whether a model tends to systematically overestimate or underestimate the target variable.
\r\n\r\nMean Bias Error is calculated as the average of the prediction errors.
\r\n\r\n\r\n Mean Bias Error = (1 / N) × Σ ( ŷᵢ − yᵢ )\r\n
Where:
\r\n\r\nA positive MBE indicates overestimation, while a negative MBE indicates underestimation.
\r\n\r\nIf the average prediction error across all samples is −1.5, then:
\r\n\r\n\r\n Mean Bias Error = −1.5\r\n
Mean Bias Error (MBE) measures the average bias in a model’s predictions by computing the mean difference between predicted and actual values.
\r\n\r\nIt indicates whether a model tends to systematically overestimate or underestimate the target variable.
\r\n\r\nMean Bias Error is calculated as the average of the prediction errors.
\r\n\r\n\r\n Mean Bias Error = (1 / N) × Σ ( ŷᵢ − yᵢ )\r\n
Where:
\r\n\r\nA positive MBE indicates overestimation, while a negative MBE indicates underestimation.
\r\n\r\nIf the average prediction error across all samples is −1.5, then:
\r\n\r\n\r\n Mean Bias Error = −1.5\r\n
Mean Bias Error (MBE) measures the average bias in a model’s predictions by computing the mean difference between predicted and actual values.
\r\n\r\nIt indicates whether a model tends to systematically overestimate or underestimate the target variable.
\r\n\r\nMean Bias Error is calculated as the average of the prediction errors.
\r\n\r\n\r\n Mean Bias Error = (1 / N) × Σ ( ŷᵢ − yᵢ )\r\n
Where:
\r\n\r\nA positive MBE indicates overestimation, while a negative MBE indicates underestimation.
\r\n\r\nIf the average prediction error across all samples is −1.5, then:
\r\n\r\n\r\n Mean Bias Error = −1.5\r\n
This competition is the ideal starting point for you to learn the in’s and out’s of how federated machine learning code competitions on the tracebloc platform work. Think of this competition as a fun space to try out all the features we offer.
The key distinction between this code competition and others found on tracebloc is that it doesn't involve sensitive information. However, we encourage you to handle the data as if it were sensitive, using only the resources provided us. This will help you become accustomed to the exciting competitions that await you in the future.
The goal of this competition is to take some work from the shoulders of doctors and classify medical x-ray mammography images using your machine learning models. If you want to get to know the dataset, simply have a look at the \"Data\" and \"Explorative Data Analysis\" tabs.
Once you’re ready to join the fun, click on the \"join competition\" button.
Did you know that breast cancer is the most common cancer found in women worldwide?
Sadly, breast cancer prevention options are limited, which makes early detection incredibly important to increase the chances of a successful treatment and cure. Mammography is currently the most effective method for detecting breast cancer in women aged 50 to 69 years. Mammography is a medical imaging test utilizing low energy X-rays. During the test, the breast is compressed between two plates and an X-ray image is taken.
The critical point in this challenge is that every X-ray image contains a suspicious mass found by a radiologist. Your challenge will be to use machine learning techniques to accurately separate the malignant masses from the benign masses.
Our competition features the CBIS-DDSM dataset (Curated Breast Imaging Subset of DDSM), an updated and standardized version of the Digital Database for Screening Mammography (DDSM).
The DDSM is a comprehensive database of 2,620 scanned film mammography studies that contains benign and malignant cases with verified pathology information. This dataset is widely regarded as a valuable tool in the development and testing of decision support systems for breast cancer detection.
CBIS-DDSM (Curated Breast Imaging Subset of DDSM), includes decompressed images, data selection and curation by trained mammographers and pathologic diagnosis for training data, formatted like modern computer vision data sets.
Join us in this exciting code competition and help to find new ways of saving lives by further improving breast-cancer screening!
Here you’ll find the evaluation metric your model's performance will be measured with, as well as a short example and explanation. The main objective of the competition is to improve that score by as much as possible.
For this competition, you are tasked with binary classification and need to improve your models F1 score on the test dataset.
The F1 score is the harmonic mean between precision and recall and can be computed like this (for an example ground truth and model prediction) (click here for reference)
\r\n# Example Ground Truth and Model Prediction\r\n\r\nground_truth = [1, 1, 1, 0, 0, 1, 0, 1, 0]\r\nprediction = [1, 1, 0, 0, 1, 1, 0, 1, 0]\r\n\r\n\r\n# implementation using sklearn\r\n\r\nfrom sklearn import metrics\r\nmetrics.f1_score(ground_truth, prediction)\r\n\r\n\r\n# basic implementation\r\n\r\ndef true_positive(ground_truth, prediction):\r\n tp = 0\r\n for gt, pred in zip(ground_truth, prediction):\r\n if gt == 1 and pred == 1:\r\n tp +=1\r\n return tp\r\n\r\ndef true_negative(ground_truth, prediction):\r\n tn = 0\r\n for gt, pred in zip(ground_truth, prediction):\r\n if gt == 0 and pred == 0:\r\n tn +=1\r\n return tn\r\n\r\ndef false_positive(ground_truth, prediction):\r\n fp = 0\r\n for gt, pred in zip(ground_truth, prediction):\r\n if gt == 0 and pred == 1:\r\n fp +=1\r\n return fp\r\n\r\ndef false_negative(ground_truth, prediction):\r\n fn = 0\r\n for gt, pred in zip(ground_truth, prediction):\r\n if gt == 1 and pred == 0:\r\n fn +=1\r\n return fn\r\n\r\ndef recall(ground_truth, prediction):\r\n tp = true_positive(ground_truth, prediction) \r\n fn = false_negative(ground_truth, prediction) \r\n prec = tp/ (tp + fn) \r\n return prec\r\n\r\ndef precision(ground_truth, prediction):\r\n tp = true_positive(ground_truth, prediction) \r\n fp = false_positive(ground_truth, prediction) \r\n prec = tp/ (tp + fp) \r\n return prec\r\n\r\ndef f1(ground_truth, prediction):\r\n p = precision(ground_truth, prediction)\r\n r = recall(ground_truth, prediction)\r\n f1_score = 2 * p * r/ (p + r) \r\n return f1_score\r\n\r\nf1(ground_truth, prediction)\r\n
This competition will run all year and around the clock, but that won’t usually be the case with other competitions.
All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise noted. The competition organizers reserve the right to update the contest timeline if they deem it necessary.
here you’ll find general information about the competition dataset. The data itself is kept federated within our partners and tracebloc’s infrastructure to ensure the privacy of the data. As this is competition is for introductory purposes, the privacy of the data doesn’t really need to be protected. However, this won’t be the case for most of our other competitions.
If you want to learn more about how federated learning works, have a look at this.
This excerpt from the CBIS-DDSM dataset contains 16000 images, with a train / test split of 0.75 / 0.25. The test dataset is further divided into a validation part (3000 images) and a test part (1000 images) for the final evaluation of your model. The images you can see here are 10 sample images that are representative but not part of the actual dataset you will be training your models on.
Your task is binary classification of the X-ray mammography images into the classes “benign” and “malignant”. The dataset contains images of both the right and left breast.
Here you’ll find the evaluation metric your model's performance will be measured with, as well as a short example and explanation. The main objective of the competition is to improve that score by as much as possible.
For this competition, you are tasked with binary classification and need to improve your models F1 score on the test dataset.
The F1 score is the harmonic mean between precision and recall and can be computed like this (for an example ground truth and model prediction) (click here for reference)
\r\n# Example Ground Truth and Model Prediction\r\n\r\nground_truth = [1, 1, 1, 0, 0, 1, 0, 1, 0]\r\nprediction = [1, 1, 0, 0, 1, 1, 0, 1, 0]\r\n\r\n\r\n# implementation using sklearn\r\n\r\nfrom sklearn import metrics\r\nmetrics.f1_score(ground_truth, prediction)\r\n\r\n\r\n# basic implementation\r\n\r\ndef true_positive(ground_truth, prediction):\r\n tp = 0\r\n for gt, pred in zip(ground_truth, prediction):\r\n if gt == 1 and pred == 1:\r\n tp +=1\r\n return tp\r\n\r\ndef true_negative(ground_truth, prediction):\r\n tn = 0\r\n for gt, pred in zip(ground_truth, prediction):\r\n if gt == 0 and pred == 0:\r\n tn +=1\r\n return tn\r\n\r\ndef false_positive(ground_truth, prediction):\r\n fp = 0\r\n for gt, pred in zip(ground_truth, prediction):\r\n if gt == 0 and pred == 1:\r\n fp +=1\r\n return fp\r\n\r\ndef false_negative(ground_truth, prediction):\r\n fn = 0\r\n for gt, pred in zip(ground_truth, prediction):\r\n if gt == 1 and pred == 0:\r\n fn +=1\r\n return fn\r\n\r\ndef recall(ground_truth, prediction):\r\n tp = true_positive(ground_truth, prediction) \r\n fn = false_negative(ground_truth, prediction) \r\n prec = tp/ (tp + fn) \r\n return prec\r\n\r\ndef precision(ground_truth, prediction):\r\n tp = true_positive(ground_truth, prediction) \r\n fp = false_positive(ground_truth, prediction) \r\n prec = tp/ (tp + fp) \r\n return prec\r\n\r\ndef f1(ground_truth, prediction):\r\n p = precision(ground_truth, prediction)\r\n r = recall(ground_truth, prediction)\r\n f1_score = 2 * p * r/ (p + r) \r\n return f1_score\r\n\r\nf1(ground_truth, prediction)\r\n\r\n\r\n
This competition will run all year and around the clock, but that won’t usually be the case with other competitions.
All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise noted. The competition organizers reserve the right to update the contest timeline if they deem it necessary.
Intersection over Union (IoU) measures the overlap between predicted and ground truth regions.
\r\n\r\nIt is commonly used in object detection and image segmentation tasks to evaluate how accurately a model predicts the location and shape of objects.
\r\n\r\nIoU is computed as the ratio of the intersection area to the union area of the predicted and ground truth regions.
\r\n\r\n\r\n IoU = Intersection / Union\r\n
Where:
\r\n\r\nIf the overlapping area between a predicted region and the ground truth is 40 pixels, and the total combined area is 70 pixels, then:
\r\n\r\n\r\n IoU = 40 / 70 ≈ 0.57\r\n