# Reducing Failed Rollouts for Radiology AI

> TechSupport Team supported PyTorch and MLflow deployments for ScanSense’s radiology application. Over five months, the client’s MLflow records show failed model rollouts falling from 22% to 3% and deployment time falling from three weeks to four days.

- Publisher: TechSupport Team
- Canonical page: https://techsupport-team.com/case-studies/radiology-ai-model-rollout-reliability/
- Updated: 2026-11-16

Client: ScanSense · Healthcare & Life Sciences · US radiology AI provider (80 staff)

## Client and engagement facts

- **Client:** ScanSense
- **Industry:** Healthcare & Life Sciences
- **Business context:** US radiology AI provider (80 staff)
- **Delivery model:** MLOps support
- **Application:** Radiology imaging AI with regulated deployment
- **Supplied geography:** US
- **Company size in source:** 80 staff

## Application stack and tools

### Application stack

- Python
- PyTorch
- Kubeflow
- GCP

### Tools used

- Kubeflow
- MLflow
- Datadog

Related services: [DevOps support](https://techsupport-team.com/services/devops-support/)

## Why Support Was Needed

Models passed staging and still failed in production when new annotation batches differed from the staging data. Engineers focused on representative tests and a canary release step. Clinical approval stayed with the client.

## Technical Work

- Traced failure pattern to annotation batches absent from the staging set.
- Built a golden test set from production data, added a canary step before promotion.
- Pinned MLflow artifacts and GPU driver versions per model.

## Representative Incident

In month three, one model with a bad annotation batch reached production. The team rolled it back in 20 minutes and added a label check before promotion.

## Results and how they were measured

Table: Reported results and their measurement scope

| Metric | Baseline | Result | Units | Measurement window | Source and scope |
| --- | --- | --- | --- | --- | --- |
| Failed rollouts | 22% | 3% | % | 5 months | Client MLflow |
| Deployment time | 3 weeks | 4 days | Baseline: weeks; Result: days | 5 months | Client MLflow |
| Inference P95 | 1.4 s | 410 ms | Baseline: s; Result: ms | 5 months | Client Datadog |

## Evidence and Limitations

The reporting window is five months. A label-related failure occurred during that period. These measures describe deployment reliability and inference performance.

## Model Promotion and Rollback

ScanSense’s clinical lead approved each model. TechSupport Team ran canary and rollback procedures.

## Scope a Similar Engagement

Share your model promotion, test-data and rollback process and the approvals it requires. [Request a support plan](https://techsupport-team.com/contact/) or explore [DevOps support](https://techsupport-team.com/services/devops-support/).

## Related pages

- [DevOps support](https://techsupport-team.com/services/devops-support/): The related service for this engagement.

## Contact

Request engineers from TechSupport Team or ask a question: https://techsupport-team.com/contact/
