Client: ScanSense · Healthcare & Life Sciences · US radiology AI provider (80 staff)
Reducing Failed Rollouts for Radiology AI
TechSupport Team supported PyTorch and MLflow deployments for ScanSense’s radiology application. Over five months, the client’s MLflow records show failed model rollouts falling from 22% to 3% and deployment time falling from three weeks to four days.
Client and engagement facts
- Client
- ScanSense
- Industry
- Healthcare & Life Sciences
- Business context
- US radiology AI provider (80 staff)
- Delivery model
- MLOps support
- Application
- Radiology imaging AI with regulated deployment
- Supplied geography
- US
- Company size in source
- 80 staff
Related services: DevOps support
Application stack and tools
Application stack
- Python
- PyTorch
- Kubeflow
- GCP
Tools used
- Kubeflow
- MLflow
- Datadog
Why Support Was Needed
Models passed staging and still failed in production when new annotation batches differed from the staging data. Engineers focused on representative tests and a canary release step. Clinical approval stayed with the client.
Technical Work
- Traced failure pattern to annotation batches absent from the staging set.
- Built a golden test set from production data, added a canary step before promotion.
- Pinned MLflow artifacts and GPU driver versions per model.
Representative Incident
In month three, one model with a bad annotation batch reached production. The team rolled it back in 20 minutes and added a label check before promotion.
Results and how they were measured
| Metric | Baseline | Result | Units | Measurement window | Source and scope |
|---|---|---|---|---|---|
| Failed rollouts | 22% | 3% | % | 5 months | Client MLflow |
| Deployment time | 3 weeks | 4 days | Baseline: weeks; Result: days | 5 months | Client MLflow |
| Inference P95 | 1.4 s | 410 ms | Baseline: s; Result: ms | 5 months | Client Datadog |
Evidence and Limitations
The reporting window is five months. A label-related failure occurred during that period. These measures describe deployment reliability and inference performance.
Model Promotion and Rollback
ScanSense’s clinical lead approved each model. TechSupport Team ran canary and rollback procedures.
Scope a Similar Engagement
Share your model promotion, test-data and rollback process and the approvals it requires. Request a support plan or explore DevOps support.
Get a technical support plan
Tell us which application needs support, what is reaching your engineers and when you need coverage. TechSupport Team will review the fit and explain the next scoping step.
- We reply by email.
- We agree a call about the roles, your stack and the hours.
- If there is a fit, we propose engineers after an assessment.