Post-Mortem: Production Availability Incident
August 12, 2026
Summary
On August 12, 2026, a production deployment caused an availability issue that affected portions of the application. A deployment configuration problem caused several services to become unavailable. The issue was corrected and services were restored progressively while system health was monitored.
Impact
Some customers experienced errors or were unable to load parts of the application.
The incident lasted approximately 1 hour and 46 minutes, from 2:07 PM to 3:53 PM MDT. Service began recovering after approximately 50 minutes, with additional functionality returning progressively until full resolution.
Resolution
Our engineering team identified the deployment configuration issue and updated the production deployment to use the correct configuration and application versions.
Services were restored gradually while we monitored application health to ensure the changes were stable before returning to full traffic.
Root Cause
A configuration difference between our deployment environments caused production services to receive an incorrect scaling configuration during a deployment. This resulted in several services becoming unavailable.
The configuration worked correctly in our staging environment, which made the issue less apparent before the production deployment.
Corrective and Preventive Actions
• Standardize deployment configuration between staging and production.
• Add automated checks to prevent unexpected service scaling during deployments.
• Validate production deployment configuration before changes are released..
• Continue using progressive deployments and health monitoring to reduce risk during releases.
We apologize for the disruption and appreciate your patience while our team worked to restore service.
