The Dependency Problem
JIT access is built on automation: Slack bots, PagerDuty integrations, Okta APIs, AWS STS endpoints. This creates a dangerous dependency chain. What happens when:
- GitHub is down and you cannot deploy a fix.
- Slack is down and the approval bot cannot receive or send messages.
- Okta is degraded and group membership changes are not propagating.
- Your JIT orchestration tool (Sym, Indent) is itself experiencing an outage.
If your JIT system is the only path to production access, and the JIT system is unavailable, you are locked out of your own infrastructure during the exact moment you need access most: a major incident.
This is not a theoretical concern. In 2021, a major cloud provider's IAM service experienced a multi-hour degradation. Organizations without break-glass procedures were unable to respond to their own infrastructure incidents.
The Break-Glass Account Pattern
A break-glass account (also called an emergency access account) is a pre-provisioned, standing-privilege account that bypasses the JIT system entirely. It exists specifically for scenarios where the normal access path is unavailable.
The name comes from the physical analogy: a fire extinguisher behind a glass panel with a sign that says "Break Glass in Emergency." The act of breaking the glass is loud, visible, and leaves evidence — which is exactly the point.
Design principles for break-glass accounts:
1. Minimal count, maximum privilege
Create as few break-glass accounts as possible (typically 2-3 per environment). Each should have the minimum permissions needed to respond to any plausible emergency — which in practice means broad admin access, because you cannot predict what the emergency will require.
2. Vault the credentials
The password and MFA secret for the break-glass account must not be stored in any system that depends on the JIT infrastructure. Options:
- A physical safe with a printed credential sheet (for the most critical accounts).
- AWS Secrets Manager with a separate IAM policy that does not route through your JIT system, accessible only to a small group of named individuals.
- A password manager account with a separate authentication path (not SSO, not Okta).
3. Require a physical MFA token
A YubiKey or similar hardware token stored in a physically secure location (a safe, a locked cabinet). This prevents remote use of the break-glass account — someone must be physically present or have arranged physical access to the token in advance.
4. The burglar alarm
Every login to a break-glass account must trigger an immediate, high-severity alert to the entire security leadership team — regardless of the time of day:
CRITICAL ALERT: Break-glass account 'emergency-admin-01' was used.
Time: 2024-01-15 03:47:22 UTC
Source IP: 203.0.113.42
Action: Console login
This is a P0 security event. Verify with the user immediately.
If this was not authorized, assume breach and initiate IR procedure.
The alert should be delivered via a channel that does not depend on the systems that might be down (e.g., SMS/phone call, not Slack).
5. Mandatory post-use review
Every use of a break-glass account must be followed by a written post-mortem within 24 hours explaining:
- What was the emergency?
- Why could the normal JIT flow not be used?
- What actions were taken with the break-glass account?
- What changes are needed to prevent the same scenario from requiring break-glass access in the future?
This creates accountability and drives continuous improvement of the JIT system itself.
Interviewers testing security maturity often ask: "If JIT is supposed to eliminate standing privileges, isn't a break-glass account a contradiction?" The correct answer acknowledges the tension directly: "Yes, it is a deliberate exception to the ZSP principle, made because the alternative — being locked out of infrastructure during an emergency — is a worse security outcome than a tightly controlled, heavily audited standing account. The break-glass account is acceptable because its use is rare, immediately visible, and always reviewed. The goal of ZSP is to eliminate unmonitored standing access, not to create a system so locked down it cannot respond to emergencies."