Threat Modeling: Reasoning about Attacks before They Become Incidents

Security work cannot begin and end with a generic checklist. A public photo gallery, payroll system, and medical device face different attackers and consequences.

Threat modeling is a structured way to ask what matters, who might attack it, how the design could be abused, and which controls meaningfully reduce the risk.

It is most useful while a design can still change cheaply, then continues as the system and threats evolve.

A team designing file sharing considers:

unauthorized reads,
guessed public links,
malicious uploads,
stolen sessions,
insider access,
accidental oversharing,
destructive deletion,
and inability to recover.

The exercise leads to scoped links, malware scanning, access logs, short-lived sessions, reviewable permissions, and protected backups.

Define the scope

Choose a specific system, workflow, or change. Record:

users and administrators,
services and data stores,
external providers,
entry points,
deployment environments,
and assumptions.

An unbounded model of “the whole company” becomes vague. A model of “upload, share, and revoke one customer file” can drive design decisions.

Identify assets and consequences

Assets include more than databases:

personal data,
credentials,
money,
service availability,
business decisions,
safety,
reputation,
and audit evidence.

Describe consequences in human and operational terms. “Database read” matters because it could expose medical records or unreleased financial results.

Map data and control flow

Draw how information and authority move between:

browser or app,
API,
background jobs,
storage,
identity provider,
administrators,
and third parties.

Include authentication, secrets, logs, backups, and update paths. Missing arrows often reveal overlooked dependencies.

Mark trust boundaries

A trust boundary is where assumptions or control change.

Examples include:

internet to application,
user tenant to another tenant,
application to third party,
ordinary service to privileged administrator,
and production to backup environment.

Every crossing deserves questions about identity, validation, authorization, encryption, logging, and failure.

Identify threat actors

Consider realistic actors:

unauthenticated attacker,
malicious customer,
compromised account,
insider,
supplier compromise,
automated bot,
and accidental operator.

Specify capability and access. Avoid an all-powerful attacker model that makes every control seem pointless.

Explore abuse paths

Ask how an actor could reach an asset:

bypass authentication,
exploit weak authorization,
inject data or commands,
replay a request,
exhaust resources,
steal a secret,
abuse a legitimate feature,
or compromise a dependency.

Frameworks such as STRIDE can prompt categories, but the goal is plausible system-specific paths.

Include business-logic abuse

Many serious failures use intended features in unintended combinations.

A coupon can be redeemed repeatedly. A support process can reset the wrong account. A public link can be indexed. A refund flow can send money to a changed destination.

These threats may not look like software vulnerabilities, so include product, operations, fraud, privacy, and support perspectives.

Existing controls and assumptions

Record what already reduces risk:

strong authentication,
least privilege,
validation,
network segmentation,
rate limits,
review,
monitoring,
and recovery.

Test assumptions. “The internal network is trusted” or “only administrators know this URL” may not survive a compromised account.

Prioritize

Estimate likelihood, impact, exposure, detectability, and cost to attack. Exact numbers are often uncertain, so use transparent reasoning rather than false precision.

Prioritize threats that combine plausible paths with serious consequences. Also address low-likelihood catastrophic cases when required by safety, law, or business continuity.

Choose mitigations

Possible responses are:

eliminate the risky feature,
reduce stored data,
prevent the action,
limit blast radius,
detect quickly,
require approval,
recover reliably,
transfer risk,
or explicitly accept it.

Prefer design changes that remove entire classes of failure over warnings that depend on perfect human behavior.

Assign owners and verify controls

Every mitigation needs:

an owner,
implementation location,
completion state,
verification method,
and residual risk decision.

“Use encryption” is not complete until the team defines data, protocol, key custody, and tests.

Model failure and recovery

Ask:

What if identity is unavailable?
What if a region is compromised?
What if logs are erased?
What if backups share credentials with production?
What if revocation is delayed?

Security includes containment and restoration, not only initial prevention.

Keep the model alive

Update it when:

architecture changes,
new tools or data appear,
permissions expand,
incidents occur,
threat intelligence changes,
or controls prove ineffective.

Link threats to tests, design records, tickets, and monitoring so the model influences engineering after the workshop.

Make modeling collaborative

The people who know the useful details are rarely in one role. Bring together engineering, product, security, privacy, operations, support, and domain specialists for the scoped workflow.

An engineer may know the network path, while support knows how identity recovery is abused and operations knows which emergency access bypasses normal controls.

Keep the session concrete:

walk through one realistic user action,
mark each data and authority transition,
ask what an attacker or mistake could do there,
capture controls and open assumptions,
and assign the next decision.

The facilitator should encourage challenge without turning the exercise into blame. The system design is under review, not the competence of the people who built it.

Knowledge check

Why should threat modeling have a specific scope?
What is a trust boundary?
Why include business-logic abuse?
How should mitigations be prioritized and tracked?
When should a threat model be updated?

The one idea to remember

Threat modeling connects a concrete design to assets, actors, trust boundaries, abuse paths, consequences, and verified controls. Its value is not the diagram itself but the prioritized design and operational decisions it changes.

A concrete example: file sharing