Started reading https://www.cisa.gov/sites/default/files/2024-04/CSRB_Review_of_the_Summer_2023_MEO_Intrusion_Final_508c.pdf and the ES states: "The Board finds that this intrusion was preventable and should never have occurred.”
Is there ever an unpreventable intrusion that should have occurred?
Mastodon Source 🐘
Sounds reasonable and I suspect is at odds with existing organizational & personal incentives.
“The Board recommends that Microsoft’s CEO hold senior officers accountable for delivery against this plan. In the meantime, Microsoft leadership should consider directing internal Microsoft teams to deprioritize feature developments across the company’s cloud infrastructure and product suite until substantial security improvements have been made in order to preclude competition for resources.”
Mastodon Source 🐘
Key rotation is a notoriously deferrable task, until it isn’t:
“Finally, this 2016 MSA key was originally intended to be retired in March 2021, but its removal was delayed due to unforeseen challenges associated with hardening the consumer key systems.”
Mastodon Source 🐘
“Microsoft continued to rotate consumer MSA keys infrequently and manually until it stopped the rotation entirely in 2021 following a major cloud outage linked to the manual rotation process.”
Possibly security corollary to Lorin’s Law: https://surfingcomplexity.blog/2017/06/24/a-conjecture-on-why-reliable-systems-fail/
Mastodon Source 🐘
There’s a theme that the enhanced logging tooling (G5 - TIL) used to produce evidence that confirmed the intrusion was both (a) an additional license that many don’t purchase and (b) a source of a tremendous amount of data that was challenging to decipher.
Mastodon Source 🐘
Electronic notifications sent to victims were ignored by some who “told FBI that they viewed these notifications as possible spam and disregarded them.”
Mastodon Source 🐘
"It requires a security-focused corporate culture of accountability, which starts with the CEO, to ensure that financial or other go-to -market factors do not undermine cybersecurity and the protection of Microsoft’s customers.”
Reminds me of Paul O’Neill’s kickoff meeting about workplace safety: https://davidburkus.com/2020/04/how-paul-oneill-fought-for-safety-at-alcoa/
Mastodon Source 🐘
I would have liked to have been a fly on the wall in this architectural design review meeting. Shout out to everyone in that meeting who argued in favor of isolation ✊:
“Further, if Microsoft had not made the error that allowed consumer keys to authenticate to enterprise customer data”
Mastodon Source 🐘
Security as an optional & paid upgrade doesn’t set customers up for success: “Security-related logging should be a core element of cloud offerings and CSPs should provide customers the foundational tools that provide them with the information necessary to detect, prevent, or quantify an intrusion”
I’ve found Colm’s hierarchy of design tradeoffs super helpful (apologies for the Melon link): https://x.com/colmmacc/status/986286693572493312
Mastodon Source 🐘
A lot of good suggestions for cell-based, strongly isolated, time-bound token design complemented by actual usage metrics that can be used to signal compromise.
Premortem prompt: we’ve been compromised. How do we know? How fast can we know? How far in the past can we look?
Mastodon Source 🐘
One of the Ironies of Notifications is that people disregard notifications or view them as malicious: “In this intrusion, the Board found that some victims ignored or did not see the notifications, and some who saw them believed them to be spam or phishing”
Mastodon Source 🐘
There are a lot of good suggestions in this writeup and CISA has done a lot of work reconstructing the timeline. I have objections to hindsight counterfactuals (eg: “If Microsoft had not paused manual rotation of keys; if it had completed the migration of its MSA environment to rotate keys automatically; …”) but overall there are constructive next steps.
I strongly suspect there are people inside MSFT at the sharp end who were and are already aware of these issues.
Mastodon Source 🐘
I hope this report provides them the support and space to make the changes they’ve been arguing for.