CrowdStrike states: "The issue has been identified, isolated and a fix has been deployed." It would seem that the official fix is as follows:
Workaround Steps:
Boot Windows into Safe Mode or the Windows Recovery Environment
Navigate to the C:\Windows\System32\drivers\CrowdStrike directory
Locate the file matching “C-00000291*.sys”, and delete it.
Boot the host normally.
As someone who is not a sysadmin, how the hell is locally logging in to millions of servers and devices stuck in a bootloop a "fix"? I understand that Windows Safe Mode is unavailable in encrypted devices, what do then?
how the hell is locally logging in to millions of servers and devices stuck in a bootloop a "fix"?
It's a fix as in it fixes the bootloop without having to format the whole system. The systems affected will bootloop as they will BSOD while Windows is booting and there is NOTHING else to be done if the update reached the system.
I understand that Windows Safe Mode is unavailable in encrypted devices, what do then?
It is available, but you need the Bitlocker recovery key which good luck finding the key on the AD domain controllers if those are affected by the boot loops(though keys can also be backed up on Microsoft Azure AD/Entra or if the sysadmin implemented another policy to back them up from AD). Bonus points if the devices are BIOS protected or have additional early boot security measures.
I saw a picture posted of Delta airport personnel going to every single airport check-in terminal and kiosk (like where you weigh your bag) and doing this manual fix.
CrowdStrike states: "The issue has been identified, isolated and a fix has been deployed." It would seem that the official fix is as follows:
As someone who is not a sysadmin, how the hell is locally logging in to millions of servers and devices stuck in a bootloop a "fix"? I understand that Windows Safe Mode is unavailable in encrypted devices, what do then?
It's a fix as in it fixes the bootloop without having to format the whole system. The systems affected will bootloop as they will BSOD while Windows is booting and there is NOTHING else to be done if the update reached the system.
It is available, but you need the Bitlocker recovery key which good luck finding the key on the AD domain controllers if those are affected by the boot loops(though keys can also be backed up on Microsoft Azure AD/Entra or if the sysadmin implemented another policy to back them up from AD). Bonus points if the devices are BIOS protected or have additional early boot security measures.
In other words, it's a fuckin disaster and sysadmins around the world are screaming.
I saw a picture posted of Delta airport personnel going to every single airport check-in terminal and kiosk (like where you weigh your bag) and doing this manual fix.
What a clusterfuck!