CrowdStrike releases preliminary review report on Windows large-scale blue screen incident: memory read out-of-bounds error, internal testing has been strengthened

This website reported on July 25 that due to a CrowdStrike failure, approximately 8.5 million Windows computers around the world have experienced a blue screen of death recently, which has also become a hot event of national concern. On July 24, the CrowdStrike official website released a preliminary review report on the Windows widespread blue screen incident, and stated that the comprehensive investigation results will be detailed in a publicly released root cause analysis.

Preliminary review reports indicate that on Friday, July 19, 2024 at 04:09 UTC (12:09 Beijing time), CrowdStrike released a content configuration update for Windows Sensors as part of regular operations to collect information on possible Telemetry data on emerging threat technologies.

These updates are a regular part of the Falcon platform’s dynamic protection mechanism. However, a problematic Rapid Response content configuration update caused Windows system crashes, affecting devices including Windows hosts running Sensor version 7.11 and higher.

These hosts were online and received the update between Friday 19 July 2024 04:09 UTC and Friday 19 July 2024 05:27 UTC. Mac and Linux hosts are not affected.

The bug in the content update was fixed at 05:27 UTC on Friday, July 19, 2024 (13:27 Beijing time). Systems that come online after this time or systems that were not connected to updates during the previous window will not be affected.

CrowdStrike delivers secure content configuration updates to sensors in two ways: content that ships directly with the sensor, and rapid response content updates. Friday's issue involved a rapid response content update with an undetected bug.

When the sensor receives and loads into the content interpreter, the problematic content causes an out-of-bounds memory read, triggering an exception . This unexpected exception was not properly handled, causing the Windows operating system to crash (BSOD).

Software Resilience and Testing

• Improve responsive content testing by using the following test types:

  • Local Developer Testing
  • Content Update and Rollback Testing
  • Stress Testing, Fuzz Testing and Fault Injection
  • Stability Testing
  • Content Interface Testing
• Add additional validation checks to the content validator for fast responsive content. A new check is in progress to prevent such problematic content from being deployed in the future.
• Enhance existing error handling in Content Interpreter.

Responsive content deployment

• Implement a staggered deployment strategy for responsive content, where updates are gradually deployed to larger parts of the sensor library, starting with canary deployments.
• Improved monitoring of sensor and system performance, collecting feedback during rapid response content deployment to guide phased rollout.
• Give customers greater control over the delivery of responsive content updates by allowing granular choice of when and where these updates are deployed.
• Provide content update details via release notes, which customers can subscribe to.

Third Party Verification

• Conduct multiple independent third-party security code reviews.
• Independent review of end-to-end quality processes from development to deployment.

In addition to the initial post-incident review, CrowdStrike is committed to publicly releasing a complete root cause analysis upon completion of the investigation. The original text of the preliminary review report is attached to this site. If you are interested, you can go to learn more details:

https://www.crowdstrike.com/falcon-content-update-remediation-and-guidance-hub/

The above is the detailed content of CrowdStrike releases preliminary review report on Windows large-scale blue screen incident: memory read out-of-bounds error, internal testing has been strengthened. For more information, please follow other related articles on the PHP Chinese website!