New Open-Source SOC Grader Solves the MDR ‘Mean Time to Detect Poor Quality’ Problem

The free tool uses a transparent rubric to score cases consistently – turning reviews into a repeatable feedback loop, with data staying in your environment

PALO ALTO, Calif. – Today, AirMDR released SOC Grader, a first-in-the-market open source tool that solves the challenge many MDR buyers face: it takes them months to discover whether the service they purchased delivers high fidelity, high efficacy, rich investigations. With SOC Grader, the “mean time to detect poor quality” goes from months to minutes.

SOC Grader applies a transparent rubric to audit investigation quality and expose gaps in casework. Available under an MIT license and free to use, it helps security teams measure, coach, and improve case quality over time – so they can verify investigations, not just trust claims.

For years, security leaders have negotiated MDR contracts and tracked speed-centric SLAs, while case quality stayed subjective. SOC Grader shows whether investigations are actually defensible, using a clear scoring model teams can apply consistently. When cases close without a clear plan, evidence, or rationale, leaders are left guessing, trust erodes, and churn follows.

SOC Grader scores investigation write-ups against a published rubric – plan, evidence, context, rationale, and next steps – and provides both a score and specific feedback on how to improve. SOC Grader:

• Scores investigations against a published and modifiable case-quality rubric.
• Returns a score plus specific feedback on what’s missing and what to improve.
• Flags cases missing critical evidence – even when the overall score is high.
• Keeps sensitive security data inside your environment.

Turning the Lights On in Security Operations

SOC Grader is designed for two distinct, high-stakes use cases:

1. Vendor Accountability: Audit MDR casework for verifiable investigations – where conclusions are supported by a clear plan, evidence, and rationale, and not just “case closed.”
2. Internal Coaching: SOC managers can standardize coaching with consistent, rubric-based feedback – moving from sporadic spot checks to broader coverage.

The goal is the same for both: consistent, explainable scoring that turns case review into a repeatable feedback loop. The best MDR providers should welcome this because it makes quality visible and trust measurable.

“SOC Grader came out of a gap we kept seeing – one that industry analysts at Gartner have also highlighted,” said Kumar Saurabh, CEO of AirMDR. “CISOs told us there wasn’t a scalable, repeatable way to judge investigation quality. Gartner analysts have highlighted the challenge that many MDR buyers end up evaluating promises and SLAs more than the day-to-day quality of casework, and trust gaps compound over time. That’s why we released SOC Grader as open source: to give teams a transparent way to measure case quality instead of just trusting the closure. Security Operations runs on proof and trust, and a quality case delivers proof and builds trust.”

Privacy-First and Open Architecture

You can deploy SOC Grader in three flexible ways to audit casework:

• Self-Hosted Open Source: Download and execute the complete Python codebase within your own environment, maintaining data control while integrating with your chosen LLM endpoint via secure API.
• Browser-Based (BYO-Key): Run the tool client-side in your browser using your own LLM API keys. APIs and case data go directly to your chosen LLM provider and are never stored by AirMDR.
• Instant Hosted Audit: Sign up instantly to run a limited number of cases against our managed backend – perfect for a rapid, zero-setup “spot check” of a case.

SOC Grader is free to use and teams can customize the tool to their needs with:

• Customizable Rubrics: Teams can tune the weighting and questions to match their specific technology stack and risk tolerance.
• MIT License: The tool is free to use, modify, and extend.

Availability

The SOC Grader quality scoring tool is available immediately. Security teams can access the tool or download the repository today.

• Direct Access: https://socgrader.airmdr.com
• Repository: https://github.com/AirMDR/SOCGrader
• Additional Resources: https://airmdr.com/soc-case-quality

Find out more in person at the RSA Conference; visit the AirMDR Booth, #6188.

About AirMDR

AirMDR offers an AI MDR service and AI SOC platform that combine agentic AI with human expertise to deliver minutes-fast alert investigations and transparent, audit-ready cases – all using the tools customers already have. Our MDR service is designed for lean security teams and provides 24/7 coverage, while our AI SOC platform supports MSSPs and Enterprise SOC teams looking to accelerate response and maintain consistent outcomes at scale. Learn more at https://airmdr.com/