On a synthetic test dataset, the model delivered 55 correct verdicts out of 55 without a single error, but on 100 real alerts from a laboratory testbed, it groundlessly escalated 39 events to Tier 2 and fabricated non-existent incidents.
In March 2026, specialized research noted that AI agents in security operations centers can save up to 30% in response time during infrastructure downtime. With a typical outage running from 8 to 12 hours, this saves up to 4 hours, which reduces problem remediation costs by millions of dollars a year for large enterprises, as noted in the article "AI Agents in SOCs Save Up to 30% of Response Time" dated 2026-03-26. However, attempting to run autonomous event processing within a closed perimeter without cloud services exposed the flip side of the math: automating Tier 1 via local large language models requires strict preliminary filtering, otherwise the costs of re-verifying false positives cancel out the projected gains.
First-Line Architectural Barrier
Tier 1 analysts are overloaded with repetitive alerts, and corporate policies prohibit transmitting raw logs to external cloud interfaces. The engineering experiment centered around the kezekshi assistant, built on a fine-tuned Gemma 4 E2B under the Apache 2.0 license, capable of running locally on a GPU with 8 GB of VRAM. The selected model relies on a Mixture of Experts (MoE) architecture, where generating each token activates only about 10 billion parameters instead of all 122 billion, as reported in the article "Local LLMs in Cybersecurity: GPU Computing and SOC Economics" dated 2026-07-18. The system ingests an evidence bundle from E1 to En, returning a structured JSON containing a verdict, description, evidence references, and mapping to the MITRE ATT&CK framework.
Yet how did the system perform when confronted with routine system background noise?
It turned out that on high-frequency testbed rules absent from the training synthetics—such as registry integrity monitoring events FIM 750, 751, 752 or rootcheck 510 checks—the model inflated the threat level and generated spurious context. To suppress noise, the pipeline was supplemented with 215 real legitimate examples and strict prompt rules. A single directive establishing that granting OAuth application permissions, changing privileges, or adding multi-factor authentication methods without confirmation must be classified as at least suspicious restored coverage to 20 out of 20 without hardware fine-tuning.
"Out of 3,000 testbed alerts, 98.1% were resolved without the model, and 1.9% went to it. This is roughly 50 times fewer inferences while maintaining the same auto-closure decisions."
Deterministic operational noise filtering and whitelist event verification prior to querying the neural network reduced GPU query volume by 50-fold while preserving the automated closure quality of trivial incidents.
Limits of Autonomy and Process Economics
Scaling training data to 3,546 examples in version six introduced a technical defect: on unfamiliar scenarios, 9 out of 20 cards failed validation due to corrupted internal identifiers within the links. The syntax issue was resolved via a strict dynamic Ollama output schema tailored to each alert, yet this rendered link verification purely formal: confirming an identifier's existence does not guarantee its semantic relevance to the output.
The severity rubric initially reduced threat detection recall from 20 out of 20 to 16 out of 20, missing OAuth incidents as standard actions. Ultimately, the security perimeter was downgraded strictly to an assistant mode: deterministic algorithms absorb background noise, while an on-duty specialist verifies any neural network outputs.
How many resources and hours will the team spend rebuilding the pipeline and manually validating tickets if the model continues to inflate severity on every non-standard rule beyond five attack families?
How the picture shifted
By mid-July, the chosen design proved capable of activating roughly 10 billion parameters to operate on modest hardware, yet its flawless run on synthetic data immediately fell apart against mundane system logs. Will the local SOC assistant maintain reliable threat classification once alert volumes expand beyond its 215 vetted production baselines without human intervention?
Our stories: On-Premise LLMs in Cybersecurity: GPU Economics for SOCs (July 18).