Is there a case for specialized LLMs in security use cases?

Medicine got Med-PaLM, finance got BloombergGPT, law got a wave of legal fine-tunes. Security was always going to get its specialized models, and by 2025 it had serious ones. The question worth asking is not "do security LLMs exist" but "under what conditions does a security-specialized model beat a strong general model wrapped in retrieval, tools and domain scaffolding" — because that second configuration is what any competent AI SOC ships by default, and it is the baseline a fine-tune has to clear.

Three teams, three different bets

Google: specialize the model, ground it in intel. Sec-Gemini v1, announced in April 2025, is an experimental cybersecurity model combining Gemini with near-real-time feeds from Google Threat Intelligence, OSV and Mandiant. Google reported it outperforming comparable models by at least 11% on CTI-MCQ (threat-intelligence knowledge) and at least 10.5% on CTI-RCM (mapping vulnerability descriptions to CWE root causes). Those are real, published deltas on public benchmarks — worth taking seriously. Worth equal attention: much of the claimed advantage rides on the integrated intelligence feeds. How much comes from specialized training versus privileged retrieval is not separable from the outside, and Google's own SecOps stack runs SecLM as a platform of models plus RAG rather than a single tuned oracle — which reads to us as an admission that grounding does heavy lifting.

Microsoft: don't specialize the model at all. Security Copilot took the opposite bet: a general frontier model (GPT-4 at launch) combined with Microsoft's threat-intelligence graph, product plugins and orchestration. Whatever you think of the product, the architecture is a large public vote for "general model plus domain grounding" from the company with more security telemetry than almost anyone. If narrow security pretraining were decisively better, Microsoft had the data to build it.

Cisco: specialize small, openly. Foundation-Sec-8B (April 2025) is a Llama 3.1 8B continued-pretrained on a curated security corpus, released with open weights. The published numbers are the most interesting in this whole discussion: on CTIBench-RCM it scored 75.26 against 66.43 for its Llama 3.1 8B base and 72.66 for Llama 3.1 70B — an 8B security model edging out a general model nearly nine times its size on an in-domain task, while staying near its base on MMLU. The follow-up instruct and reasoning variants (technical reports through 2025 and January 2026) held that pattern: parity with 70B-class general models on security benchmarks at an eighth or less of the serving cost. Cisco also reported that fine-tuning the security base for ATT&CK-technique extraction beat fine-tuning the equivalent general base.

Read together: nobody serious is betting on a narrow model to replace general capability. The wins cluster where the task is narrow, the domain vocabulary is dense, and the model can be small.

Why the specialized-investigator dream keeps not arriving

Two structural problems cap how far security specialization can go, and neither is fixable with more GPUs.

The data problem. The corpus that would teach a model to investigate — full incident timelines with labeled root causes, analyst decisions and outcomes — is the most sensitive data a company holds, is scattered across thousands of organisations that will not pool it, and is scarce even internally: a busy SOC confirms far fewer true incidents a year than it closes alerts in a day. What is abundantly trainable is public security text: CVE descriptions, threat reports, ATT&CK, vendor advisories. That is why the published specialized-model wins land on CTI knowledge and CWE mapping — the tasks the available corpus actually covers — and not on end-to-end investigation. There is also a moving-target problem: attacker tradecraft shifts monthly, and weights baked at training time age in a way that retrieval does not.

The reasoning problem. Investigation quality tracks general long-horizon capability — tool use, state tracking, hypothesis revision — which is exactly what frontier labs improve fastest and what a domain corpus does not teach. CyberSOCEval (Meta and CrowdStrike, September 2025) found no model near saturation on SOC reasoning tasks and, strikingly, that reasoning-tuned models did not gain the improvements on cyber tasks that the same tuning delivered in maths and coding. The authors suggest closing that gap may require training on the investigation task itself — which circles straight back to the data problem above. We cite this against our own position deliberately: it is the best current argument that some form of specialization will eventually matter for investigation. It just is not deliverable with today's public corpora.

What production taught us

We have run the experiment customers ask about. In one composite engagement in late 2025, a customer asked us to slot a 70B-class security fine-tune in as the lead investigation model, reasoning that domain fluency would beat a general frontier model. It read beautifully — correct ATT&CK references, idiomatic analyst prose. The evidence told a different story within three weeks: unassisted completion of multi-step investigation playbooks ran roughly 20 points below the frontier configuration on the same alert mix. The recurring failure was fluent tunnel vision: the model pattern-matched alerts to the nearest well-known campaign from its training data and investigated that, and our citation gate kept flagging verdicts whose graph evidence didn't support the campaign story. Domain vocabulary made the failures more convincing, not less frequent. The customer moved investigation back to a general frontier model and kept the fine-tune for report drafting — where its fluency is an asset and its judgement is not load-bearing.

The opposite experiment also ran, and it succeeded. At two sovereignty-constrained deployments we moved the high-volume narrow work — log parsing to structured graph facts, entity extraction, first-pass phishing verdicts — from the general model plane to small security-tuned open models in the Foundation-Sec-8B mould. Composite results: extraction agreement with the frontier configuration in the mid-90s percent on sampled audits, at well under a tenth of the serving cost, fully on-prem, with latency that let us push parsing to near-ingest-time instead of on-demand. Phishing first-pass verdicts (verdict plus cited indicators, human review above a risk threshold) held audited accuracy on par with the previous configuration. Caveats as always: operational medians, dissimilar sites, not a controlled study.

The specialized model didn't replace the investigator. It replaced the parser — and freed the investigator budget for the model that's actually good at investigating.

Our position, stated plainly

For investigation — the long-horizon, evidence-weighing core of SOC work — retrieval plus tools plus domain scaffolding on a strong general model beats a narrow fine-tune, in our production experience and by the visible architecture choices of Microsoft's and, in effect, Google's platforms. The domain knowledge a fine-tune bakes into weights, we deliver as live graph context and threat-intel retrieval — which stays current and cites its sources, two things weights cannot do.

For high-volume narrow tasks — log parsing, entity extraction, classification, phishing first-pass — small specialized models have crossed from interesting to economically obvious, and the Foundation-Sec-8B results give the published grounding for what we see operationally: in-domain, a well-fed 8B can punch at 70B weight. Netgraph's BYO-LLM plane treats them as first-class citizens, and several of our deployments run a frontier investigator over a fleet of small security-tuned workers.

So: is there a case for specialized LLMs in security? Yes — a narrower and more interesting one than the pitch decks claim. Specialization is winning as infrastructure, not as oracle. If the field ever assembles the cross-organisation incident corpus that investigation-tuning would need — consented, anonymised, at scale — that answer could change, and the CyberSOCEval results suggest it might change meaningfully. We are watching for it. We are not holding our breath, and neither should your SOC roadmap.

Deployment anecdotes are anonymised composites of representative real outcomes; operational figures are medians across engagements, not controlled evaluations. Published sources cited: Google Security Blog, "Google announces Sec-Gemini v1" (April 2025); Google Cloud SecOps documentation on SecLM; Microsoft Security Copilot public architecture materials; Cisco Foundation AI, Foundation-Sec-8B technical reports (2025–January 2026); Meta and CrowdStrike, CyberSOCEval (September 2025).