Can stripping standard JSON formatting out of prompt contexts lower infrastructure overhead and prevent model hijacking in lightweight enterprise deployments?
Large language models rely entirely on their context window to generate the next set of tokens. Over a long operational run, this context accumulates system prompts, past conversation turns, tool responses, user data, reasoning chains, and instructions directed at retrieved documents. The engineering team behind IBM Granite notes that encoding these complex payloads in standard JSON creates a structural bottleneck:
"JSON was designed for parsers, not for causal language models."
Because tokenizers treat structural punctuation marks like quotation marks or colons the exact same way as text characters, models frequently stumble over escaping rules and nested bracket counts. To eliminate this friction, IBM released two experimental preview models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, built around LLMON (LLM Object Notation, pronounced "lemon"), an LLM-native markup language designed for explicitly communicating structure and semantics to models.
Structural Separation Inside the Context Window
How does changing the underlying data syntax prevent an enterprise workflow from failing under attack?
The answer lies in moving structure away from regular character tokens into dedicated control signals. LLMON operates as an XML-like markup whose structural delimiters are reserved special tokens rather than standard characters, keeping schema instructions and message content in separate vocabulary spaces. Data schemas and nesting appear locally so a causal transformer does not have to count brackets across extended text spans. The preview models ship with a dedicated chat template where turns mark individual elements with execution tags: an element marked True binds directly into an execution block at the bottom of the rendered output to dictate which instruction to run, while a part marked False stays in the context without being referenced. Furthermore, two-character block IDs are drawn from a fixed set encoded as single tokens by the tokenizer, keeping block addressing costs fixed at two tokens regardless of payload size.
The architectural shift produces measurable security gains. For some benchmarks, attack success rates for indirect prompt injections hidden inside user data drop to nearly zero with LLMON, whereas regular instruction-tuned models get hijacked with high probability, all while the preview models maintain general capability relative to Granite 4.1 Instruct models.
Managing the Pipeline Migration
A common operational assumption is that adopting specialized syntax requires rewriting enterprise application pipelines from scratch. IBM released a GitHub repository featuring a toolkit for LLMON, workflow examples, templates, and conversion tools that translate existing JSON into LLMON and back without loss. Developers can also use the format in Mellea, which includes native LLMON support by automatically converting its model calls.
Eliminating parsing overhead through native token delimiters provides 3B and 8B models with unambiguous boundaries between runtime instructions and untrusted data payloads. As companies shift reasoning tasks from costly hosted APIs to local on-premise hardware, decoupling context schemas from legacy parser formats like JSON will determine whether compact models can reliably execute multi-turn enterprise workflows without prompt injection vulnerabilities.
Against the field
Two-character block identifiers from a fixed set cost exactly two tokens, turning structural markup from bracket counting into a rigid system protocol. Where lightweight causal models previously stumbled over escaping rules, reserved control tokens effectively neutralize indirect injections without diluting the prompt. The trade-off is that developers dodging prompt hijacking must now route their data pipelines through GitHub converters and custom frameworks like Mellea. Years of forcing causal models to reliably parse human-readable syntax ended the moment engineering simply swapped punctuation for dedicated control tokens.
Our stories: IBM Granite 4.0: Document Data Extraction Without APIs (April 10); IBM Granite 4.2: Local Dense Reasoning for Enterprise (August 29).