Honest, detailed technical articles from the development of DocAI: LLM inference tuning, GPU optimisation, document processing pipelines, enterprise AI architecture. Negative results and lessons learned included. New post every week.
The raw data and methodology behind the measurements are public: github.com/k3net/docai-evals
In 64 minutes on a desktop machine we fine-tuned an open language model on the 485 poems of Arany János, the great Hungarian poet, then measured the result with a ruler. On form it got worse: it lost on every axis of regularity to a few-dozen-line reranker that learned nothing and merely asked eight times. Style behaved the other way round: the reranker moved nothing there, and training alone took recognisability from 56% to 92%. Plus the ruler on which Arany himself only scores 0.577, and the price the machine pays in invented words.
We put the twice-as-large Llama-3.3-70B up against our production Qwen3.6-35B-A3B on 100 real Hungarian documents. The expected result: 30 F1 points behind and a 26× slowdown. The unexpected one: the single dimension where Llama looked better was the output of an error detector — and once we checked ground truth, that number turned out to show the exact opposite of reality. Plus the formula that tells you in five minutes whether a model is even in the game on your own hardware.
A vendor publishes an “agent-ready” serving recipe for our model family. We measured all three flags on our own FP8 model: −2.2%, −2.4%, −3.6% — none of them made it in. The interesting part is the third arm: a longer speculative draft is +19% on JSON extraction and −9% on short chat, because the fault line runs along acceptance rate. Plus an uncomfortable bit of self-examination: we measured at concurrency 8 while production peaks at 3.
The same model in NVFP4: +40% throughput, +69% single-stream extraction — and block-FP8 stayed anyway. Our established F1 measurement couldn’t decide (after normalization the gap is 0.0001), and neither could our conversation eval. A third measurement made the call: on 100 real documents, the “which company is us” error jumped from 7 to 12–18. Plus a methodological reckoning and one piece of good news: the error is total, so three lines of code catch 100% of it.
Two vendors, the same DGX Spark, opposite launch commands for the same model — one forbids exactly what the other prescribes. We measured it: the wrong backend is a 2.0–3.3× slowdown, 10 measurement points, 0 crossovers. Then the twist: the same marlin backend wins on NVFP4 and loses on block-FP8. The right backend isn’t a property of the hardware, but of the hardware and the quantization format together — and no vendor model card says so.
Magyar Telekom says token cost is the biggest AI risk — nobody knows how its price will move. We looked at our own numbers: two DGX Sparks, 30 days of Prometheus data, GPU averaging 11.56 W, the whole system ~100 kWh/month, i.e. roughly 6,500 forint. I am not claiming it is cheaper — only that it does not move. The box has a ceiling, the token bill does not. Plus the engineering honesty: when NOT to do it this way.
In 2026 the question is no longer whether to use AI, but where the data ends up. We walk through what is going live now under GDPR and the EU AI Act after the Digital Omnibus, the five real pain points (legal basis, information, data-subject rights, automated decisions, third-country transfer), and why most of compliance is not a legal but an architectural question. Plus: a practical checklist and the decisive question — cloud or on-premise?
This month’s DocAI developments: Partner 360 brings every detail about a partner onto a single screen, company-data lookup now includes EU VIES checks, uploaded bank statements are matched to invoices automatically, there are new contract and payslip registers, and the chat assistant now answers questions about specific invoice line items. Plus: documents outside the NAV system and a Telegram integration.
What is AI document processing, and how do OCR, IDP and KIE differ? We walk through the market’s three solution types — the cloud hyperscaler (Azure Document Intelligence, Copilot), the consultancy project and SharePoint + RAG chat — and where all three give way: the data leaves the company, there’s no Hungarian-language optimisation, the solution is never truly yours. Plus DocAI’s measured answer: on-premise, F1 = 0.975 on a Hungarian corpus.
Eight partner companies, one day from the familiar DocIT foundations to a live DocAI demo. A computer science class from LLM to agent, spontaneous questions run live, a DGX Spark that drew attention even switched off — and an idea that grew out of the conversation: a NIS2 audit module we’re starting this summer.
A systematic head-to-head between a 35-billion and a 122-billion parameter model on DocAI’s own Hungarian business eval harness. The larger model didn’t win on structured extraction and ran 2.6× slower — but on complex multi-step analyses it gave a better answer on five out of six tasks, with five times fewer data lookups. Both models in production, for different purposes. Plus the pitfalls: NVFP4 on sm_121a, FlashInfer MoE, KV-cache FP8 trade-offs.
How did Gree Hungary in Szarvas turn a multi-day approval process into same-day flow? 30–70 documents per day, automated partner verification against four authoritative public sources, 1+ hour daily savings on payments alone, 3–4 month payback — in the honest words of our first customer.
Gemma4 candidate eval on the DocAI Hungarian invoice KIE corpus: F1 0.890 vs Qwen3.6 0.975, single-stream decode 30-80% slower. But the speed measurement’s side effect revealed that MTP acceptance rate on the JSON-KIE workload is 99% — the previous article’s 72.5% global figure hid this completely. The DocAI workload is MTP’s architectural best case.
Qwen3.5-122B-A10B NVFP4 on a single Spark, vLLM 0.19.2, with MTP: 30 tok/s JSON-KIE at 100% MTP acceptance, 64 tok/s aggregate across 4 concurrent users — and the stress test that breaks the Spark when 100K concurrent contexts hit. Production-relevant memory budget, prefix caching, and an honest closing call: is it worth putting in DocAI?
Qwen3.6-35B-A3B-FP8 + MTP (multi-token prediction) benchmark on DGX Spark, GB10 chip. The vanilla model is just as fast as 3.5, but on the 16-concurrent stress test MTP delivered +24% throughput and −56% TTFT — exactly where spec decoding should have been negative in theory. The unexpected symbiosis of unified memory architecture and speculative decoding.
A two-day vLLM + Triton MoE tuning marathon on DGX Spark with Qwen3.5-35B-A3B-FP8. In the end the production config was 5-7% worse. What I learned about the difference between pure-kernel and serving benchmarks — plus six concrete takeaways you can apply.
The next article is already in the works. If you don’t want to miss it, subscribe through the contact form or check back next week.