Ops Log tag
#Self-hosted LLM
Reviewed Ops Log notes tagged Self-hosted LLM, written by Michal Jatczak with source links, test context and operational checks.
2 reviewed notes
Operator RunbookAI & Automation
Self-hosted LLM serving in the EU: runtime choice, GPU sizing, and the sovereignty argument
A runbook for deciding whether to self-host an open-weights model in the EU: the runtime field after TGI entered maintenance mode, GPU sizing computed from the model config instead of copied from a table, a corrected token-throughput cost model, and the DORA, NIS2 and AI Act record you will be asked for.
By Michal Jatczak
Lab NoteAI & Automation
Retrieval over a permissioned corpus: ACL handling, the update pipeline, and what to measure
A bench specification for retrieval over a corpus that carries permissions: the four access-control failure modes, why post-filtering starves an approximate index, where the cache breakpoint belongs, and the six numbers to record on every run.
By Michal Jatczak