Question → hypothesis/design → architecture → evaluation → results → limitations → next questions
Serving behavior constrains every architecture built on top of it. miniserve is a small LLM serving engine written from scratch in PyTorch to separate two effects that are usually discussed together: how requests are scheduled, and how key-value memory is reserved. It runs Qwen3-0.6B on Apple Silicon or CPU and states what it does not show.
Continuous batching raises throughput and cuts the wait for new requests at some cost to each user's token rate, and exact-length block reservation helps only when key-value memory is scarce.
Measured on one Apple M1 Max. A canary measurement before and after each cell guards against machine slowdown. No comparison with vLLM or any other server was run.
One complete repetition. A first attempt was discarded after a four-fold slowdown that coincided with the laptop sleeping and waking, and a second run was stopped early. Treat differences under about 10% as noise.
Public and Apache-2.0. Engine, tests and benchmark harness are complete; an independent review recomputed every headline number from the raw results and all matched.