Systems for AI
How should AI inference systems schedule compute, memory and requests as workloads scale?
Franklin Okpako
My work connects production systems and distributed architecture to AI inference, trustworthy agent infrastructure and usable business workflows. At BlockOps I build model infrastructure; through Agent Rails I investigate the controls around agents; with businesses I am testing how those components can become useful systems.
Built on
I began in systems engineering and moved through networking, cloud engineering, DevOps, security, platform engineering and infrastructure architecture. My focus increasingly became designing observable, fault-tolerant, self-healing systems that reduce unnecessary operational intervention and give teams safe, self-service platforms to build on.
At Exodus, from February 2022 to August 2026, I designed and operated Kubernetes and distributed infrastructure across five AWS EKS clusters, two of them production, supporting wallet and blockchain systems at global scale. That work included GitOps architecture, networking, stateful workloads, observability and incident response, and turning recurring failures into automation and more resilient system design.
Today, my applied work at BlockOps focuses on AI infrastructure: model serving, GPU-backed workloads, isolation, observability and simplifying deployment across cloud and privately managed environments. I am going deeper into inference and serving internals so that design decisions follow from how the components actually work.
Alongside that work, I build Agent Rails, an open-source effort exploring authorization, memory, observability, delegated authority, orchestration and evaluation for autonomous AI systems. Agent Guard is the flagship: a concrete place to examine policy, identity, human approval, audit and execution boundaries together.
I am also working with businesses to understand how these components can become usable AI tools. The SME Agent Template supports two different workflow shapes; workflow validation with businesses and development toward an MVP are in progress, not completed real-world validation.
Read the full research visionHow should AI inference systems schedule compute, memory and requests as workloads scale?
What has to sit around software that acts on its own before it can be trusted to act?
Three systems, each built to answer a question: what batching and KV-cache management change in inference serving (miniserve), what boundary makes an agent's actions safe to authorize (Agent Guard), and how long-lived agents keep memory without it going stale or contradicting itself (MemKit). Agent Rails is the umbrella for the agent-side work, and Vigilo and the rest are in the full list.
Three decisions from production platforms. Each says what the problem was, what I chose, what it cost where that applies, and what happened. The rest of the experience is on the CV.
Understand the infrastructure, build the controls, compose them into workflows, then test whether people can use them.
Teams need a usable path from choosing a model to operating it on cloud or privately managed GPU infrastructure.
Current applied AI-infrastructure work, informed by deeper study of inference runtimes and serving internals.
Read the design and current limits →Control primitives matter only if they can be composed into systems that organizations can understand and operate.
Applied systems work composing Agent Rails controls into a business-facing template; workflow validation is in progress.
Read the design and current limits →The harder problem is deciding what should be remembered, and what should be allowed to fade.
The most interesting failures are not when an agent is wrong, but when it is uncertain and acts anyway.
The repositories are components of one research effort. Agent Guard is the flagship; the other components investigate the memory, evidence and coordination it needs around it.
Who may act, with what authority, and where is it enforced?
What should an agent retain, revise or discard when sources disagree?
What happened, what was claimed, and what can the record establish?
How do these boundaries hold across a complete workflow?
At Exodus, I operated five EKS clusters and built distributed infrastructure and self-service delivery platforms. At BlockOps, I now apply that experience to model serving and deployment infrastructure. The recurring goal is observable, fault-tolerant systems that reduce unnecessary intervention.
AWS · Kubernetes · Linux · Cilium · Terraform · ArgoCD · Cloudflare · reliability · security
The CV has the full record. For roles, collaborations, or consulting on agentic AI infrastructure, write to me.