Ideas

What Is an AI Systems Architect?

A working definition — from someone who designs, codes, and operates the systems.

The title gets used two ways, and they are not close to the same job. One is a person who advises on AI systems. The other is a person who designs, builds and operates them. I mean the second one, and the distinction is not vanity — it changes what the work produces.

The three things the role actually owns

• The data layer. Where state lives, who can read which row, and how the schema survives contact with a changing product. In my case that is a self-hosted Postgres with row-level security enabled and forced — not a hosted backend with a permissive default.
• The tool surface. What the model is allowed to call, and what each call is permitted to do. A custom MCP server, with write access gated separately from read.
• The prompt infrastructure. Not prompts — infrastructure. Versioned, reusable libraries that turn a brief into a deliverable, the same way every time.

An architect who owns none of those is a consultant. That is a legitimate job; it is a different one.

Why building it changes the advice

The reason to insist on this is not craft pride. It is that the failure modes only show up in operation.

You do not learn that a tool definition is ambiguous until a model calls it wrong at two in the morning. You do not learn that a retrieval strategy is brittle until the corpus grows past the shape you tested. You do not learn what a prompt library actually costs to maintain until the underlying product changes and forty of them are suddenly wrong.

Someone who has only designed these systems will give you an architecture that is defensible on a slide. Someone who has run one will tell you which parts of it will hurt.

The uncomfortable part of the definition

It also means the role is accountable for a number, not a document. If the system does not reduce implementation time or increase throughput, the architecture was wrong — however elegant the diagram was.

That is the version of the title I am willing to defend: you designed it, you wrote it, you run it, and you are on the hook for what it does.

Keep reading