The moment the question becomes unavoidable
At some point in 2024, the engineering leadership at a B2B SaaS platform serving global enterprise clients made a decision that most SaaS companies were making around the same time.
Not "we should explore AI." Not "let's run a pilot." Non-negotiable. The competitive pressure was real, the board had questions, and the window for being early rather than late was closing fast.
Here is what they did not have at that moment: a clear picture of what AI integration should actually look like for their product.
They knew what they were not. They were not a model training company. They were not building a RAG pipeline or fine-tuning their own models. They were a content infrastructure platform, and whatever AI meant for them, it had to integrate into existing offerings, not replace them.
So they started building. Clarity, as it usually does, arrived through the work rather than before it.
What they built, two AI features, one proven in production and one at prototype stage, teaches something that no AI readiness framework will tell you: the infrastructure challenges of adding AI to a SaaS product are almost never what you expect them to be. Some things that sound hard are simple. Some things that sound simple are the actual work.
Four patterns. Four different infrastructure problems.
Not all AI features are the same infrastructure problem. The compute requirements, the networking architecture, the data preparation work, and the security model all look completely different depending on which pattern your AI feature follows.
What users ask AI to do: Talk to the product, query records, trigger actions using natural language
Primary infrastructure question: Can the AI safely inherit API permissions, latency budgets, and action controls?
What users ask AI to do: Search, summarise, extract, reason over documents and information
Primary infrastructure question: Can the data be ingested, cleaned, indexed, retrieved, and governed?
What users ask AI to do: Draft, generate, rewrite, create assets, messages, code, or descriptions
Primary infrastructure question: Can output quality, brand safety, review, versioning, and IP risk be controlled?
What users ask AI to do: Score, rank, forecast, recommend, optimise decisions
Primary infrastructure question: Can the model be evaluated, explained, monitored, and overridden?
These four patterns are not variations on the same problem. They require different infrastructure decisions at every pillar. This article covers all four at decision level, then goes deep on Pattern 1 because that is where we have direct implementation experience. Pattern 2 is illustrated through a fictional B2B SaaS company.
Not sure which pattern applies to your product? Use the 2-minute pattern selector before reading further — the infrastructure decisions below look very different depending on which pattern you are building.
Pattern 1: Nine lessons from building a real AI interface layer
When the B2B SaaS platform committed to AI integration, the first challenge was not technical. It was definitional.
What does AI actually mean for this product?
For a content infrastructure platform, the answer that emerged was an MCP (Model Context Protocol) server, a bridge that allows AI agents (Claude, GPT, and others) to interact directly with the platform's content API using natural language. Instead of writing API calls, users could describe what they wanted, and the MCP server would handle the translation.
The MCP server did not require a new infrastructure platform. It extended what already existed.
The existing API already handled authentication, data scoping, content model queries, and responses. The MCP server sat in front of it, receiving natural language from LLM agents, translating that into API calls, and returning the results. The bridging layer between the MCP protocol and the API emerged naturally once the decision to build an MCP server was made.
The same principle applied to the second AI feature: an agent harness that allowed users to spin up AI agents inside the platform to handle content creation and editing tasks autonomously. The harness took user instructions, called AWS Bedrock's managed LLM service, received responses, and executed actions in the platform.
An AI interface layer does not process large datasets. It interprets user intent and makes API calls. The compute requirement per request is modest. That said, the API calls the AI interface triggers do still consume compute resources. The compute is shifted, not eliminated.
For the agent harness, the team chose AWS Bedrock's managed LLM service deliberately, avoiding the need to provision, scale, and manage their own GPU infrastructure. Cost scales with usage.
The compute decision framework for Pattern 1:
- If your AI feature interprets intent and calls APIs: use a managed LLM service or third-party API. Do not provision your own GPU infrastructure.
- If your AI feature processes documents or large datasets: re-evaluate. The compute question looks different for Pattern 2.
- For most SaaS teams at Series A-C scale, the crossover point where owned GPU compute makes financial sense is significantly higher than teams assume.
The full round trip for the MCP server, user types natural language, LLM interprets it, MCP server queries the API, result is returned, MCP App renders the UI, comes in at 400 to 500 milliseconds.
For a REST API call, 400 milliseconds is slow. For an LLM agent interaction, it is entirely acceptable. Users interacting with AI assistants expect a brief pause. They do not expect sub-100 millisecond API response times.
Latency expectations differ significantly across the four patterns. If you have not yet confirmed which pattern you are building, the pattern selector takes 2 minutes.
For a content platform adding an AI interface layer, data transfer costs are not the primary networking concern. The payloads are modest, structured content queries, not large file uploads.
The networking question is: how much latency can the AI layer add to an already latency-sensitive product?
For this platform, serving enterprise clients across multiple global regions, the answer required running one MCP server deployment per region. The codebase is shared. The deployment is copied per region. Cost and engineering effort scale directly with the number of regions.
For this platform, "data readiness" did not mean cleaning unstructured documents or building ingestion pipelines. Their data, structured content models, API schemas, field definitions, was already clean and well-organised by the nature of the product.
Data readiness for Pattern 1 means: does the AI understand your content model well enough to query it correctly?
One nuance: the MCP App layer requires content responses to be structured in ways the rendering layer can interpret. The team identified that five to six generalised UI patterns would cover the majority of content type variations. Edge cases fall back gracefully to text-based responses.
When enterprise clients ask "can the AI access our content and nobody else's?", the answer, for a well-architected API product, is yes. By design. Through your existing authentication model.
The MCP server authenticates to the platform API using a Personal Access Token (PAT) created by the user. That token is scoped to the content models the user is authorised to access. It can be further de-scoped, read-only, access to fewer models, if the use case requires it. The AI agent inherits the token's boundaries automatically.
That said: PAT-based auth is one pattern. OAuth and other delegation mechanisms are equally valid. The principle is the same, the AI layer should inherit, not bypass, your existing access controls.
The risk of an AI agent making an incorrect inference and changing content the user did not intend to change is real but lower than it sounds, for two reasons.
First, the user specifies which content model they want to work with before the query. The LLM does not browse or guess, it queries exactly what the user named. This explicit scoping dramatically reduces the chance of wrong content being fetched or modified.
Second, any destructive action requires a clear user choice before it executes. This does not necessarily mean a confirmation dialog. What matters is that the user makes a deliberate, intentional choice rather than being surprised by an action they did not authorise.
The minimum viable governance model for an AI interface layer:
- Explicit model naming before any query
- Clear user choice before all destructive actions
- Blanket authorisation available as a conscious opt-in, not a default
The single most important finding from building both the MCP server and the MCP App layer:
When you build to the protocol specification correctly, the LLM handles the rest.
- MCP server built to spec: LLM agents understand it natively. No custom integration, no additional wiring.
- MCP App built to spec: UI components render in supported AI clients without application-specific rendering logic.
- Graceful fallback from UI to text: handled by the spec behaviour. No additional product-specific engineering.
Pattern 2: The data processing challenge, Meridian HR
Meridian HR is a fictional Series B SaaS company providing HR workflow automation to professional services firms across the UK and Germany. Sixty employees, four-person engineering team, running on AWS eu-west-1. They want to add AI-powered candidate screening and performance review summarisation.
Their infrastructure challenge has almost nothing in common with the B2B SaaS platform above. Different pattern, different problems.
Candidate screening does not require real-time inference. A recruiter uploads a batch of CVs and waits hours for results, not milliseconds. This latency tolerance changes the compute calculus significantly.
- Batch workload: CPU or managed batch processing for ingestion and orchestration; managed LLM APIs for reasoning. Do not start by provisioning owned GPU infrastructure.
- Real-time workload: managed AI API first; consider on-demand GPU only if latency, volume, or privacy requirements justify it.
- For Meridian HR at current scale: API-based AI processing. Pay per document, avoid fixed infrastructure entirely.
The crossover point where owned compute becomes economical is typically far beyond where most Series A-C SaaS teams start. Model it before assuming it.
For Pattern 2, the networking question is about data movement costs, not latency. Three questions Meridian HR's CTO needs to answer:
- Where are documents uploaded from? EU clients uploading to eu-west-1 S3, same region as compute, minimises transfer costs.
- Are AI API calls going through NAT Gateway? VPC Interface Endpoints for AWS Bedrock eliminate this cost.
- What is the data residency requirement? Calling a US-based AI API with EU candidate data without an appropriate transfer mechanism and contracts creates GDPR exposure. That decision should be deliberate, not accidental.
This is where most AI projects for Pattern 2 companies actually fail.
The minimum viable data pipeline for Meridian HR:
For a three-person engineering team, this pipeline is two to three months of work before any AI screening logic is written. Planned for in advance, it is manageable. Discovered mid-implementation when the feature is already promised to clients, it is a crisis.
On vector databases: Postgres with pgvector is often enough for early similarity search. A dedicated vector database becomes relevant when corpus size, filtering complexity, latency SLAs, query volume, or operational complexity outgrow what the existing database can comfortably handle.
Meridian HR's security and governance requirements are not optional overhead. They are legal obligations under GDPR. Three questions that must be answered before launch:
- Does your AI API provider train on submitted data? Get written confirmation, a signed Data Processing Agreement, before any client data flows through.
- Can you support human review and contestability if the AI affects candidate outcomes? GDPR Article 22 creates specific protections around solely automated decisions. Log the input, model version, output, human review, and final decision path before launch.
- Does your AI processing stay within the required geography? Using EU AWS regions helps with residency, but model availability, cross-region inference settings, contracts, and subprocessors still need to be reviewed.
Patterns 3 and 4 below describe the content creation and decisioning patterns. If you are reading this to understand your own product, use the pattern selector to confirm which of the four patterns applies before reading further.
Pattern 3: The content creation challenge
Pattern 3 looks simple until you try to run it in production. A user gives a prompt. The AI produces a draft. The demo works. Then someone asks: what happens when the output is wrong, off-brand, legally risky, or the model changes?
For Pattern 3, infrastructure readiness means prompt and version control, review workflow, output logging, policy checks, human approval, cost tracking, and quality evaluation. The mistake is assuming that because the feature creates text, images, or code, it is mainly a UX feature. It is not. It is a lifecycle and governance feature.
Pattern 4: The decisioning and prediction challenge
Pattern 4 is where AI stops helping users interact or create and starts influencing decisions. That shift changes the infrastructure requirements entirely.
A churn prediction system, lead scoring model, fraud detector, or pricing recommendation engine may look like a simple score in the UI. Behind that score sits a much harder question: can the company trust, explain, monitor, and override the decision?
How the four patterns differ technically
| Dimension | P1 Interface | P2 Data / Knowledge | P3 Content / Creation | P4 Decisioning |
|---|---|---|---|---|
| Compute | Managed LLM API + orchestration | Managed AI API + data pipeline | Managed LLM/media APIs + quality controls | Training/inference/evaluation pipeline |
| Networking | Latency budget + regional deployment | Data movement, residency, NAT routing | Output latency, asset storage, review workflow | Data freshness, feature pipelines, feedback loop |
| Data readiness | API schema, content model, permissions | Document quality, extraction, metadata, retrieval | Brand rules, examples, prompt context, policies | Historical data, labels, feature quality, eval sets |
| Security / governance | Tenant isolation, scoped auth, action confirmation | DPA, residency, sensitive data, audit trail | Human review, IP risk, brand safety, unsafe output | Explainability, bias, human override, drift monitoring |
| Common trap | Building new infra instead of using existing APIs | Discovering data prep work after promising the feature | Treating generation as a demo problem, not lifecycle | Launching a score without evaluation or override |
Before the FAQ — if you have not yet confirmed which of the four patterns your product is building, the pattern selector classifies your AI feature in 2 minutes and flags any governance risks for autonomy and data sensitivity.
Frequently asked questions
What are the four AI patterns for B2B software products?
The four patterns are: AI Interface / API Action Layer, AI Data / Knowledge Processing Layer, AI Content / Creation Layer, and AI Decisioning / Prediction Layer. MCP usually belongs under the interface/API action pattern. RAG usually belongs under the data/knowledge pattern. AI agents are not a separate pattern; they are an autonomy layer on top of any pattern.
How do you prevent an AI agent from accessing content it should not in a multi-tenant SaaS product?
Through your existing token-based access control, no new infrastructure needed. The AI authenticates using a scoped Personal Access Token that defines exactly which content it can access. The agent inherits those boundaries automatically. OAuth and other delegation mechanisms are equally valid; the principle is the same.
Can an AI interface layer meet GDPR and enterprise data isolation requirements?
Yes, when built correctly. An AI interface layer inherits the same data boundaries that govern your existing API access. Enterprise clients asking "can the AI access our content and nobody else's?" get a clear answer: yes, through the same authentication scoping that already controls their API permissions.
Do SaaS companies need to buy GPU infrastructure to add AI features?
Almost never at Series A-C scale as the first move. For AI interface layers, the heavy compute is handled by the LLM service. For AI data processing, managed services like AWS Bedrock can handle model compute without requiring you to provision GPU infrastructure. The crossover point should be modelled rather than assumed.
What is the biggest infrastructure mistake teams make when adding their first AI feature?
Starting the infrastructure conversation before answering the product question. The infrastructure requirements for each AI pattern are fundamentally different. Teams that assume adding AI is one infrastructure problem often build the right infrastructure for the wrong pattern.
How long does it realistically take to go from "we have data" to "our data is AI-ready"?
For Pattern 2 companies with unstructured data in multiple formats, typically two to three months of data engineering work for a small team before any AI feature logic is written. The audit takes a day. The preparation work takes months.
The question that changes everything
Before committing to an AI feature, before choosing a compute model, before reviewing VPC configuration, one question changes everything:
The B2B SaaS platform knew AI integration was non-negotiable. What they did not know, what took time to emerge, was what AI-native actually meant for a content infrastructure product. The MCP server was the right answer. But it only became obvious through the work of building toward a question that had not been fully articulated at the start.
Most infrastructure problems are solvable once the product question is clear. Most infrastructure problems that become expensive surprises mid-implementation exist because the product question was not clear at the start.