Two posts of AI building blocks raise the platform question: where do teams assemble them with governance instead of scattered API keys and notebook folklore? Azure AI Foundry is Microsoft’s answer, the workbench and control plane for generative AI development, and treating it as estate infrastructure rather than a portal toy is what this post is about.

Hubs, Projects, and the Governance Unit

The structure mirrors the landing zone philosophy: a hub carries the shared, governed configuration, the network isolation (private endpoints, managed VNet), the connections to your AI Search, storage, and model endpoints with their credentials held once, the policy surface, while projects hang off the hub as the team workspaces where the actual building happens. Platform team provisions hubs through Terraform with the invariants baked in; product teams get projects the way they get subscriptions in the vending pattern, self service within guardrails. Identity follows series law, developers authenticate with Entra, workloads with managed identity, and the connection abstraction means a project’s code never sees a raw key, which is precisely the property that makes the inevitable key rotation a non event.

The Catalog and the Deployment Decision

The model catalog spans the Azure OpenAI models, open weight families (Llama, Mistral, Phi and their successors), and task specific models, with three consumption shapes that map to real tradeoffs. Serverless pay per token endpoints for the frontier and popular open models, zero infrastructure, the default for most workloads. Managed compute deployments, the model on GPU capacity you size, for open models needing customization, data proximity, or throughput economics at sustained volume. And the provisioned throughput option from the August OpenAI post for latency and capacity guarantees on the flagship models. The selection discipline that keeps costs and quality honest: benchmark candidate models on your golden evaluation set (the AI Search post’s habit generalized), because leaderboard rank and your task’s reality correlate loosely, and the model that is three times cheaper and two points worse on your eval is usually the right answer for the high volume tier, with the flagship reserved for the queries a router escalates.

Agents, Evaluation, and Tracing

The agent service is the assembly point: a model plus instructions plus tools, grounding through your AI Search indexes, actions through function calling and connectors into the Logic Apps integration surface from September, code execution where enabled, with the service managing the conversation state and tool orchestration loop. The engineering stance to take: agents are software, so their instructions and tool definitions live in the repo, their changes go through review, and their capabilities follow least privilege exactly like any service principal, an agent with a tool is an identity with permissions, and the PIM mindset applies. Evaluation is the promotion gate: the built in evaluators (groundedness, relevance, coherence, safety) plus your task metrics run against the golden set in CI, and a prompt or model change that regresses does not ship, which is the only known cure for the vibes driven prompt editing that quietly degrades production systems. Tracing closes the loop in production, full request trees, prompts, retrievals, tool calls, latencies, flowing into Application Insights beside every other service this series instruments, because an AI feature at 2 AM is debugged with the same discipline as any other feature: from the trace, not the vibes. Content safety wires in by default and gets tomorrow’s post to itself.

Cheers
Osama