BRK442 Breakout Thu 19 Nov 09:00 PST
Build sovereign AI with Azure Local and Foundry Local
Kushmeen Kambow, Anders Zwartjes
Summary
From the recording / Microsoft AI summary
The session defined sovereign AI as running workloads under set controls, with models, prompts, indexes, embeddings and inference logs kept in the customer's environment, connected or disconnected. Foundry Local on Azure Local and Foundry Local knowledge are generally available; tools (MCP), Agent Service and Guardrails are in public preview. A Woodgrove Bank demo covered model evaluation and two agents with different data access. Pricing and licensing were not stated, the Blackwell GPU name is inconsistent in captions, and preview is only targeted for the first half of next year.
Session description
Microsoft's catalogue description
Learn how Foundry Local on Azure Local brings model deployment, evaluation, agents, retrieval, and scalable inference to customer-controlled datacenters, factories, branches, and sovereign environments. See live demonstrations, evaluate model quality without data leaving the environment, and explore patterns for sovereign AI, disconnected operations, multi-GPU inference, and Azure-consistent management.
Demo timestamps
Microsoft's player does not support links to a specific time, so open the recording and skip to the time shown.
- Skip to 0:07:06
Model catalogue and evaluation with Foundry Local
Installs the Foundry Local extension on Arc-enabled AKS on Azure Local, browses 300+ catalogue entries, scores three models against a CSV of banking questions, and shows OpenAI-compatible secured endpoints.
- Skip to 0:13:33
Deploying a model from the Foundry portal
Picks GPT-OSS 20B in the Foundry portal, targets Azure Local and an Arc-enabled Kubernetes cluster with the extension, sets replicas and runtime, and tracks the deployment.
- Skip to 0:21:12
Agentic retrieval over NFS and SharePoint
Ask, retrieve, answer over indexed NFS file shares and SharePoint Server, with the activity log showing the source of each answer.
- Skip to 0:24:13
Agent service in public preview
From the CLI, connects the agents extension to the local model endpoint, defines a bank risk investigator and teller assistant with MCP tools, and tests them in a chat UI, including a EUR 750,000 transfer investigation.