How to deploy an LLM in the EU

"Hosted in the EU" is a location. It does not by itself say who operates the service, who can read the data in support cases, or which laws the operator is subject to. Before you compare providers, write down which of these you need: data stored at rest in the EU, requests processed in the EU, administrative and support access from the EU, logs and backups in the EU, and an operator with no legal exposure outside the EU.

Most teams can get the first two from a hosted provider and have to negotiate or choose carefully for the rest. Be clear which requirements are firm (a regulator or customer contract says so) and which are preferences. Firm requirements decide the route; preferences decide between routes that pass.

Also decide what data may go into prompts at all. A residency setting does not make a prompt that contains special-category data safe to send. Your GDPR for AI and DPIA work decide that, not the hosting choice.

There are three families of route. Hosted APIs with an EU scope are quickest and the right start for most teams. Self-hosting an open-weight model on EU infrastructure gives you the most control. On-premises gives you the same plus physical custody. The table below summarises what the providers documented on the dates we read them. Treat it as a starting point and re-read the page before you decide, because these options change.

Notice the pattern in the wording. Storage and processing are separate promises, "zone" and "geography" mean different things, and there are global options sitting next to the regional ones. The cheapest or newest option is often the global one. Choosing the EU-scoped option is an explicit decision.

On Azure, the deployment type decides where inference is processed. The documentation lists DataZoneStandard and DataZoneProvisionedManaged for processing within a data zone, and GlobalStandard for any Azure region. Create your deployment with a Data Zone type, in a region inside the EU zone, and check the model is available for that type, since not all models support all types.

Then stop someone creating the global type by mistake. The documentation gives an Azure Policy definition that denies a named deployment type. Replace the SKU name with GlobalStandard to deny the global option. Test it in a non-production subscription first.

On Bedrock, call a geographic EU inference profile rather than a global one, and make sure your organisation's policies allow all destination Regions in the profile. On Vertex, call a regional or EU multi-region endpoint rather than the global one. With Mistral, use the EU endpoint host named in its help centre, api.eu.mistral.ai, rather than relying on a default. Provider-specific console steps change often, so follow the provider's current guide for the click path.

The model endpoint is rarely the only place data lands. List the vector store that holds your documents, the application database, the logging and monitoring service, the queue, the error tracker, the evaluation tool, backups, and any support tool that agents use. For each, write down where it runs, who operates it, and how long it keeps data.

Logs deserve special attention. Request and response logging is useful and it creates a second copy of the data. Decide whether you log content at all, where the logs live, and how long you keep them. On Bedrock, for example, the documentation says CloudTrail logs cross-Region inference requests in your source Region, so your logging sits in the Region you called from.

Do the same for people. Who can administer the cloud account, and from where? If your customers need EU-only administrative access, a provider that supports it contractually or a self-hosted route is the answer, and a residency toggle is not.

Ask each provider for its data processing agreement and its list of sub-processors, and read both. Mistral's help centre, for instance, says some features can transfer data temporarily outside the EU to the locations on its sub-processor list, and that contracts with service providers processing personal data outside the EU include safeguards under Article 46 of the GDPR. That is the type of statement you need to find and file for every provider.

Where personal data goes outside the EU or EEA, you need a lawful transfer mechanism. Commission Implementing Decision (EU) 2021/914 of 4 June 2021 contains the standard contractual clauses for transfers to third countries, and they have applied to new transfers since 27 September 2021. Whether SCCs, an adequacy decision or another tool applies, and whether extra measures are needed, is for your counsel to decide for your facts.

Do the same exercise for your own stack. If you use Swfte, the sub-processor list is published at /subprocessors and the security and hosting statements are on /trust. The trust page states that customer data is stored in AWS eu-west-1 (Ireland), and that other locations are agreed per dedicated deployment. The platform includes many model providers, some outside the EU, so EU-only routing depends on which providers you allow.

Do not stop at configuration. Send a set of test requests that contain only synthetic data and then look for the provider's own record of where they ran. On Bedrock, the documentation says to look at the additionalEventData.inferenceRegion field in CloudTrail to see where a request was processed. Check that every value is a Region in the EU geography.

For other providers, find the equivalent evidence: a region field in the response headers or metadata if there is one, an audit log, or the provider's own statement for the deployment type you chose. If a provider gives you no evidence, record that as a finding. It means your assurance rests on their document alone.

Check the network side too. From your application servers, record which hostnames the application calls for model requests. A configuration that sends a fallback request to a global hostname when the regional one fails is exactly the type of thing that goes unnoticed. Block the global hostnames at your egress proxy if you can.

Put the results in a short record: the date, the configuration, the evidence, and the open risks. That record is what you will show the next customer who asks.

Self-hosting removes the questions about a provider's routing and sub-processors for the model itself. You choose an EU region or your own site, you run an open-weight model, and prompts never go to a third-party model API. You take on the work of operating GPUs, patching, capacity and monitoring, and you still depend on your cloud or colocation provider for the hardware.

Check the licence of the model for EU use before you commit. Some open-weight licences restrict EU-domiciled users or companies for particular models. How to evaluate an open-source LLM shows how to read the licence and test the model on your tasks, and how to self-host an LLM covers the server.

A common middle path is to self-host for the most sensitive workloads and use an EU-scoped hosted option for the rest, behind one gateway so applications do not care which is which. How to set up an LLM gateway explains that pattern.