Connecting Synology AI to a local LLM with Ollama or LM Studio
Running a language model at home, exposing it as an OpenAI-compatible API and wiring it into Synology AI Console to power MailPlus and Office without any online service.
This guide explains how to connect Synology AI to a language model running on your own premises, with Ollama or LM Studio, so you can use the AI features of DSM without depending on an online service.
Introduction
The aim of this text is to show how to attach a local AI to Synology AI, nothing more. I add a few personal conclusions and some hardware and software recommendations, but those are side notes meant to make the picture complete.
Synology does not develop its own AI model. What the company does provide is an interface, Synology AI Console, which lets you attach third-party models to it. Other manufacturers follow a comparable approach, but Synology implemented it fairly early.
Most cloud-hosted models involve a subscription or pay-per-use billing of their API, as well as a transfer of your content to a third party’s infrastructure. For a company server or a family NAS, those two points are often enough to rule the solution out.
Coincidence or not, an update to Synology AI Console made it possible to attach any model compatible with the OpenAI API, including a local one. That is exactly what I show here. The extra benefit is privacy: the data does not leave for an external service, it stays inside your own ecosystem.
How the AI features behave inside the applications is covered separately, in the guides devoted to Synology MailPlus and to Synology Office.
System requirements
Running a local AI calls for suitable software and hardware. On the software side, two tools fit: Ollama and LM Studio. Ollama runs on Windows, Linux and macOS, in graphical as well as server mode; it is probably the better choice with an NVIDIA or AMD card. LM Studio only exists as a graphical application and suits macOS on Apple M1 and later processors particularly well.
- Ollama is a server engine meant to run language models. It runs in the background, like a service, and exposes a simple API. Its main job is to load and run models efficiently, over long periods. It is the sensible choice if you want to integrate a model into an application or publish it as a network service.
- LM Studio is a graphical desktop application for Windows and macOS that lets you search, download and test models in an interface close to that of a chat assistant. Its distinctive feature is the “Local Server” function: in one click, any loaded model becomes an OpenAI-compatible API server. It is the ideal tool to start quickly, experiment and manage your models visually, with no command line.
💻 Minimum configuration (to make it work)
| Component | Requirements | Consequence |
|---|---|---|
| Processor (CPU) | 4 cores or more (Intel i5 / AMD Ryzen 5 / Apple M1 and later). A modern architecture counts for more than the clock speed. | Works without a GPU, but slowly (1 to 5 tokens per second). |
| Memory (RAM) | 16 GB — minimum. 24 GB and more — recommended for stable operation. | An 8B model + the system + the applications ≈ 12 to 14 GB. With 16 GB, it is tight. |
| Storage (SSD) | An NVMe SSD of at least 256 GB. A 7B-8B model takes 5 to 10 GB. | Fast loading of the model into memory. A mechanical hard drive is unusable here. |
| System | Linux (Ubuntu), Windows 10/11, macOS Sonoma and later. | Linux brings 10 to 15 % extra performance. |
🎮 Recommended configuration (to work comfortably)
| Component | Requirements | What it brings |
|---|---|---|
| Graphics card (GPU) | Essential for speed. • NVIDIA: GTX 1660 6 GB (minimum), RTX 3060 12 GB (optimal) • AMD: RX 6700 XT 12 GB and above • Apple: M1 Pro / M2 / M3 with 16 GB and above |
A 10 to 50 times speed-up. The model works in VRAM and frees the main memory. |
| GPU memory (VRAM) | 8 GB minimum for 7B models. 12 GB and more recommended for 8B-14B with some headroom. | The larger the VRAM, the wider the context and the faster the processing. |
| Memory | 32 GB DDR4/DDR5 | Headroom for the system, the browser and working on documents. |
| Processor | 8 cores or more (Intel i7 / AMD Ryzen 7) | Better request handling when part of the model stays in main memory. |
From experience, a GPU built into the processor — or the processor alone — is a poor fit for AI tasks. You can of course run a model that way, but AMD support in ROCm remains weak. NVIDIA cards give the best results; the detailed list is in the manufacturer documentation.
Generally speaking, I consider the Mac mini with an ARM M1 Pro processor or newer to be the ideal solution for this purpose. You simply have to aim for 32 to 64 GB of memory, or even 128 GB: the more, the better. To my mind that is the best result-to-price ratio for a local AI — who would have thought Apple would be the cheaper option.
Choosing the model
Choose according to the memory available. The most balanced model in my view is gpt-oss:20b, published by OpenAI.
- With 16 GB of RAM: models of 7B to 8B parameters fit best, for example Qwen2.5-7B-Instruct or DeepSeek-R1:8b.
- With 32 GB of RAM: you can run sturdier models, from 14B to 20B parameters, for example DeepSeek-R1:14b for analysis or gpt-oss:20b for the best tool compatibility.
- With 64 GB of RAM: the whole range up to 30B-32B parameters becomes reachable, including Qwen3:30b and DeepSeek-R1:32b, which gives the best answer quality and the widest context.
Always use the Instruct versions of the models and the GGUF format, which guarantees compatibility.
Installing Ollama
Let us assume you have a PC running Linux Ubuntu and want to try it there. On my side, I ran the test on an Aoostar WTR Pro with the Proxmox hypervisor: I created a virtual machine there and gave it 8 cores and 16 GB of memory.
The rest is simple: install Ollama with a web interface, which is more convenient to administer.
Commands:
sudo apt update
sudo apt upgrade
sudo apt install docker.io docker-compose -y
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
sudo rebootNow let us prepare the folder for Ollama:
cd /opt/
sudo mkdir ollama
sudo chown $USER ollama
cd ollama/Create a file with nano docker-compose.yml holding the following content:
version: '3.8'
services:
ollama-webui:
image: ghcr.io/open-webui/open-webui:ollama
container_name: ollama-webui
restart: unless-stopped
ports:
- "11434:11434" # Ollama API port
- "8080:8080" # Open WebUI web interface port
environment:
# Key setting: allows connections to the Ollama API from other addresses
- OLLAMA_HOST=0.0.0.0
- OLLAMA_ORIGINS=*
volumes:
- ./ollama_data:/root/.ollama # Storage of the downloaded models
- ./openwebui_data:/app/backend/data # Open WebUI data (conversations, settings)
volumes:
ollama_data:
openwebui_data:Then start the container:
docker-compose up -dOnce the container is deployed, open the Ollama web interface at http://IP:8080. After creating a user and signing in, download the gpt-oss:20b model or any other model of your choice.
Configuring LM Studio
On a Mac, LM Studio is the better choice, but it needs configuring. You first have to enable developer mode, then allow the server to be reached from the local network, as in the screenshots below.
Configuring Synology AI Console
To attach the local AI to the NAS, open Synology AI Console and add a new integration. Give it a name, choose the “OpenAI-compatible API” interface and enter a URL of the form http://IP:11434/v1. Then click “Get the list of generative models”: the system returns every loaded model. Select the one you want. For the context window and the output tokens, use at least 8192 and 4096. The higher, the better, but it all depends on your hardware.
Checking the result
If everything is right, the AI features become usable in Synology Office and Synology MailPlus. They run locally, stay entirely private, and their speed now depends on nothing but your hardware.
That is all. You now know how to attach Synology AI to a local language model run by Ollama or LM Studio.
Discussion
Comments
No comment yet. Feel free to open the discussion with a question or your own experience.
Signing in keeps the discussion free of spam and avoids publishing your e-mail address.
Sign in to comment →