Ollama

    Ollama: open language models on your own hardware.

    Meeting notes that must not leave the building are summarised by an open model on your own server. We measure on your tasks whether quality and hardware are enough.

    For organisations whose data should stay inside their own network or whose tasks have to run without an internet connection.

    • Ollama
    • gpt-oss
    • OpenClaw
    • Hermes Agent
    • Docker
    • Microsoft Foundry
    A rustico in the Verzasca valley looking out at the Ponte dei Salti, a man works with Ollama on his laptop, a cable runs to a small computer on the table, chestnuts and the robot agent beside him
    01

    Where local models fit

    The legal department of Muster AG summarises confidential meeting notes: the model runs on a server in its own data centre, Ollama provides it on the internal network through an interface, and no request goes outside. Local models fit where data should not leave your network, where a task has to run without an internet connection or where many similar texts come in. For planning, research and agents with tools, large models in Microsoft Foundry usually remain the stronger choice, and the two can be combined in one application.

    02

    What we build with Ollama

    Ollama provides open models through a REST interface, including OpenAI-compatible endpoints; an application written for the OpenAI API switches to a local model by changing its base address. We use it for these tasks:

    • Local text serviceSummarising, translating, classifying and extracting fields from documents, on a server in your network, connected to the application that processes these texts today.Result: confidential texts are processed without a request leaving your network.
    • Model for your personal agentOpenClaw or Hermes Agent pass private notes and first drafts to a local model, mail and planning to a cloud model.Result: the text that should stay private stays on your machine.
    • Development and testsYour team tries prompts, embeddings and workflows with a local model and test data before the application runs against a model in Microsoft Foundry.Result: workflows are checked before they reach Azure.
    • Secured operationAccess from the internal network only, sign-in through a proxy in front, cloud models and web search switched off, a list of approved models and a log of requests.Result: IT knows which model runs, who uses it and that nothing goes out.
    03

    Hardware, network, model licence

    A test set of typical tasks and sample data that shows what the model has to deliver. Hardware that fits the model: model size, graphics memory and the number of simultaneous requests determine what a server can carry, and more parallelism needs more memory. We check the licence of every open model for your use. We record every connection that leads out of the environment and approve it explicitly. And one person who owns updates, model changes and the server.

    04

    What the measurement decides

    We measure quality, response time and resource use on your test set, compared with a model in Microsoft Foundry. We rehearse failures, updates and access, and we check in the log that no outbound connection exists without approval. The decision covers maintenance and who owns operation as well as the hardware.

    What you receive from us

    The model comparison on your tasks.

    Before you choose a model, we test the candidates on your own tasks and record which one fits what. Shown here as a sample with invented results and neutrally named models.

    Test report

    Sample

    Model comparison on three tasks

    Customer
    Muster AG, Bern (fictional)
    Tasks
    Summarising contracts, questions on internal rules, code review
    Tested
    14 to 20 October 2026
    Test data
    Anonymised documents and a test repository
    • 4models compared
    • 3tasks
    • 120test cases
    • 2models recommended

    Summary

    We tested two cloud models and two open models on three Muster AG tasks, with 120 test cases from anonymised documents. For contracts and code we recommend a cloud model with governed data processing. For questions on internal rules, an open model on your own hardware answered just as accurately and stays inside your network.

    CasesTask and resultRecommendation
    40Summarising contracts: cloud model A found every notice period, open model C missed two in appendices.Cloud model A
    50Questions on internal rules: all four answered with a source, open model C as accurately as the cloud models.Open model C, in your network
    30Code review: cloud models A and B found the planted bugs, the open models about half of them.Cloud model A or B

    To settle before use

    • Legal reviews the data processing agreement for the cloud model.
    • The open model needs a server with a GPU; IT has the quote.

    Next steps

    With your decision we set up the chosen models and rerun the test cases after every model change. The real report names each model and version.

    Samples with invented details for a fictional Muster AG.

    Sound familiar? Tell us where you stand today.

    Contact

    What's new.

    As of: October 2026

    • As of October 2026

      By default, Ollama listens on its own machine only.

      The server binds to 127.0.0.1 on port 11434. Whoever opens it to the network with OLLAMA_HOST controls access in front of it, for example through a reverse proxy, and records who can reach the model.

      Source: Ollama Docs: FAQ
    • As of October 2026

      Cloud models and web search can be switched off.

      Alongside local models, Ollama offers cloud models that run at Ollama and require a sign-in. With OLLAMA_NO_CLOUD=1 or disable_ollama_cloud in server.json, the server switches both off. Run locally, Ollama states that it sees neither prompts nor data.

      Source: Ollama Docs: FAQ
    • As of October 2026

      OpenAI-compatible endpoints on your own server.

      At localhost:11434/v1, chat completions, completions, embeddings, models and the Responses API without stored state are available. Applications written for the OpenAI API switch to a local model by changing the base address.

      Source: Ollama Docs: OpenAI compatibility
    • As of October 2026

      MIT licence, for macOS, Windows, Linux and Docker.

      Ollama itself is MIT-licensed and runs on macOS, Windows and Linux and as an official Docker image. The models carry their makers' licences, which we check for each use.

      Source: GitHub: ollama/ollama

    What IT teams ask before the first model.

    Does our data leave the building?

    In local operation, prompts and answers stay on your server. Outbound connections come from cloud models, web search and downloading models. We switch cloud features off, load models through an approved route and record every outbound connection.

    What hardware do we need?

    Model and load decide. A small model runs on a current laptop, a large one needs a server with graphics cards and plenty of graphics memory. If a model does not fit into graphics memory, Ollama computes partly on the processor and answers come more slowly. We measure with your test set before hardware is bought.

    How many people can work at the same time?

    By default, each loaded model processes one request at a time, and further requests wait in a queue. OLLAMA_NUM_PARALLEL raises that; memory use grows with parallelism and context length. For one department we plan the load on a server, for the whole company Microsoft Foundry is usually the more sustainable choice.

    Are open models as good as GPT or Claude?

    For narrowly defined tasks such as summarising, classifying or extracting fields, often good enough. For long planning, research and agents with tools, the large cloud models are usually ahead. By default Ollama works with a context window of 4096 tokens, which we raise for long documents at the cost of memory. The comparison on your test set decides.

    Who runs the server?

    Your IT or our support and operations team. That covers updates to Ollama and the operating system, the list of approved models, backing up the configuration and monitoring load and access. Who takes this on is settled before the start.

    Agree on a useful next step

    Tell us about your current workflow, the systems involved and the result you need. In our first conversation, we identify what else we need to know to plan a useful next step.