Recently, I realized that I have a gaming laptop doing little more than gathering dust. It has an NVIDIA GeForce RTX 4060 with 8 GB of VRAM—sure, it’s not a monster, but it still seems a shame not to get any use out of it other than playing games, for which I sadly don’t have much time anymore.
I thought I could use it to run some small LLMs and make them available over my local network. Why over the network? Well, most of my after-hours work on various projects takes place on my MacBook Air, so it would be nice to have the option of using some open-source models there.
To be completely honest, though, I was mostly curious about how this could be done - and whether it would work at all. :-) Besides, I think this setup could be useful when planning integrations with paid models and tools such as Codex or Claude Code. I could experiment locally first, refining the code and prompts without incurring additional costs or using any tokens.
Since I had previously used Ollama to experiment with local models, I thought it would be a great candidate for this small project. After some quick research online, it became clear that I was good to go.
Plan
Install Ollama and several models on the Windows gaming laptop.
Configure the server so that I can interact with the models over the local network.
Use Postman or cURL on the MacBook - or experiment with the AI integration available in .NET.
Execution
You can download Ollama from here. Once it is installed, launch the application and open Settings, where you will need to enable network access.
In addition, we need to configure the firewall to accept connections from other devices. This can be done manually through Windows Defender Firewall or by running the following PowerShell command:
New-NetFirewallRule `
-DisplayName "Ollama API - Private LAN" `
-Direction Inbound `
-Action Allow `
-Protocol TCP `
-LocalPort 11434 `
-Profile Private `
-RemoteAddress LocalSubnet
Next, we have to setup the environment variable for Ollama service
[Environment]::SetEnvironmentVariable(
"OLLAMA_HOST",
"0.0.0.0:11434",
"User"
)
After creating the environment variable, restart the Ollama application. That’s all we need to do on the Windows side.
We can now verify that the Ollama server is accessible from another device by calling one of its API endpoints:
curl http://192.168.100.202:11434/api/tags | python3 -m json.tool
If everything is working correctly, we should receive a JSON response containing a list of the available models and their capabilities. In my case, two models are available.
{
"models": [
{
"name": "gemma4:12b",
"model": "gemma4:12b",
"modified_at": "2026-08-05T16:43:15.3260319+02:00",
"size": 7556508396,
"digest": "4eb23ef187e2c5462566d6a1d3bbbc2f1346d0b4327cbb66d58fffbcc9b2b05c",
"details": {
"parent_model": "",
"format": "gguf",
"family": "gemma4",
"families": [
"gemma4"
],
"parameter_size": "11.9B",
"quantization_level": "Q4_K_M",
"context_length": 262144,
"embedding_length": 3840
},
"capabilities": [
"completion",
"tools",
"thinking",
"vision"
]
},
{
"name": "qwen3.6:latest",
"model": "qwen3.6:latest",
"modified_at": "2026-08-05T15:56:43.4560821+02:00",
"size": 23938333577,
"digest": "07d35212591fc27746f0a317c975a6d68754fb38e9053d82e25f06057af28522",
"details": {
"parent_model": "",
"format": "gguf",
"family": "qwen35moe",
"families": [
"qwen35moe"
],
"parameter_size": "36.0B",
"quantization_level": "Q4_K_M",
"context_length": 262144,
"embedding_length": 2048
},
"capabilities": [
"vision",
"completion",
"tools",
"thinking"
]
}
]
}
Now, using the Ollama server endpoints we can start talking with the model. Complete list of endpoints can be found here.
For example, we can use chat endpoint to start the conversation:
http://192.168.100.202:11434/api/chat
,with following payload:
{
"model": "gemma4:12b",
"messages": [
{
"role": "system",
"content": "You are an experienced Python developer."
},
{
"role": "user",
"content": "Write a function that reverses a string. Give me only the code - no additional info. Just pure code of that function."
}
],
"stream": false
}
Here is the response from the gemma4 model:
{
"model": "gemma4:12b",
"created_at": "2026-08-05T16:23:03.5613682Z",
"message": {
"role": "assistant",
"content": "```python\ndef reverse_string(s: str) -> str:\n return s[::-1]\n```",
"thinking": "* Task: Write a Python function to reverse a string.\n * Constraint 1: Experienced Python developer persona (implies clean, idiomatic code).\n * Constraint 2: Only code - no additional info.\n * Constraint 3: Just pure code of that function.\n\n ```python\n def reverse_string(s: str) -> str:\n return s[::-1]\n ```\n Or simply:\n ```python\n def reverse_string(string):\n return string[::-1]\n ```\n\n The user wants *only* the code. No explanation, no markdown surrounding it? Actually, usually \"just pure code\" in this context means no chatty text, but formatting as a code block is standard unless specified otherwise. I will provide just the function.\n\n *Refining choice:*\n `def reverse_string(s: str) -> str:`\n ` return s[::-1]`\n\n Wait, sometimes users want it exactly as requested. \"Just pure code\".\n\n Let's go with a standard Pythonic implementation."
},
"done": true,
"done_reason": "stop",
"total_duration": 24660633100,
"load_duration": 321959300,
"prompt_eval_count": 48,
"prompt_eval_duration": 803372000,
"eval_count": 264,
"eval_duration": 23471301000
}
We can also use generate endpoint:
http://192.168.100.202:11434/api/generate
{
"model": "gemma4:12b",
"prompt": "Write a Python function that reverses a string. Propose different approaches (max 3) for solution. I'm expecting code only, no additional explenation.",
"stream": false
}
, and receive the following response
{
"model": "gemma4:12b",
"created_at": "2026-08-05T16:30:59.3725895Z",
"response": "```python\n# Approach 1: Slicing\ndef reverse_string_slicing(text: str) -> str:\n return text[::-1]\n\n# Approach 2: Using reversed() and join()\ndef reverse_string_builtin(text: str) -> str:\n return \"\".join(reversed(text))\n\n# Approach 3: Iterative approach\ndef reverse_string_loop(text: str) -> str:\n reversed_str = \"\"\n for char in text:\n reversed_str = char + reversed_str\n return reversed_str\n```",
"done": true,
"done_reason": "stop",
"context": [
2,
105,
9731,
107,
98,
107,
106,
...// I have removed aroun 400 lines with context items
],
"total_duration": 39022107700,
"load_duration": 367006400,
"prompt_eval_count": 49,
"prompt_eval_duration": 406029000,
"eval_count": 418,
"eval_duration": 38021484000
}
To be fair, it’s slow - but performance will depend on your hardware and the size of the model. On the machine hosting the Ollama service, you can check how the model is distributed between the CPU and GPU by running the following command:
ollama ps
Additionally, if you have NVIDIA GPU you can verify whether the Ollama server is actually using the GPU by calling this command
nvidia-smi
We can also integrate the model—and, by extension, the Ollama server—with a .NET application. The integration uses the Microsoft.Extensions.AI abstractions, which should make it relatively easy to switch to another provider or client later if needed.
Here is a small console application that demonstrates how it works. First, install the following packages:
Microsoft.Extensions.AI
OllamaSharp
using Microsoft.Extensions.AI;
using OllamaSharp;
var endpoint = new Uri("http://192.168.100.202:11434");
var model = "gemma4:12b";
IChatClient client = new OllamaApiClient(endpoint, model);
ChatResponse response = await client.GetResponseAsync(
[
new ChatMessage(
ChatRole.System,
"You are an experienced Python developer."),
new ChatMessage(
ChatRole.User,
"Write a function that reverses a string. Give me only the code - no additional info. Just pure code of that function.")
]);
Console.WriteLine(response.Text);
And here is the response
Summary
Clearly, this approach will not replace high-performance paid solutions - but that was never my goal. Still, as ridiculous as it may sound, this experiment was a lot of fun, and seeing one device communicate with a model running on another device in my apartment genuinely made me happy.
As mentioned earlier, this setup could be useful for testing AI integrations locally before switching to a paid provider once everything is ready. Now I hope to find some nice use cases for this stack - if any I will post them here.
