ha_tinkerer OP
2 days ago
I'm building a setup where an Arduino Nano 33 IoT acts as a Matter bridge for custom sensors — CO2, PM2.5, and a soil moisture array in my greenhouse — and I want Home Assistant to expose a local Ollama API so I can query the state of the house in natural language without anything going to the cloud. The Nano 33 IoT isn't Matter-native so I'm bridging via a Raspberry Pi 5 running the Matter SDK as a controller, exposing the sensors as Matter virtual devices, and HA reads those over the Thread/Wi-Fi bridge. The local LLM piece uses Ollama running Llama 3.2 3B — light enough for a Pi 5 — connected to HA via the custom conversation agent integration. Anyone done something similar and found a way to keep LLM response latency under 2 seconds consistently?
dr_homelab
2 days ago
Good setup, and yes sub-2-second is achievable. The biggest lever is context window size — keep your system prompt compact and don't inject the full HA state JSON into every call. I wrote a small Python script that extracts only entities matching keywords from the user's query before passing to Ollama, cutting prompt token count by roughly 70%. Also quantize to Q4_K_M rather than Q8 — you lose very little quality for home automation intents but inference speed nearly doubles on CPU. Here's the conversation agent hook that calls local Ollama:
import requests, json
OLLAMA_URL = "http://localhost:11434/api/generate"
SYSTEM_PROMPT = """You are a home assistant AI.
Answer using only data from the provided HA state JSON.
Be concise. Reply in one or two sentences max."""
def query_ollama(user_text: str, ha_state: dict) -> str:
# Filter to only relevant entities
relevant = {k: v for k, v in ha_state.items()
if any(kw in user_text.lower()
for kw in k.split("."))}
payload = {
"model": "llama3.2:3b-instruct-q4_K_M",
"prompt": f"State: {json.dumps(relevant)}nQ: {user_text}",
"system": SYSTEM_PROMPT,
"stream": False,
"options": {"num_predict": 80, "temperature": 0.2}
}
r = requests.post(OLLAMA_URL, json=payload, timeout=4)
return r.json()["response"]
The num_predict: 80 cap is the most important line — home automation answers don't need to be long, and capping output tokens cuts tail latency by 40–60% on average. Temperature at 0.2 keeps answers factual and deterministic.
smarthome_ss
1 day ago
One thing to understand about Matter bridging from a non-Matter device: the Arduino Nano 33 IoT talks standard Wi-Fi so your bridge Pi polls the Nano's sensors via MQTT or HTTP, then the Matter bridge daemon on the Pi creates synthetic Matter endpoints for each sensor. The actual ConnectedHomeIP SDK has a bridge example that's the cleanest starting point. Matter 1.3 — released late 2024 — added proper air quality cluster definitions with CO2 concentration and PM2.5 measurement characteristics built in, so your sensors will appear as proper air quality entities in Home Assistant rather than generic number sensors. That makes a huge UX difference in the HA dashboard and for automation triggers.
futura_labs
20 hours ago
Worth knowing for 2026: the Home Assistant voice pipeline has matured dramatically. Running HA OS 2025.x, you can wire Whisper ASR + local Piper TTS + your Ollama conversation agent entirely on-device with no cloud dependency anywhere. Whisper tiny.en on Pi 5 transcribes a short utterance in under 500ms, leaving a comfortable budget for the LLM inference step. Most people don't realize you can swap just the conversation agent and keep the official HA voice stack intact — you don't have to build the whole pipeline yourself. Check the HA docs for the conversation.process service and the homeassistant.conversation_agent config key. Total cost: Pi 5, a USB mic, and an afternoon of setup.
