Home Assistant: Arbitrary Weather with Assist
Home Assistant: Arbitrary Weather with Assist
Over the last year, I have been working to setup Home Assistant Assist satellites to replace my current proprietary smart speakers. As good as assist has gotten since the "Year of the Voice", there are still missing features when comparing Assist to Google, Amazon, or Apple's offerings. One of the hang-ups has been asking Assist for the weather in a location other than my home.
When I set out to solve this problem, there were a few options that I considered or tried:
One option that can work, though doesn't move you away from giving your data away, is using LLMs to search the web to provide a weather report. I tested this option by setting up Google's Gemini as conversation agent following the instructions from the Home Assistant documentation here. For this to work, you do need to additionally follow the instructions in the "Workaround for Google Search tool" section. Then edit the description of the script to prompt the LLM to use it for weather searches. An example is :
Another way to accomplish this method, and keep your data, is to self host an LLM. I have tried a few permutations of this with varying levels of success. The largest hurdle is the state of tools to link them to Home Assistant. I have tested the 4B, 8B, and 8B-VL versions of Qwen3, and the 8B models work relatively well when properly integrated into home assistant. Properly integrating the models is where I struggled. The Home LLM custom integration is easy to setup, but I could not get it to control Home Assistant when running models with both llama.cpp and LM Studio. When looking at the debug, the system prompt is always provided to the LLM, but the user's input doesn't get passed along. The Extended OpenAI Converstation custom integration was also easy to set up, but doesn't allow turning off the assist functionality to use external tools like web search MCP servers.
Both of these options can work, though there are a few issues. The largest issue is that the LLM can and will misunderstand and hallucinate the weather in another location based on weather entities that are available in your Home Assistant instance. This drawback, along with the unpredictable nature of LLMs, led me to the second option. As for self hosted models, limitations lead to an implementation that is not as capable as proprietary LLM APIs, though this will hopefully change soon with the rate of work in the AI sector.
While LLMs offer a large degree of flexibility, this option is better defined. In the Assist pipeline, the automation will receive a location in plain text, and then return the weather of that location. To accomplish this, I used the RESTful Command integration that is native in Home Assistant. Looking at weather forecast APIs such as MET Norway's Locationforecast, I first needed to geocode the location from plain text to the latitude and longitude. I used the Nomanatim API offered by Open Street Map. The search endpoint of the API is quite flexible and allows passing in text straight from the pipeline. With the two rest commands below added to my configuration file, it was time to work on the automation.
The automation needs to use the sentence trigger so that it is accessible with the Assist pipeline. In these sentence triggers, the location of interest needs to be captured for later use. The two sentences that I use are:
Both triggers capture the location of interest in trigger.slots.place for later use. This variable can then be passed to the rest command get_lat_long_of_place. Saving the response from Nomanatim, the json can be parsed for the latitude and longitude. The latitude and longitude is then passed to the Locationforecast API for the forecast. Finally, the json from Locationforecast can be parsed with a template to provide a conversation response. Finally, since MET Norway provides all data in metric units, the temperature values need to be converted. The full automation is available below:
While the second option using a custom sentence works well enough that I haven't tried anything else, I did have some other ideas. One option is to use the Home Assistant Core Integration: Set location action. I haven't spent any time trying to get this method to work, though from a brief look, it seems like there are the same geocoding hurdles. Additionally, this has the side effect of moving the virtual location of your home when you as for the weather.