Project Caretaker. Part 11. Home assistant

The project is positioned as remote monitoring, but how can you do that if the device is only accessible from the internal network? The straightforward option is to expose access to the device externally (router and white IP, tunnel to a server, third-party service), but you'll have to solve security issues on your own. Smart home devices, especially DIY ones, rarely bother with this, which is a shame.

We'll take the approach of integrating the device into an existing ecosystem - I have Home Assistant with public access (through my VPS) - we'll offload security concerns to it, we just need to implement the integration.

It would seem that we already have a ready-made HTML interface for the robot with requests, there are plenty of advanced AIs that can do anything - implementing the integration shouldn't be problematic even without deep knowledge of Python and Home Assistant?

Proxy

Initially, I went down the path of proxying the robot's web interface into the Home Assistant interface. As a result of collaborative development, an integration was created with the following features:

• Adding and installing through HACS (custom repository);
• To add the integration, you need to specify the robot's IP address on the internal network;
• A Lovelace card (Home Assistant UI) is added, which shows the robot's status (online or not) and a button to open the interface;
• The standard web interface of the robot opens in an iframe, and all requests are wrapped in a custom proxy.

An interesting implementation detail is the proxy itself, which adds its JavaScript code when opening the original page in an iframe to redefine all URLs to which requests are made. This is somewhat similar to the embedding of advertisements in unprotected HTTP traffic by some mobile operators.

All would be fine, but I couldn't manage to either pass the Home Assistant authentication mechanism through the iframe, use a long-lived token (the documentation is silent on how to verify it, all found instructions were unavailable in the current version), or use the short-lived authSig in the proxy context (the signed token is unique for each URL, this option is not very suitable for an iframe that makes many requests to itself).

IoT certainly can't boast perfect security, but in my solution, I couldn't allow myself not to close a security hole. Any resourceful user would not only be able to watch the camera stream but also drive around the house.

Second Approach

From the already working solution, I had to discard almost everything that had been written, except for status checking, and implement my own interface in Home Assistant that interacts with the server without an iframe. Such an integration can be considered native, using the standard authentication mechanism, since without an iframe, the browser passes authorization headers in requests (if you send requests not with a regular fetch, but through the Hass API).

But authorization through headers doesn't work everywhere - you can't use such a link in img.src for streaming. For this case, the vendor has a mechanism for signed links. That is, you get a signed short-lived link (the parameter authSig is added) through websockets in the card and put it in the container for video streaming - the signature is checked once when receiving the request, and then the stream can live forever; after following the link, the signature is invalidated, so the link becomes one-time. Standard components for video streaming from cameras work on the same principle.

The current version of the integration doesn't have all features - there's no BT functionality, no switching between joystick and slider modes, and no separate FPS display toggle - it's always displayed.

And to add to the non-obvious solutions - to add a logo for your custom integration in Hass, you need to submit a PR to a specially created separate brands repository. That's why my integration doesn't have an icon yet, even though it's already designed - I'll add it later.

Installation

To add the integration to Home Assistant, you need HACS installed.

Through it, you add a custom repository - https://github.com/positron48/hass-caretaker.

Then in the list of integrations, you add the esp32-robot integration, which requires entering the robot's IP address and the status update period for the Hass sensor (and for display on the card, accordingly):

Next, you can add the esp32-robot card to the dashboard and have the ability to track status, as well as a robot control interface and video streaming.