Google Home Windows Bridge: An approach to Hardware


Controlling .NET hardware on a RPi using Google Home voice commands.

So I have a Google Nest (Mini) and can tell it do “soft” things like get the local temperature, set temperature mode to Celsius etc, tell me the time, make cat sounds etc using, “Hey Google … “ voice commands. But how do I get it to similarly issue commands to control hardware? Ok so I can purchase of off-the-shelf products that “Works with Google”. Indeed, I setup a “Mirabella Genio Wi-Fi Power Plug with Two USB ports”. I was able to turn it off and on with “Hey Google,” commands. I could try other “hard” things such as lights and other smart home devices.

But what about controlling some embedded hardware using “Hey Google,” with devices such as a RPi. How can you turn a relay off/on to control a secondary device, or control a motor; turn it on/off, forwards/reverse, with voice commands to the Nest when the motor is controlled by a RPi. I have a H-Bridge .Net Core app for doing so with menu interaction. How can I control the app with voice? Also, the app can read temperature sensors connected to the RPi. How can I get the Nest to activate such readings and then enunciate the result?

There are a couple of issues here. How can I remote control a computer whether desktop or embedded like a RPi, using “Hey Google,”. Also how can a .NET Core app send text back to the Nest and have it spoken there? These are the two main issues that aren’t covered by Google itself but are answered by third parties. Yes, these can be done, and I have done it!

Another Wishlist is to able to be interact with an Azure IoT Hub using “Hey Google,”! That is to get enunciated Hub telemetry data upon request, or periodically, to turn the telemetry on/off, to set trigger levels and to sound alarms triggered by the hub. I haven’t done these yet but given that the capabilities in the previous paragraph have been achieved, it’s only a matter of time … To be covered in a later blog post. Update: Actually I have now done some of that. Blog coming later.

Some Google-RPi projects

Here are some projects found from a quick internet search. I guess there are lots of others

Requirements to move forward

  1. An ability to action functionality on a remote device using “Hey Google,” commands on the Nest, on a desktop or RPi.

    1. Also the ability to initiate such functionality on a remote device from a second remote device, in particular from a RPi to a Windows desktop.

  2. An app/s running on a RPi to action hardware:

    1. Relays

    2. Motor Control

    3. Read Sensors

  3. An ability to send text to the Nest so that it can be spoken.

Remote Access

There are many software mechanisms for remote for running apps remotely, such as Linux RPC, Java RMI, Microsoft .Net Remoting, rsh,ssh etc. What is needed here is a robust reliable universal mechanism that can respond to the Google Nest and to activate voice on it. A search comes up with a few such as IFTT, Push2Run and TRIGGERcmd. I have elected to use the latter as it is available for both Windows and the RPi and can be installed without any other components being used. With it a service runs on the target listening for calls for commands to run. You can add commands on the target, written as Json, for example

{
“trigger”: “Calculator”,
“command”: “calc”,
“ground”: “foreground”,
“voice”: “calculator”,
“allowParams”: “false”
}

With this, you would say “Hey Google, start calculator”. The voice property is what is used which when commanded, the Calculator app run on the default computer in the Home. You can add “on mypc” for example, to run it on a specific device.

A RPi example would be: “Hey Google, start motor on Pi”.

The App

The app used, DNETCoreGPIO, on the RPi is one I created nearly 2 years ago to simplify the dotnet/iot sample code using Nuget packages rather than the source code build for the hardware drivers. It was a .NET Core app and runs satisfactorily on a RPi whether the OS is Raspian or IoT Core. This can read a switch, toggle a LED, read temperature a humidity using a DHT22 as well control a DC motor via a H-Bridge. For all these, the app runs in a loop until the app is exited. The motor uses a textual menu to control its functionality. That version of the repository is located here.

The app has since been updated, see the latest version here, so that when it runs in one of the modes, it only does one pass. Thus, if reading temperature, after one successful reading, it writes that to a temporary file and exits. With the motor it runs once to either enable, disable, set to forward or set to reverse and then exits. Thus, with the Nest, the commands are such as “Hey Google, start Motor” which causes the app to start and enable the motor then exit.

Although they all run the same app on the RPi, each google command is a separate TRIGGERcmd command. The Json script calls a specific bash shell script calling the app with the specific parameter for the required action. Where a result is to be returned, that is processed in the script after the app runs. For example, with the temperature sensor reading, that script reads the string from the temporary file and forwards it for enunciation on the Google Nest. (it’s a little more complex than that but more later.)

Sending back speech

An app was found that can speak text locally or on a device that is part of a Google Home. That app, cast takes specific text in its command line or reads it from a file. This app runs only on Windows but works simply and well there, but not on the Pi!

The initial approach therefore with the Temperature reading was do action Text to Speech on the RPi. That works well outputting the result through the audio jack on the RPi. The command for this is

“Hey Google, start local on Pi.” Pi the name of the RPi and local is the Voice parameter of the TRIGGERcmd Json command.

The second approach was to use curl to forward the text to a Windows PC that is part of the Home, has TRIGGERcmd installed and has cast installed. TRIGGERcmd command there takes text as a parameter there and calls cast to send the text to the Google Nest to be spoken. Curl then, is used to remotely call that command from the RPi!

One thing not so far investigated is whether cast will run on a Window IoT_Core device which would cut out the middleman?.

 TopicSubtopic
  Next: > Nuget Packages 101
   
 This Category Links 
Category:IoT Index:IoT
  Next: > Google Home Windows Bridge
<  Prev:   Google Home Windows Bridge