Voice Kit (V1)

The Voice Kit requires a special version of the Raspbian operating system that includes some extra AIY software. So before you begin, you need to download the Voice Kit system image and flash it to your MicroSD card.

1. Download the latest .img.xz file from the releases page on GitHub.
2. Use an adapter to connect your MicroSD card to your computer.
3. Use a program such as Etcher to flash the .img.xz file onto your MicroSD card. (Etcher is free and works on Windows, Mac OS, and Linux.)

Flashing the system image onto the card can take a several minutes. So while that’s going, start assembling the kit. Once the kit is assembled, you'll put the card into it.

Troubleshooting Tips

1. A red LED on the Raspberry Pi near the power connector should light. If it doesn't, unplug the power, unplug the connector to the microphone, and power-up again. If it lights after powering-up without the microphone, then the microphone board may be defective.
2. The lamp in the button will not light up until you run a demo, so don't worry that it's off. (This is different from the AIY Essentials Guide, which describes an older software version.)
3. If you don't see anything on your monitor, make sure the HDMI and power cables are fully inserted into the Raspberry Pi.
4. If you see "Openbox Syntax Error", you'll need to rewrite the image to the SD card and try booting the device again.

Congrats! You’ve set up your very own intelligent speaker.

Now that you’ve got a taste for the Voice Kit can do, we’d love to see what you do with it. In the following Makers Guide section, you'll find documentation about the Python APIs, hardware, and tools that help you to build your own intelligent speaker projects.

Share your creations with the maker community at #aiyprojects

Heads up! The next part assumes a much higher level of technical experience. So if you're new to programming, don't be discouraged if this is where you stop for now.

To support various features in the Voice Kit, we've built a Python library that handles a lot of programming dirty work for you. It makes it easy to interact with services like Google Assistant and Cloud Speech-to-Text, and to use kit peripherals such as the button and LEDs.

These APIs are built into a Python package named aiy, which is pre-installed in the kit's system image. Just be sure that you've installed the latest system image.

To learn more about these APIs, refer to the API reference. In particular, the following APIs will be of interest for use with your Voice Kit:

• aiy.assistant: APIs that simplify interaction with the Google Assistant API.
• aiy.cloudspeech: APIs that simplify interaction with the Google Cloud Speech-to-Text service.
• aiy.voice.audio: APIs to record and play audio files.
• aiy.voice.tts: An API that performs text-to-speech (reads some text aloud).
• aiy.board: APIs to use the button attached to the kit, and the button's LED.

Examples

You might find it easier learn the aiy Python API if you start with an existing demo and modify it to do what you want.

These files are already installed on your kit at ~/AIY-projects-python/src/examples/. You can also browse the examples on GitHub, where you'll find the source code for all the examples and more.

For instance, to learn more about the aiy.voice.audio API, try running the voice_recorder.py example:

cd ~/AIY-projects-python/src/examples/voice

./voice_recorder.py

This waits for you to push the button, then begins recording what you say. When done speaking, press the button again and it will play the recording.

To see how it works, open this file on your Raspberry Pi or see the source code here. Then start tweaking the code.

If you're more interested in programming hardware such as buttons and servos, see the section below about the GPIO expansion pins, which includes some other example code.

# =========================================
# Makers! Implement your own actions here.
# =========================================

import aiy.audio
import aiy.cloudspeech
import aiy.voice


def main():
    '''Start voice recognition when motion is detected.'''
    my_motion_detector = MotionDetector()
    recognizer = aiy.cloudspeech.get_recognizer()
    aiy.audio.get_recorder().start()
    while True:
        my_motion_detector.WaitForMotion()
        text = recognizer.recognize()
        aiy.audio.say('You said ', text)


if __name__ == '__main__':
    main()

if __name__ == '__main__':
    main()

...
for event in assistant.start():
   process_event(event)

...
def on_button_press(_):
    assistant.start_converstation()


...
aiy.voicehat.get_button().on_press(on_button_press)
for event in assistant.start():
   process_event(event)

...
class MyAssistant(object):
    def __init__(self):
        self._task = threading.Thread(target=self._run_task)

...
def _run_task(self):
    credentials = aiy.assistant.auth_helpers.get_assistant_credentials()
    with Assistant(credentials) as assistant:
        # Save assistant as self._assistant, so later the button press handler can use
        # it.
        self._assistant = assistant
        for event in assistant.start():
            self._process_event(event)

...
def _process_event(self, event):
    ...
    if event.type == EventType.ON_START_FINISHED:
        # The Google Assistant is ready. Start the button trigger.
        aiy.voicehat.get_button().on_press(self._on_button_pressed)

Want to try another API? Follow the instructions below to try the Google Cloud Speech-to-Text service, which converts spoken commands into text you can use to trigger actions in your code. This API supports 80 languages, long audio clips, and the ability to add hint phrases that help improve the accuracy of speech recognition.

(using the local file system)
`cp /path/to/downloaded/credentials.json ~/cloud_speech.json`

(from another machine)
`scp /path/to/downloaded/credentials.json pi@raspberrypi.local:~/cloud_speech.json`

cd ~/AIY-projects-python/src/examples/voice

./cloudspeech_demo.py

def get_hints(language_code):
    if language_code.startswith('en_'):
        return ('turn on the light',
                'turn off the light',
                'blink the light',
                'goodbye',
                'repeat after me')
    return None

            if 'turn on the light' in text:
                board.led.state = Led.ON
            elif 'turn off the light' in text:
                board.led.state = Led.OFF
            elif 'blink the light' in text:
                board.led.state = Led.BLINK
            # Our new command:
            if 'repeat after me' in text:
                # Remove "repeat after me" from the text to be repeated
                to_repeat = text.replace('repeat after me', '', 1)
                aiy.voice.tts.say(to_repeat)
            elif 'goodbye' in text:
                break

import aiy.voice.tts

"""A demo of the Google CloudSpeech recognizer."""
import argparse
import locale
import logging

from aiy.board import Board, Led
from aiy.cloudspeech import CloudSpeechClient
import aiy.voice.tts


def get_hints(language_code):
    if language_code.startswith('en_'):
        return ('turn on the light',
                'turn off the light',
                'blink the light',
                'goodbye',
                'repeat after me')
    return None

def locale_language():
    language, _ = locale.getdefaultlocale()
    return language

def main():
    logging.basicConfig(level=logging.DEBUG)

    parser = argparse.ArgumentParser(description='Assistant service example.')
    parser.add_argument('--language', default=locale_language())
    args = parser.parse_args()

    logging.info('Initializing for language %s...', args.language)
    hints = get_hints(args.language)
    client = CloudSpeechClient()
    with Board() as board:
        while True:
            if hints:
                logging.info('Say something, e.g. %s.' % ', '.join(hints))
            else:
                logging.info('Say something.')
            text = client.recognize(language_code=args.language,
                                    hint_phrases=hints)
            if text is None:
                logging.info('You said nothing.')
                continue

            logging.info('You said: "%s"' % text)
            text = text.lower()
            if 'turn on the light' in text:
                board.led.state = Led.ON
            elif 'turn off the light' in text:
                board.led.state = Led.OFF
            elif 'blink the light' in text:
                board.led.state = Led.BLINK
            # Our new command:
            if 'repeat after me' in text:
                # Remove "repeat after me" from the text to be repeated
                to_repeat = text.replace('repeat after me', '', 1)
                aiy.voice.tts.say(to_repeat)
            elif 'goodbye' in text:
                break

if __name__ == '__main__':
    main()

If you plan to take your project beyond the cardboard box, you might be wondering which GPIO pins are available for your other hardware. So figure 1 shows which pins from the Raspberry Pi are used by the Voice HAT.

The pins highlighted in dark green are used by the HAT and not available to you, while the pins in light green are available with special GPIO pins on the Voice HAT (see the following section).

Figure 1. GPIO pins used by the Voice HAT

GPIOs, servos, and drivers

Several GPIO pins from the Raspberry Pi are accessible on the Voice HAT as shown in figure 2.

On the left side are sets of three pins that can control servos:

• Left: GPIO pin to control servo (with 220 ohm resistor)
• Middle: 5V power
• Right: Ground

Note: The drive limit for servos is 25 mA.

On the right side are sets of three pins that can function as high-current drivers (they are open drain drivers and can handle inductive loads with a 1 amp max current):

• Left: GPIO pin (direct GPIO; use this for non-driver applications, or to create a push-pull driver by adding a p-channel MOSFET here)
• Middle: 5V power (includes a 1 amp PTC fuse)
• Right: Ground, via a transistor gated by the GPIO pin (with 10k ohm pull-down resistor)

So when the middle and right pin are connected to a load, they create an open drain output controlled by the GPIO pin. For example, if you want to control a DC motor, connect your motor's negative terminal to the pin on the right (such as Driver0) and connect the positive terminal to the corresponding 5V pin in the middle (leave the left pin alone). Then you can turn on the motor by enabling the GPIO pin (enable GPIO04 to control Driver0).

Note: The drivers include a PTC fuse with a trip current of 1 amp and a hold current of 0.5 amp. That means it will trip for any current greater than 1 amp, and will not trip for any current less than 0.5 amp. If it trips, it needs about a minute to reset at room temperature, then the driver can be used again.

All GPIO pins originate from the Raspberry Pi's 40-pin header, as shown in figure 1, so your ability to control analog outputs with PWM is limited by the Raspberry Pi's capabilities.

Tip: If your project requires an analog-to-digital converter (ADC), notice that the row of I2C pins at the top-left side of the Voice HAT align nicely with boards such as the ADS1015 12-Bit ADC and the ADS1115 16-Bit ADC.

Figure 2. GPIO, servo, and driver pins on the Voice HAT

The rest of the pins at the top of the board are exactly as provided from the Raspberry Pi.

Stereo speakers

If you'd like to add a right channel speaker for stereo sound, we recommend you add the Adafruit I2S Amplifier (MAX98357A) board to the Voice HAT. Using the pins included with that board, it sits perfectly onto the set of pins at JP6 on the Voice HAT, as indicated in figure 3.

When connecting the Adafruit board, also connect the amp board's speaker pins to corresponding pins on the Voice HAT, which carry the signal to the footprint for the right channel terminals on the Voice HAT.

Then to enable the right channel, add a small piece of solder on the jumper at JP4.

Figure 3. Stereo terminals and amp expansion on the Voice HAT

Want to learn how to use your Voice Kit to control other IoT devices? You can start here with a Particle Photon (a Wi-Fi development kit for IoT projects) and Dialogflow (a tool for creating conversational interfaces). This tutorial will show how to make your Voice Kit communicate with Dialogflow (and Actions on Google) to control an LED light with the Photon by voice.

Get all the code for this example here.

What's included

This example ties together multiple technology platforms, so there are a few separate components included in this repo:

• dialogflow-agent - an agent for Dialogflow
• dialogflow-webhook - a web app to parse and react to the Dialogflow agent's webhook
• particle-photon - a Photon app to handle web requests, and to turn the light on and off

We've included two separate web app implementations. Choose (and build on) the one that best suits your preferences:

• 1-firebase-functions - a microservices-oriented implementation, built for deployment to Cloud Functions for Firebase — a serverless, on-demand platform
• 2-app-engine - a server-based implementation, designed to run on Google App Engine (or your server of choice)

This should be enough to get you started and on to building great things!

What you'll need

We’ll build our web app with Node.js, and will rely on some libraries to make life easier:

• The Actions on Google Node.js Client Library
• The Particle API JS SDK

On the hardware side, you will need:

• Your AIY Voice Kit running the Google Assistant demo (or another device running Google Assistant)
• a Particle Photon (or a similar web-connected microcontroller, like the Redbear Duo)

It's handy to have a breadboard, some hookup wire, and a bright LED, and the examples will show those in action. However, the Photon has an addressable LED built in, so you can use just the Photon itself to test all the code presented here if you prefer.

You'll also need accounts with:

• Dialogflow (for understanding user voice queries)
• Google Cloud (for hosting the webhook webapp/service)
• Particle Cloud (for deploying your Photon code and communicating with the Particle API)

If you're just starting out, or if you're already comfortable with a microservices approach, you can use the 1-firebase-functions example — it's easy to configure and requires no other infrastructure setup. If you'd prefer to run it on a full server environment, or if you plan to build out a larger application from this, use the 2-app-engine example (which can also run on any other server of your choosing).

If you've got all those (or similar services/devices) good to go, then we're ready to start!

Getting started

Assuming you have all the required devices and accounts as noted above, the first thing you'll want to do is to set up apps on the corresponding services so you can get your devices talking to each other.

Local setup

First, you'll need to clone this repo, and cd into the newly-created directory.

git clone git@github.com:google/voice-iot-maker-demo.git
cd git@github.com:google/voice-iot-maker-demo.git

You should see three directories (alongside some additional files):

• dialogflow-agent - the contents of the action to deploy on Dialogflow
• dialogflow-webhook - a web application to parse the Google Actions/Dialogflow webhook (with server-based and cloud function options)
• particle-photon - sample code to flash onto the Particle Photon

Once you‘ve taken a look, we’ll move on!

Dialogflow

Using the Dialogflow account referenced above, you‘ll want to create a Dialogflow agent. We'll be setting up a webhook to handle our triggers and send web requests to the Particle API.

1. Create a new agent (or click here to begin). You can name it whatever you like
2. Select Create a new Google project as well
3. In the Settings section (click on the gear icon next to your project name) and go to Export and Import
4. Select Import from zip and upload the zip provided (./dialogflow-agent/voice-iot-maker-demo.zip)

You've now imported the basic app shell — take a look at the new ledControl intent (viewable from the Intents tab). You can have a look there now if you're curious, or continue on to fill out the app's details.

1. Head over to the Integrations tab, and click Google Assistant.
2. Scroll down to the bottom, and click Update Draft
3. Go back to the General tab (in Settings), and scroll down to the Google Project details.
4. Click on the Google Cloud link and check out the project that's been created for you. Feel free to customize this however you like.
5. Click on the Actions on Google link, and go to 2 - App information
6. Click Add, and fill in the details of your project there
   1. Add some sample invocations, as well as a pronunciation of your Assistant app's name
   2. Fill out the other required fields (description, picture, contact email, etc.)
7. Scroll down to the bottom, and click Test Draft

You can now test out the conversational side of the app in one of two ways:

• Talk to the Google Actions simulator: https://developers.google.com/actions/tools/simulator
• Test queries (by the GUI or cURL) via the try it now interface in the Dialogflow GUI

You can also try talking to your application on any Assistant-enabled device that you‘re signed into.

However, if you’re following along step-by-step, it won't turn any lights on yet — we still have to set up the web service and the Photon app. Onward then!

Google Cloud

Depending on which hosting environment you want to use, cd into either ./dialogflow-webhook/1-firebase-functions or ./dialogflow-webhook/2-app-engine, and continue the setup instructions in that directory's README.md file.

IMPORTANT: Regardless of what hosting/deployment method you choose, make sure you return to the Dialogflow panel and go into the Fulfillment tab to update the URL field. Also, check that the DOMAINS field is set to "Enable webhook for all domains". Without doing these things, Dialogflow won't be able to talk to your new webhook.

Particle

Make sure the Photon is correctly set up and connected. (If it’s not configured yet, follow the steps in the Particle docs

You can upload your code to your photon via the Particle web editor, the Particle Desktop IDE (based on Atom), or the Particle command-line tools.

We'll be using the CLI for this example, which you can install thusly:

sudo npm i particle-cli -g

To deploy via the command line, first make sure you’re logged in:

particle login

You can find out the ID of your device by running:

particle list

Then upload the code using that ID:

particle flash [YOUR-DEVICE-ID] particle-photon/particle-blink-demo.ino

The Photon should blink rapidly while the upload is in process, and when it's done (and calmly pulsing cyan), you're ready to go.

Note: Make sure you generate a Particle access token, and add that token (along with your Photon's device id) to your config.js file.

You can make sure it all works by running the following from your terminal:

curl https://api.particle.io/v1/devices/[YOUR-DEVICE-ID]/led -d access_token=[YOUR-ACCESS-TOKEN] -d led=on

If everything is configured properly, you should see something like the following:

{
    "id": "[YOUR-DEVICE-ID]",
    "last_app": "",
    "connected": true,
    "return_value": 1
}

You should see the Photon's light come on (along with an LED on the breadboard, if you've wired one up)! Doing the same with led=off will return a 0 instead of a 1, and will (you guessed it) turn the light off.

Note: If you ever see a "return_value":-1, that's an error message — something has gone wrong somewhere.

Putting it all together

Once you’ve uploaded all the code and each service is configured, it’s time to give it all a try! You can confirm that everything went to plan by going to either your Assistant-enabled device or the Google Actions simulator, asking to talk to your app ("talk to [APP-NAME]"), and typing "turn the light on". If all goes well, your LED should turn on!

Further reading

This application is just a taste of what's possible — how far you take this framework is up to you! Here are a few resources to help you continue on your journey:

• Actions on Google: Invocation and Discovery
• Actions on Google: Dialogflow
• Sample Google Actions
• Particle/Google Cloud Platform integration

By default, your Voice Kit doesn't do anything when it boots up; you must manually execute a program such as the Assistant Demo. But you can make this program (or any other program of your choice) run automatically upon bootup by creating a systemd service.

All you need is a .service configuration file. This file accepts a long list of configuration options, so we've provided an example for you that starts the Assistant Demo on bootup. You can find the file at ~/AIY-projects-python/src/examples/voice/assistant_grpc_demo.service and it looks like this:

[Unit]
Description=Assistant gRPC Demo
Wants=network-online.target
After=network-online.target
Wants=systemd-timesyncd.service
After=systemd-timesyncd.service

[Service]
Environment=DISPLAY=:0
Type=simple
Restart=always
User=pi
ExecStart=/usr/bin/python3 -u /home/pi/AIY-projects-python/src/examples/voice/assistant_grpc_demo.py --language en-US

[Install]
WantedBy=multi-user.target

To put this file into action, you need to put it into the /lib/systemd/system/ directory. But instead of actually moving this file there, you can create a symbolic link (a "symlink") in this directory that points to the file that already exists. You can do this as follows:

# Move to the directory with the .service file
cd ~/AIY-projects-python/src/examples/voice/

# Create the symlink
sudo ln -s `pwd`/assistant_grpc_demo.service /lib/systemd/system

# Reload the service files so the system knows about this new one
sudo systemctl daemon-reload

Now tell the system to run this service on bootup:

sudo systemctl enable assistant_grpc_demo.service

All set! You can try rebooting now to see it work.

Or manually run it with this command:

sudo service assistant_grpc_demo start

Note: Even after the prompt returns from this command, it takes a few moments for the program to start. When the Assistant Demo is ready, the button lights up.

If you want to stop the service from running on bootup, disable it with this command:

sudo systemctl disable assistant_grpc_demo.service

And to manually stop it once it's running, use this command:

sudo service assistant_grpc_demo stop

You can check the status of your service with this command:

sudo service assistant_grpc_demo status

Now to create a similar service for your own program, just copy the configuration from above to a new file such as my_program.service (the name must end with .service). Then (for most programs) all you need to do is change ExecStart so it points to your program's Python file (and passes it any necessary parameters), and change Description to describe your program. Then follow the steps above using your service name instead.

If you'd like to better understand the service configuration file, see the .service config manual.

If you need to see more logs to help with debugging (or you're simply curious to see more output), you can view system logs and program-specific logs using the journalctl tool.

By default, this prints a lot of system information that won't be useful to you, so it's best if you launch your program as a service and then tell journalctl to print only the logs from that service.

For example, if you start the Assistant Demo as a service, you can begin printing all log output for that service with this command:

sudo journalctl -u assistant_grpc_demo -f

The -f option continuously prints new log entries as they occur. To stop printing the log, press Control+C.

To get the latest bug fixes and features, update the system image for your kit as follows:

1. Download the latest .img.xz file from our releases page on GitHub.
2. Use an adapter to connect your microSD card to your computer.
3. Download, install, and launch the Raspberry Pi Imager.
4. Click Choose OS, scroll to the bottom, select Use custom, and find the .img.xz file you downloaded above.
5. Click Choose storage to select your microSD card and then click Write to begin flashing the SD card.

When flashing is done, put the microSD card back in your kit and you're good to go!

If you're having trouble assembling your kit or running the demos, try the following resources:

• Help documentation
• AIY Forums
• AIY Stack Overflow
• AIY GitHub Issues
• support-aiyprojects@google.com