Maker Kit

The following instructions show how to flash your microSD card with our AIY Maker Kit system image and assemble your case. The Maker Kit system image includes Raspberry Pi OS with all software and settings that enable the ML projects that follow.

Before you start, double check that you have all the required materials. It's okay if you don't have your case yet; you can start programming now and put that together later.

The following instructions are for the 3D-printed case. If you don't have your case yet, that's okay, just connect the hardware as described below (connect the camera and USB Accelerator) and you can put it in the case later.

If you're building the cardboard case, instead follow the instructions in the PDF download and when you're done, skip to turn it on.

Open the camera latch on the Raspberry Pi

Between the HDMI ports and the headphone jack, you'll find the camera cable latch.

Pull up on the black latch to open it.

Insert the flex cable and close the latch

Grab the end of the camera flex cable (it should already be attached to the Pi Camera) and insert it so the blue strip is facing toward the black latch.

Be sure the cable is inserted fully, and then push down on the latch to lock the cable in place.

Insert the Raspberry Pi

Start by inserting the side of the board with the headphone jack so those plugs rest in the corresponding holes.

Then press down on the opposite side of the board (with the 40-pin header) so it snaps into place. You can gently pull outward on the case wall so it snaps in easier.

Make sure it's inserted fully: The Ethernet port should rest flat on the case opening, and the SD card slot on the opposite side should be accessible.

Note: The SD card should not be in the board yet—it can make inserting the board more difficult.

Attach the Camera

First set the camera's top mounting holes onto the matching pegs, and hold them firmly in place.

Be sure the holes on the camera board are aligned with the posts, and then press firmly on the cable connector to snap the camera into place. You might need to press hard.

Ensure that both side clips are now holding the board and the camera lens is set in the hole.

Fold the flex cable

Press the cable against the roof of the case and then fold it backward where it meets the wall, creasing the cable.

Then fold the cable again at a 90-degree angle after it passes over the camera, creasing it so the cable turns toward the outer USB cutout, as shown in the photo.

Don't worry, you will not damage the cable by creasing it.

Set the cable inside

As you close the lid, the flex cable should turn backward one more time, right behind the USB ports. You do not need to crease the cable here.

Snap the lid shut

First set the clips on the side near the board's GPIO header.

Then set the opposite side by pressing gently on the side with the plug openings, while pressing down on the lid.

Attach the USB Accelerator

Start by setting the top corners of the USB Accelerator into the case, so the USB plug is on the same inside as the microSD port on the Raspberry Pi.

Then press down to snap it in.

Connect the USB cable

Connect the USB cable to one of the blue ports on the Raspberry Pi, and press the cable into either of the two cable guides.

The blue ports support USB 3.0, which provides much faster data-transfer speeds than USB 2.0.

Leave the USB Accelerator unplugged so you can access the microSD slot.

Attach the USB microphone

If you plan on using audio classification (speech recognition), then attach your USB microphone to any of the available USB ports.

Note: The headphone jack on the Raspberry Pi does not support input, so you must use a USB microphone (unless you're also using audio HAT with a mic input).

You just got started, so don't shut down yet, but when you decide you're done for the day, it's important that you properly shut down the system before unplugging the power.

You can shut down the system in two ways:

• When viewing the Raspberry Pi OS desktop, open the applications menu (the Raspberry Pi icon in the top-left corner of the desktop) and click Shutdown.

• From the Raspberry Pi terminal, run the following command:

    sudo shutdown

  Copy

Wait for the green LED to stop blinking (the red LED stays on even when the operating system is shut down), and then unplug the power.

Now the real fun begins! Let's build some Python projects with machine learning models!

To make these projects easier, we created some helpful Python APIs in the aiymakerkit module. These APIs perform the complicated (and boring) tasks that are required to execute an ML model. Our APIs also perform common tasks such as capturing images from a camera, drawing labels and boxes on the images, and capturing audio from a microphone. This way, you need to write a lot less code and can focus on the features that make your projects unique.

So let's get started!

All the software you need is already installed on the system image you flashed to your SD card. So let's run a quick test to make sure your hardware is connected and working as expected.

1. Open the TerminalThe Terminal is an app that allows you to issue command-line instructions for the computer. On the Raspberry Pi desktop, the Terminal icon appears in the task bar at the top (it looks like a black rectangle). Click it to open a terminal window. on your Raspberry Pi. By default, the terminal opens to your Home directory and the prompt looks like this:

   pi@raspberrypi:~ $
   

   pi is the username and raspberrypi is the device hostname. These are the default names, but perhaps yours are different if you changed them in the Raspberry Pi Imager options.

   The squiggly line (called a tilde) represents the Home directory, which is currently where the command prompt is located. (You can run pwd to see the full path name for your current directory.)

2. Change your directory to the aiy-maker-kit folder by typing this command and then pressing Enter:

   cd aiy-maker-kit
   

   Copy

3. Then run our test script with this command:

   python3 run_tests.py
   

   Copy

   This runs some tests to be sure the required hardware is detected. If all the tests succeeded, the terminal says "Everything looks good." If something went wrong, you should see a message with a suggestion to solve the problem, so take any necessary steps and try again.

Now you're ready to build some projects with ML!

But before you continue, change your directory again to the examples folder where we'll get started:

cd examples

This project builds upon the detect_faces.py example, so let's start by running that code:

1. Your terminal should already be in the examples directory, so just run this command:

   python3 detect_faces.py
   

   Copy

   Make sure the camera is pointed toward you and your face is well lit (point the camera away from any windows/lights).

   When the video appears on the screen, you should see a box drawn around your face. Also try it with multiple faces. If there's nobody else around, you can hold up a photo instead.

   Note: If you see a warning that says, "Locale not supported by C library," you can ignore it. It's harmless and related to your locale change during setup, so it will go away after a reboot.

2. Press Q to quit the demo. (If it does not close, click on the camera window so it has focus and then press Q again.)

3. To start building the smart camera, make a copy of detect_faces.py with a new name that will be your project file. Here's a quick way to make a copy from the terminal:

   cp detect_faces.py smart_camera.py
   

   Copy

4. Now open the new file for editing in ThonnyThonny is an integrated development environment (IDE) for Python programming—basically a fancy text editor that specializes in Python code. (learn more about Thonny):

   thonny smart_camera.py &>/dev/null &
   

   Copy

Next, we'll start writing some code to build the smart camera.

for frame in vision.get_frames():
    faces = detector.get_objects(frame, threshold=0.1)
    print(faces)  # You add this line

from pycoral.adapters.detect import BBox

# Define the auto shutter detection zone
width, height = vision.VIDEO_SIZE
xmin = int(width * 0.25)
xmax = int(width - (width * 0.25))
ymin = int(height * 0.2)
ymax = int(height - (height * 0.2))
camera_bbox = BBox(xmin, ymin, xmax, ymax)

for frame in vision.get_frames():
    faces = detector.get_objects(frame, threshold=0.1)
    print(faces)  # You can delete this line now
    vision.draw_objects(frame, faces)
    vision.draw_rect(frame, camera_bbox)

def box_is_in_box(bbox_a, bbox_b):
    if ((bbox_a.xmin > bbox_b.xmin) and (bbox_a.xmax < bbox_b.xmax)) and (
        (bbox_a.ymin > bbox_b.ymin) and (bbox_a.ymax < bbox_b.ymax)):
        return True
    return False

for frame in vision.get_frames():
    faces = detector.get_objects(frame, threshold=0.1)
    vision.draw_objects(frame, faces)
    vision.draw_rect(frame, camera_bbox)
    if faces and box_is_in_box(faces[0].bbox, camera_bbox):
        print("Face is inside the box!")

timestamp = datetime.now()
filename = "SMART_CAM_" + timestamp.strftime("%Y-%m-%d_%H%M%S") + '.jpg'
filename = os.path.join(PICTURE_DIR, filename)
vision.save_frame(filename, frame)

This project has some similarity with the smart camera because you'll inspect the location of an object. But this project expands on the concepts you already learned because it teaches you how to handle a model that can recognize lots of different things (not just faces) and you'll perform more sophisticated bounding-box comparisons.

First, try the object detection demo that we'll use as our starting point:

python3 detect_objects.py

See if it can correctly label your keyboard, cell phone, or scissors.

Remember that each ML model can recognize only what it was trained to recognize. Whereas the previous face detection model could recognize only faces, this model can recognize about 90 different things, such as a keyboard, an apple, or an airplane. To see what objects it was trained to recognize, open the coco_labels.txt file in aiy-maker-kit/examples/models/.

Play with this for a while by showing it different objects that are listed in the labels file (try searching the web for images on a phone or computer, and hold that up to the camera). You'll probably notice the model is better at recognizing some things more than others. This variation in accuracy is due to the quality of the model training. But one thing the model seems to detect consistently well is people.

So let's build a program that responds when a person appears in the image and detect when they only partially enter an area.

To get started, make a copy of detect_objects.py and open it:

cp detect_objects.py security_camera.py

thonny security_camera.py &>/dev/null &

# Define the protected fence region
width, height = vision.VIDEO_SIZE
xmin = 0
ymin = 0
xmax = int(width * 0.5)
ymax = int(height * 0.5)
fence_box = BBox(xmin, ymin, xmax, ymax)

for frame in vision.get_frames():
    vision.draw_rect(frame, fence_box)

[Object(id=0, score=0.79140625, bbox=BBox(xmin=227, ymin=84, xmax=347, ymax=227))]

for frame in vision.get_frames():
    vision.draw_rect(frame, fence_box)
    objects = detector.get_objects(frame, threshold=0.4)
    for obj in objects:
        label = labels.get(obj.id)
        if 'person' in label:
            print('A person was detected!')

if 'person' in label:
    person_area = obj.bbox.area
    overlap_area = BBox.intersect(obj.bbox, fence_box).area
    if (overlap_area / person_area) > 0.3:
        vision.draw_rect(frame, obj.bbox, color=RED)
    else:
        vision.draw_rect(frame, obj.bbox, color=GREEN)

from pycoral.adapters.detect import BBox

RED = (0, 0, 255)
GREEN = (0, 255, 0)

The object detection model used in the previous project was trained to recognize only a specific collection of objects (such as people, bicycles, and cars)—you can see all the objects it's trained to recognize in the coco_labels.txt file (about 90 objects). If you want to recognize something that's not in this file, then the model won't work, you would have to train a new version of the model using sample images of the object you want it to recognize.

However, training an object detection model (as used for face and person detection) requires a lot of work to prepare the training data because the model not only identifies what something is, but where it is (that's how we draw a box around each person). So to train this sort of model, you need to provide not only lots of sample images, but you must also annotate every image with the coordinates of the objects you want the model to recognize.

Instead, it's much easier to train an image classification model, which simply detects the existence of certain objects, without providing the coordinates (you cannot draw a bounding box around the object). This type of model is useful for lots of situations because it's not always necessary to know the location of an object. For example, an image classification model is great if you want to build a sorting machine that identifies one item at a time as they pass the camera (perhaps you want to separate marshmallows from cereal).

There are several ways you can train an image classification model that works with the Coral USB Accelerator, and we've linked to some options below. However, our favorite (and the fastest) method is to perform transfer learningTraining an ML model ordinarily takes a long time and requires a huge dataset. However, transfer learning allows you to start with a model that's already trained for a related task and then perform further training to teach the model new classifications but with a much smaller training dataset. on the Raspberry Pi, using images collected with the Pi Camera.

So that's what we're going to do in this project: collect some images of objects you want your model to recognize, and then train the model right on the Raspberry Pi, and immediately try out the new model.

python3 train_images.py -l labels.txt

Class: Can (id=1)
  capture/Can/PI_CAM_20210827_013859660080.png => [-0.0546875  0.015625   0.0078125 ... -0.03125   -0.0234375  0.0390625]
  capture/Can/PI_CAM_20210827_013845829924.png => [-0.0390625  0.0078125  0.0078125 ... -0.0234375 -0.0078125  0.03125  ]

python3 classify_video.py -m my-model.tflite -l labels.txt

If you finished the project above to train an image classification model then you've taken your first big step into the world of machine learning, because you actually performed the "learning" part of machine learning (usually called "training"). In this next project, we're going to continue with model training, but things are going to get more interesting (and a bit more complicated).

In this project, we're going to train a brand new model that can recognize a variety of different poses that you make with your body. The basis for our pose classification model is the output from a pose detection model (also called a pose estimation model), which is trained to identify 17 body landmarks (also called "keypoints") in an image, such as the wrists, elbows, and shoulders, as illustrated below.

Photo credit: imagenavi / Getty Images

This pose detection model is already trained and we won't change it, but it only identifies the location of specific points on the human body. It doesn't know anything about what sort of pose the person is actually making. So we're going to train a separate model that can actually label various poses (such as the name of yoga poses) based on the keypoints provided by the pose detection model.

To get started, try the pose detection model:

python3 detect_poses.py

Orient your camera and step away so it can see all or most of your body. Once it's running, you should see a sort of stick figure drawn on your body.

To understand this a little better, open the detect_poses.py file and update it so it prints the pose data returned by get_pose():

for frame in vision.get_frames():
    pose = pose_detector.get_pose(frame)
    vision.draw_pose(frame, pose)
    print(pose, '\n')

Now when you run the code, it prints a large array of numbers with each inference, which specifies the estimated location for 17 keypoints on the body. For each keypoint there are 3 numbers: The y-axis and x-axis coordinates for the keypoint, and the prediction score (the model's confidence regarding that keypoint location). The draw_pose() function simply uses that data to draw each point on the body (if the prediction score is considered good enough).

This is very cool on its own, but we want to create a program that actually recognizes the name of each pose we make.

To do that, we'll train a separate model that learns to identify particular patterns in the keypoint data as distinct poses. For example, the model might learn that if both wrist keypoints are above the nose and elbow coordinates, that is a "Y" pose. So, this model will be trained, not with images from the camera, but with the keypoint data that's output from the pose detection model above.

So let's start by collecting images of these poses…

In this project, we'll train an audio classification model to recognize new speech commands. There are lots of things you can build that respond to voice comamnds, but we'll start with a voice-activated camera.

Before we begin, let's quickly discuss what goes into and comes out of the audio model. For comparison, the input for the previous image ML models is simple: they take an image. Even when using an object detection model with video, the model takes just one image at a time and performs object detection, once for every frame of video. But audio is a time-based sequence of sounds, so you might be wondering how we put that kind of data into the model.

First of all, how an audio model receives input and provides results depends on the type of task you want to perform. For example, a model that performs voice transcription (speech to text) is very different from a model that recognizes specific words or phrases (such as "Hey Google"). Our model is the second type; it will recognize only specific words or phrases. In fact, it can recognize only speech commands or sounds that fit within one second of time. That's because the input for our model is a short recording: the program listens to the microphone and continuously sends the model one-second clips of audio.

What later happens inside our speech recognizer model is similar to what happens in one of the image classification models, because the one-second audio clip is first converted into an image. Using a process called fast fourier transformation, the model creates a spectrogram image, which provides a detailed image representation of the audio signal in time. Though you don't need to understand how this works, it's useful to simply know that the audio model actually learns to recognize different sounds by recognizing differences in the spectrogram image. Then the model outputs a classification (such as "on" or "off") with a confidence score (just like an image classifier or object detector).

So, to create our own speech recognizer, we need to record a collection of one-second samples for each word or phrase we want to teach the model. When the training is done, our program will repeatedly feed one-second sound recordings to the model and the model will do its best job to predict whether we said one of the learned commands.

Training a custom model has never been easier than this: We're going to use a web app called Teachable Machine to record samples for training, and then train the model with the click of a button.

To get the best results from your trained model, it's important that you capture training samples that are very similar to (if not identical to) the type of data you will feed the model when you perform predictions. This means, for example, you should record the training samples in the same environment and with the same microphone you will use when running the model. Thus, we recommend you use your Raspberry Pi with the connected microphone to perform the following sample collection.

Open Teachable Machine

On your Raspberry Pi, open the web browser and go to teachablemachine.withgoogle.com/train/audio.

Record background noise

1. In the Background Noise panel, click Mic.
2. Click Record 20 Seconds and then don't speak while it records.
3. When it's finished recording, click Extract Sample.

It will convert the 20-second recording into 20 one-second samples.

Record the first speech command

1. In the second panel, click Class 2 and rename it to the first voice command, "Cheese".
2. Click Mic to open the recorder. This time, it will record just 2 seconds at a time, and then split that into 1-second samples. So for each recording, you need to say "cheese" twice.
3. Get ready... As soon as you click Record 2 seconds, you need to say "cheese" once and then say it again right after the cursor passes the middle marker. Give it a try.

The recorded sample shows the spectrogram of your recording, so if you timed it well, you should see the bright color when you said "cheese" on the left and right side. If either "cheese" recording overlaps the white middle line, click Record 2 Seconds to try again.

1. If the "cheese" samples look good, click Extract Sample and the recording will be split into separate samples on the right side.
2. Repeat steps 3 and 4 nine times so that you get a total of 20 audio samples on the right.

The more samples you record, the more accurate your model becomes. For this project, 20 samples should work fine.

Tip: Each time you record yourself, adjust your pronounciation slightly (for example, say it faster/slower and with a higher/lower pitch). This will help the model learn to recognize the same word even when said in a different manner. You can make your model even better if you get samples from several different people.

Record another speech command

You can add as many other speech commands as you want. For this project, let's do one more. The "Cheese" command will take a photo and then keep listening for more commands, so let's add the ability to quit the program when we say "Stop."

1. Below the Cheese panel, click Add a class.
2. Click Class 3 and rename it "Stop."
3. Repeat the steps above to record samples. As before, perform 10 recordings (each time, say "Stop" twice) so you get a total of 20 audio samples.

Train and review the model

1. Click Train Model.

In less than a minute, the model will be ready.

1. Teachable Machine then starts running the model by listening to your microphone and passing one-second clips to the model.

2. Say "Cheese" and "Stop" or whatever else you recorded to see how well the model recognizes each command.

If you think the model could be better, you can record more samples and/or add more classes, and then click Train Model again.

Note: The model always performs inference using one second of audio, but the preview includes an option to adjust the overlap factor, which is set to 0.5 (half a second) by default. This means, instead of recording one second at a time and sending data to the model just once per second, it records one-second samples that overlap each other by a half second, thus sending data to the model every half second.

Download the TF Lite model

1. If you're satisfied with the model performance, click Export Model.
2. In the dialog that appears, click the Tensorflow Lite tab and then click Download my model.

The model should appear on your Raspberry Pi at ~/Downloads/converted_tflite.zip.

Backup your dataset

Collecting audio samples is tedious work, so you should keep the recordings you created, in case you want to retrain your model later. (If you close the web browser now, you will loose all those samples!)

1. For each class, click the three-dot button in the top-right corner of the class panel and click Download Samples or Save Samples to Drive.

2. If you later decide to retrain the model, such as to record a larger dataset or add more commands to the model, you can easily upload these recordings by clicking the Upload button (next to the Mic button).

from picamera import PiCamera
from datetime import datetime
import os

PICTURE_DIR = os.path.join(os.path.expanduser('~'), 'Pictures')
IMAGE_SIZE = (640, 480)
camera = picamera.PiCamera(resolution=IMAGE_SIZE)

try:
    audio.classify_audio(model_file=args.model, callback=handle_results)
finally:
    camera.close()

def capture_photo():
    timestamp = datetime.now()
    filename = "VOICE_CAM_" + timestamp.strftime("%Y-%m-%d_%H%M%S") + '.png'
    filename = os.path.join(PICTURE_DIR, filename)
    camera.capture(filename)
    print('Saved', filename)

def handle_results(label, score):
    if label == 'Stop':
        return False
    elif label = 'Cheese':
        capture_photo()
    return True

python3 voice_camera.py --model_file ~/Downloads/soundclassifier_with_metadata.tflite

Now that you've learned a bit about what's possible using machine learning, what else can you create?

Before you start a new project, it's a good idea to get familiar with the available tools. The project tutorials above introduce several important features from the aiymakerkit API, but there are still more APIs available.

To get started, read through the aiymakerkit API reference (it's not very long). We didn't use all these APIs in the projects above, so you might discover some APIs that will help you with your next project. Plus, some of the functions we already used support more parameters that we didn't use. For example, the draw_objects() function allows you to specify the line color and line thickness for the bounding boxes.

Explore the source code

On your Raspberry Pi, you might have noticed there's a lot more in the aiy-maker-kit/ directory than just example code. This directory also includes the source code for the aiymakerkit Python module (which is what we import at the top of all the examples).

So if you're curious how these APIs work (are you curious how draw_pose() actually draws the keypoints on your body?), just open the aiy-maker-kit/aiymakerkit/ directory and view the files inside. You'll notice that the aiymakerkit API depends on a variety of other APIs, such as PyCoral to perform ML inferencing, OpenCV for video capture and drawing, and PyAudio for audio capture.

These other API libraries offer a wealth of capabilities not provided by aiymakerkit, so you might want to explore those libraries as well to find other APIs you can use in your project.

And, if you see something you'd like to change or add to the aiymakerkit API, just edit the code right there. To then use the changes in your project, just reinstall the aiymakerkit package:

python3 -m pip install -e ~/aiy-maker-kit

The -e option makes the installation "editiable." That means if you continue to change the aiymakerkit source code, any projects that import these modules automatically receive those change—you won't need to reinstall aiymakerkit again.

Finally, if you want to explore the aiymakerkit code online, you can see it all on GitHub.

The number of projects you can build using vision intelligence (such as image classification, object detection, and pose detection) is endless. So here are just a few ideas to get you going:

cd ~/aiy-maker-kit/examples

curl -OL https://raw.githubusercontent.com/google-coral/test_data/master/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite

python3 classify_video.py -m mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite

timestamp = datetime.now()
filename = "BIRD_CAM_" + label + "_" \
           + timestamp.strftime("%Y-%m-%d_%H%M%S") + '.jpg'
filename = os.path.join(PICTURE_DIR, filename)
vision.save_frame(filename, frame)

import time

DISPLAY_LENGTH = 1
PAUSE_LENGTH = 1
poses = ['left', 'right', 'forward', 'stop']

for pose in poses:
    print(' -> ', pose, end='\r')
    time.sleep(DISPLAY_LENGTH)
    print(' -> ', ' ' * len(pose), end='\r')
    time.sleep(PAUSE_LENGTH)
print('GO!')

So far, we've created just one project using audio from a microphone, but there are so many things you can create with a simple speech or sound classification model.

Here are some other project ideas:

When you're done developing your project and ready to put your Maker Kit to work, you might want your Python program to automatically start when the Raspbery Pi boots up. That way, if the board loses power or you need to reset for any reason, all you need to do is power it up and your program starts without requiring you to connect to the desktop or a remote shell.

You can make this happen by creating a systemd service, which is defined in a .service file.

As an example, we included the following detect_faces.service file in ~/aiy-maker-kit/examples. It's designed to run the detect_faces.py example when the device boots up, but it's disabled by default.

[Unit]
Description=Face Detection example

[Service]
Environment=DISPLAY=:0
Type=simple
Restart=on-failure
User=pi
ExecStart=/usr/bin/python3 /home/pi/aiy-maker-kit/examples/detect_faces.py

[Install]
WantedBy=multi-user.target

To enable this service so it runs the face detection example at bootup, run the following commands in the Raspberry Pi terminal:

1. Create a symbolic link (a "symlink") so the .service file can be found in /lib/systemd/system/ (do not move the original file):

sudo ln -s ~/aiy-maker-kit/examples/detect_faces.service /lib/systemd/system

1. Reload the service files so the system knows about this new one:

sudo systemctl daemon-reload

1. Now enable this service so it starts on bootup:

sudo systemctl enable detect_faces.service

All set! You can try manually running the service with this command:

sudo service detect_faces start

Notice that you don't see any terminal output from the Python script, because you didn't actually execute the script from the terminal—it was executed by the service manager. But you can still press Q to quit the example when the camera window is in focus.

Or you can reboot the system to see it automatically start at bootup:

sudo reboot

As soon as the desktop reappears, you should see the camera window and a bounding box around your face.

Customize the service config

To create a service configuration for your own project, just copy the above file using unique file name (the name must end with .service) and save it somewhere in your Home directory (such as /home/pi/projects/). Then edit the Description and change change ExecStart so it executes your program's Python script.

You must then repeat the steps above to create symlink to your service file, reload systemctl, and enable the service.

The .service file accepts a long list of possible configuration options, which you can read about in the .service config manual.

Other tips

To manually stop the service once it's running, use this command:

sudo service detect_faces stop

If you want to disable the service so it no longer runs at bootup, use this command:

sudo systemctl disable detect_faces.service

You can also check the status of your service with this command:

sudo service detect_faces status

The projects above use some trained models and even train a couple new ones, but there are more models you can try and the opportunities to train your own are endless. This section introduces just a few places where you can start exploring other models.

The following models are fully trained and compiled for acceleration on the Coral USB Accelerator.

Each of the following links take you to a page at www.coral.ai where you'll find the model files. On that page, there's a table that describes each available model and the last column includes the download links. Make sure you download the Edge TPU model.

Image classification

Models that recognize the main subject in an image, and are compatible with the Classifier API.

Most models are trained with the same dataset of 1,000 objects, but there are three models trained with the iNaturalist dataset to recognize different types of birds, insects, or plants.

Browse models

Object detection

Models that identify the location and label of multiple objects, and are compatible with the Detector API.

These are trained with a dataset of 90 objects (except for the one model trained to recognize faces—the same model used above). Notice that although some models are more accurate (the mAP score is higher), they are also slower (the latency is higher).

Browse models

Pose detection

Models that identify the location of several points on the human body, but only the MoveNet models are compatible with the PoseDetector API.

The PoseNet models detect the same 17 keypoints but they output a different format, so although they do work with the Maker Kit, you must use different code as demonstrated in this PoseNet example. One of the only reasons you might want PoseNet instead is to detect the pose of multiple people.

Browse models

Semantic segmentation

Models that can classify individual pixels as part of different objects, allowing you to know the more precise shape of an object, compared to object detection (which provides just a bounding-box area).

We do not offer an API for this type of model in aiymakerkit because the applications for such models are more complicated. To try it out, you can use this code with the PyCoral API.

Browse models

Nowadays, crating a model for common machine vision tasks such as image classification and object detection does not require that you actually design the neural network yourself. Instead, you can (and should) use one of many existing network designs that are proven effective, and instead focus on the training effort (gathering your training dataset and fine-tuning the training script).

Training a neural network model from scratch means you start with a network that has no learned weights—it's just a blank neural network. Depending on the model, training a model from scratch can take a very long time. So another shortcut often used (in addition to using an existing network design) is re-using the weights somebody else created when training the model, and then "fine tuning" the weights with a much shorter version of training.

This simplified training process is often called "transfer learning" or just "retraining," and it's the technique we recommend you use with the following training scripts.

All the training scripts below run on Google Colab, which we introduced in the pose classification project. Each script includes its own training dataset so you can immediately learn how to retrain a model, but you can then replace that dataset with your own. To learn more about Google Colab, watch this video.

The following Python APIs are provided by the aiymakerkit module, which is used in the above tutorials.

We designed these APIs to simplify your code for the most common TensorFlow Lite inferencing tasks. But by no means does this represent everything you can do with ML on the Raspberry Pi and Coral USB Accelerator.

To learn more, check out the source code.

To learn more about some of the tools and topics covered above, check out these other resources:

• Linux basics: This is a great resource if you're new to Raspberry Pi OS (which is a Linux operating system). It includes lots of information and tips about using the terminal (the command line) and many other features of a Linux OS.

• Python basics: This is a good reference for basic Python programming syntax. If you want a comprehensive introduction to Python, there are lots of online tutorials you can find just by searching the internet for "python for beginners."

• Machine learning basics with TensorFlow: You can find a plethora of educational content online about ML, but this page offers some specific recommendations about learning ML from the TensorFlow team.

Or maybe you're looking for the Maker Kit source code:

• aiymakerkit Python library
• Maker Kit system image

If you're having trouble with any part of the AIY Maker Kit code or documentation, try these resources:

• Get programming help on Stack Overflow: This is a great place to search for answers or post a new question about various programming topics.

• Get help on the Raspberry Pi Forums: The conversations here are all about Raspberry Pi and include various topics, including Python, robotics, and AIY projects.

• Report an issue with Maker Kit: If you discover something wrong with the AIY Maker Kit software or any of the website documentation, please report it here.

For experienced Raspberry Pi users only!

If you have a fully-functional Raspberry Pi system and you want to upgrade it to accelerate TensorFlow Lite models with the Coral USB Accelerator, then these instructions are for you.

Before you start, be sure you have all the required hardware:

• Raspberry Pi 3 or 4 running Raspberry Pi OS with desktop (Buster only)
• Raspberry Pi Camera
• Coral USB Accelerator

Here's how you can upgrade your system with the Maker Kit software:

1. Before you power the board, connect the Pi Camera Module. Leave the USB Accelerator disconnected until you finish the setup script.

2. Power the board and connect to your Raspberry Pi terminal. You must have access to the Raspberry Pi OS desktop becuase our demos depend on the desktop to show the camera video.

3. Open the Terminal on your Raspberry Pi. Navigate to a path where you want to save the Maker Kit API repository and run this command:

   bash <(curl https://raw.githubusercontent.com/google-coral/aiy-maker-kit-tools/setup.sh)
   

   Copy

That's it! Now go checkout the Maker Kit project tutorials!