Activity 1: Image Recognition
Train and Test the Image Classifier
The first step is to open Google Teachable Machine in order to create an image recognition model.
Decide how many categories, or classes, you want and name them. Then use the webcam to take pictures for each class. For example, you might take pictures of you and your partner, or you with and without a stuffed animal. Aim for about 300 pictures for each class (this only takes a few seconds) and try to move around slightly so the camera gets a variety of angles.
Once you have collected data for each of your classes, click Train Model.
The training will take about a minute. Make sure to leave the tab open while the model is training, even if your browser pops up a warning that the window is unresponsive.
When the training is complete, you will be able to test your model in the Preview panel. Make sure that your model works the way that you want it to before moving on. If it doesn’t, you may need to add more image samples for each class and train again. When you are happy with your model, click Export Model.
Keep all of the defaults as they are, and click Upload my model.
After your model has uploaded, copy your sharable link. You will need this link to create a Snap! project with your model. After you copy the link, you can close the export window.
It is a good idea to save your model in case you want to reference or change it later. To do this, click on the Teachable Machine menu and either download the file or save it to your Google drive.
Using the Image Classifier in Snap!
If you are using the BlueBird Connector, open this project in Snap! and save a copy for yourself.
If you are using snap.birdbraintechnologies.com, import this project into Snap!.
Click on the Settings menu and enable JavaScript extensions. You will need them for this project.
Modify the top script to set the URL variable to the link for your classifier. Then click on this stack of blocks to run the top script. You will only need to run this script once to load the libraries and the model. If the URL was correct and the model loaded correctly, you will see a message that reads “Model loaded successfully”. If this did not happen, check that the URL is correct and try clicking the stack again.
Press the spacebar to see your classifier make predictions in Snap!. It will take up to a minute for the classification to start the first time you run the script. When it starts, the Snap! stage will show you an image of what the webcam sees and a table of prediction information. The table lists each classification class and the probability that the current image on the webcam belongs to that class. A probability close to 1 tells you that the image probably belongs to that class, while a probability close to 0 tells you that the image probably doesn’t belong to that class. For example, in this picture, the probability of “Green Bubblegum” is greater than 0.99, while the probabilities of all other classes are very small. Note sometimes probabilities will seem to show values greater than one; that is only because extremely small values are shown in scientific notation.
The stage also shows a list variable named Prediction. While the classifier is predicting, you can use the values in this list to control Hummingbird components. Each item in Prediction corresponds to a row in the table. For example, the first item in Prediction is the probability for the first class.
You can use the probabilities to make the Hummingbird perform certain actions when each class is detected. For example, this code turns on an LED when there is a high probability that the webcam image is in class 1, and turns it off if not.
Challenge: Modify the program to make the Hummingbird respond to a variety of objects. As you test your program, notice what happens if the webcam sees an image that is not similar to something in your training data. In the example below, the zombie robot responds differently to candy than it does to a picture of brains (for full enjoyment of the video, turn on sound).