Note: the above illustration was generated using GPT4o

The source code for the project can be found here.

The Pet Security Cam project explores the development of a compact, AI-powered security camera system designed to monitor and record pet activity. The primary goal is to run the system on the BeagleY-AI board, using a mixture of C and Python to utilize machine learning libraries like the YOLOv8 model and OpenCV, enable UDP communication, and interface with additional hardware components.

The embedded hardware can detect and track a pet's real-time movement. When the pet is detected, the system keeps track of the starting frame and trims the footage into clips. The user is notified and can access the clips via their email or through a local web app with a built-in video player and clip details.

Here is the diagram of the hardware and software of the project:

A layout of hardware components diagram

A layout of software components diagram

Camera for Streaming/Recording with Web Camera Setup

The core hardware aspect of our project is the camera. The camera allows users to stream live footage of the pet while recording the video throughout the session until the user chooses to terminate the running stream. It works like any mass-production security camera and has basic functionality

GStreamer Framework

Our pet security camera uses the GStreamer framework for real-time video streaming and recording. We use a custom pipeline to handle the video feeds from the camera to the UDP network:

v4l2src → image/jpeg → jpegdec → videoconvert → jpegenc → rtpjpegpay → udpsink

Here is a breakdown of each pipeline:

• v4l2src: Captures raw video from the USB camera using Video4Linux2.
• image/jpeg: Indicates the input video format.
• jpegdec: Decodes the JPEG video frames.
• videoconvert: Converts colour formats as needed.
• jpegenc: Re-encodes frames into JPEG.
• rtpjpegpay: Wraps JPEG frames into RTP packets.
• udpsink: Sends the video stream over UDP to a target IP and port.

By default, GStreamer is not included when installing OpenCV via pip install. To enable GStreamer support, we had to compile from source, following the instructions from the OpenCV GitHub repository. However, our MacBook Pro with an M1 chip ran into compatibility issues with Apple’s native libraries. We resolved these by rebuilding OpenCV specifically for our project, following the steps outlined in this Blue Robotics forum post.

Web Application

We used Create React App for the front end to initialize the project and separate the client from the server. This setup allows us to define specific endpoints and ports for sending and retrieving data between the host and the target device. We also integrated TailwindCSS via Content-Delivery Network (CDN) to quickly style the interface and give it a clean and modern look.

In addition, we use an Express.js server on the backend to handle communication endpoints for the React client and the BeagleYAI board. This gives us finer control over how the host and target exchange data. In addition, we use WebSockets to stream the webcam feed to the client in near real-time.

Pet Detection AI

We used a pre-trained YOLOv8 model by Ultralytics to detect the "pet" class. For our purposes, we grouped the following labels under "pet": Rabbit, Dog, Cat, Animal, Raccoon, and Squirrel. We also observed that the model is biased toward detecting dogs and cats, making it less reliable for identifying other pets like rabbits. The model can be fine-tuned to improve detection accuracy with more targeted training data.

Video Clipping Implementation

This component allows the system to extract defined segments from longer recordings when pet motions are detected, creating video clips that can be shared via email notifications or viewed through the web interface. By combining GStreamer for continuous recording and FFmpeg for precise clip extraction, the clipping function can integrate with our pet detection system through the basic workflow:

• The pet detection module identifies activity and triggers a clip request with specific start and end times.
• The Board captures that request and starts a background thread, starting with clipping.
• After clip generation, the thread verifies the clip's integrity by checking file existence and size.
• Then it initiates the email notification process with the newly created clip as an attachment.
• Finally, it sends the clip back to the front end so it can be displayed on the website.

Initially, we had separate processes for clipping and emailing, and a script would run these processes, but this caused the streaming to stall, so we transitioned to using a single executable and background threads. This solved the stream stalling and still let up clip and email the footage.

Emailing Functionality

Our system also supports sending the clipped video of the last 5 seconds, when the pet was seen in the frame and disappears from it, and attaching it to a chosen default email address. The camera will start sending an email with the clipped video attached to it and send it to the receiver's email address. Then the process will rinse and repeat as mentioned. The primary purpose of this functionality is to ensure the pet is always present in a short time around the room/house, so our program will consistently send the clipped video to ensure the owner that their pet is active.

Pan-Tilt Kit

The Pan-tilt kit gives the user control over the web camera's direction. Users can control the camera movement by utilizing the Zen Hat’s joystick. The pan-tilt kit consists of 2 motors, each requiring an individual GPIO for control: GPIO 6 and 10. Regarding the code, the camera movement is done in C and was based on our assignments’ code used in the joystick and pan-tilt kit. The motor functionality utilizes PWM to precisely manipulate the duty cycle and period to control the movement and boundaries of the camera. Due to the kit’s placement with the 3D model, valuations of determined values for the motor (starting/center position, maximum and minimum) were adjusted based on preference.

The kit consists of 2 servo motors. Each servo has three wires: one for power (4.8- 6.0V), ground, and signal. The necessary 5V was found by utilizing the 5V port for the LED strip. GPIO pins used to control the motors were GPIO 6 and 10, which were from the PYMNL header and the UART pins. To link the GPIO pins to the motors, 2 470Ω resistors were utilized to prevent overvoltage, as the GPIO pins use 3.3V while the servo motors utilize 5V. Regarding the pin used for the grounding wire, there are multiple options on the Zen Hat for every header. Any is viable, but the grounding pin on the UART was used for simplicity and preference. To set up the PWM, we needed to change the extlinux.conf file so that Linux knows those pins are going to use PWM.

The circuit diagram and physical layout of the motor connected to the BeaglyY-AI board using the breadboard provided are shown here:

A diagram of board layout.

A diagram of servo circuit.

Pet Tacking

The YOLO pet detection outputs a square around the target. That square is used to calculate where the pet is in the frame. The location is normalized to the frame size, and then if the pet is outside a threshold of 0.5, it is considered not centred. When that happens, the web app sends a signal over UDP to the board. The signal contains which axes the pet has moved on and how many degrees it has moved. The board captures that signal in its network module and moves the corresponding motor that many degrees (degrees can be positive or negative).

Production Deployment

The system is designed to run locally within a Wi-Fi-enabled home network. However, the Wi-Fi connection must have internet access to enable automatic email delivery of video clips. The pet security camera should be placed in a corner of the room where pets frequently roam for optimal performance. Ensure the area has clear visibility and good lighting, as this helps the camera capture higher-quality video and helps AI detect pets more accurately and reliably.