Object Detection with a Raspberry Pi and OpenCV

A demonstration of how to perform AI object detection in real time using a Raspberry Pi and a camera module.

This post is an update to the code from this blog by Tim from Core Electronics: “Object and Animal Recognition With Raspberry Pi and OpenCV”.

The main problem found when following this guide was it uses OpenCV to capture the image from the camera, however I could not get OpenCV to work with the official Raspberry Pi camera module that plugs into the CSI camera port, which is the camera I have. One source I read suggests OpenCV image capture only works with USB web cams. I do not have one to test that.

There is another Python module called “camerapi” for capturing images from the official camera module, however this does not work with the 64-bit version of the Raspberry Pi OS (I am using “bookworm” released in March 2024 which is the current recommended version at the time of writing). There is an alternative more recent module called “camerapi2” that does work with the 64-bit Raspberry Pi OS however.

The newer Raspberry Pi O/S encourages uses of Python virtual environments (venv) however the version of “camerapi2” published at the time of writing does not seem to work in a virtual env. When used in a virtual env it complains it cannot find the “libcamera” module even though it is installed. To resolve this problem, the required modules were installed system-wide on the pi and are not used in a virtual environment.

To install the dependencies:

sudo apt install -y python3-picamera2
sudo apt install -y python3-opencv

The object detection neural network model and other files used in this code (files in the directory “Object_Detection_Files”) can be obtained from a ZIP file attached right at the end of this blog. The model (MobileNet v3) is trained on the “Common Objects in Context” data set.

The working code I ended up with is as follows.

Note my camera module is upside down and there is a line that rotates the image 180 degrees. If your camera module is right way up, simply comment this line out.

#!/usr/bin/env python3

import cv2
from picamera2 import Picamera2
import numpy as np


def configDNN():
 classNames = []
 classFile = "Object_Detection_Files/coco.names"
 with open(classFile,"rt") as f:
     classNames = f.read().rstrip("\n").split("\n")

 #This is to pull the information about what each object should look like
 configPath = "Object_Detection_Files/ssd_mobilenet_v3_large_coco_2020_01_14.pbtxt"
 weightsPath = "Object_Detection_Files/frozen_inference_graph.pb"

 dnn = cv2.dnn_DetectionModel(weightsPath,configPath)
 dnn.setInputSize(320,320)
 dnn.setInputScale(1.0/ 127.5)
 dnn.setInputMean((127.5, 127.5, 127.5))
 dnn.setInputSwapRB(True)
	
 return( dnn, classNames )

#thres = confidence threshold before an object is detected
#nms = Non-Maximum Suppression - higher percentage reduces number of overlapping detected boxes
#cup = list of names of objects to detect or empty for all available objects
def objectRecognition(dnn, classNames, image, thres, nms, draw=True, objects=[]):
    classIds, confs, bbox = dnn.detect(image,confThreshold=thres,nmsThreshold=nms)
    
    if len(objects) == 0:
     objects = classNames
    recognisedObjects =[]
    if len(classIds) != 0:
        for classId, confidence,box in zip(classIds.flatten(),confs.flatten(),bbox):
            className = classNames[classId - 1]
            if className in objects: 
                recognisedObjects.append([box,className])
                if (draw):
                    cv2.rectangle(image,box,color=(0,0,255),thickness=1)
                    cv2.putText(image,classNames[classId-1] + " ("+str(round(confidence*100,2))+")",(box[0]-10,box[1]-10), cv2.FONT_HERSHEY_SIMPLEX,0.5,(0,0,255),2)
    
    return image,recognisedObjects

#Below determines the size of the live feed window that will be displayed on the Raspberry Pi OS
if __name__ == "__main__":

    ( dnn, classNames ) = configDNN()

    picam2 = Picamera2()
    #Set the camera format to RGB instead of the default RGBA
    config = picam2.create_preview_configuration({'format': 'RGB888'})
    picam2.configure(config)
    picam2.start()
    while(True):
     #Copy the camera image into an array
     pc2array = picam2.capture_array()
     
     #Rotate the image 180Degrees if the camera is upside down
     pc2array = np.rot90(pc2array, 2).copy()
     
     #Do the object recognition
     result, objectInfo = objectRecognition( dnn, classNames, pc2array,0.6,0.6)
     
     #Show it in a window
     cv2.imshow("Output",pc2array)
     cv2.waitKey(50)