Skip to content

TensorFlow Lite Setup and Object Detection on Raspberry Pi

Introduction

TensorFlow Lite is Google's lightweight machine learning framework optimized for mobile and edge devices like Raspberry Pi. It enables you to run pre-trained neural networks efficiently on resource-constrained hardware, making real-time object detection and image classification accessible for IoT and embedded projects.

This guide walks you through setting up TensorFlow Lite on Raspberry Pi and implementing a real-time object detection system using the Raspberry Pi Camera or USB webcam. You'll learn how to leverage pre-trained models like MobileNet SSD to detect common objects with impressive accuracy while maintaining reasonable performance on Raspberry Pi's limited resources.

Prerequisites

Before starting, ensure you have:

  • Raspberry Pi 4 (2GB+ RAM recommended) or Raspberry Pi 5
  • Raspberry Pi OS (64-bit recommended for better performance)
  • Raspberry Pi Camera Module or USB webcam
  • Internet connection for downloading packages and models
  • At least 2GB free storage space

Optional but recommended: - Coral USB Accelerator (Edge TPU) for 10x faster inference - Active cooling (heatsink or fan) for sustained performance

Installing TensorFlow Lite

TensorFlow Lite can be installed directly using Python's package manager:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# Update system packages
sudo apt update
sudo apt upgrade -y

# Install Python dependencies
sudo apt install -y python3-pip python3-opencv

# Install TensorFlow Lite runtime
pip3 install tflite-runtime

# Install additional required packages
pip3 install opencv-python pillow numpy

Method 2: Building from Source (Advanced)

For the latest features or custom optimizations:

1
2
3
4
5
6
7
8
9
# Install build dependencies
sudo apt install -y git cmake build-essential

# Clone TensorFlow repository
git clone https://github.com/tensorflow/tensorflow.git
cd tensorflow

# Build TensorFlow Lite (this takes 1-2 hours on RPi 4)
./tensorflow/lite/tools/make/build_rpi_lib.sh

Camera Setup

Enabling Raspberry Pi Camera

1
2
3
4
5
6
7
8
9
# Enable camera interface
sudo raspi-config
# Navigate to: Interface Options → Camera → Enable

# For libcamera (Raspberry Pi OS Bullseye and later)
sudo apt install -y python3-picamera2

# Test camera
libcamera-hello

USB Webcam Setup

1
2
3
4
5
6
7
8
# Install v4l-utils
sudo apt install -y v4l-utils

# List connected cameras
v4l2-ctl --list-devices

# Test webcam
ffplay /dev/video0

Downloading Pre-trained Models

TensorFlow Lite provides several optimized models for object detection:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# Create directory for models
mkdir -p ~/tensorflow_lite/models
cd ~/tensorflow_lite/models

# Download MobileNet SSD v1 (COCO dataset)
wget https://storage.googleapis.com/download.tensorflow.org/models/tflite/coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip
unzip coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip

# Download label file
wget https://storage.googleapis.com/download.tensorflow.org/models/tflite/coco_labels.txt

Available Models: - MobileNet SSD v1: Balanced speed and accuracy (~30 FPS on RPi 4) - MobileNet SSD v2: Better accuracy, slightly slower (~25 FPS) - EfficientDet-Lite: State-of-the-art accuracy for edge devices (~15 FPS)

Object Detection Implementation

Basic Object Detection Script

Create a file object_detection.py:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
#!/usr/bin/env python3
"""
Real-time object detection using TensorFlow Lite on Raspberry Pi
"""

import cv2
import numpy as np
from tflite_runtime.interpreter import Interpreter
from picamera2 import Picamera2
import time

class ObjectDetector:
    def __init__(self, model_path, label_path, threshold=0.5):
        """Initialize TensorFlow Lite interpreter and load labels"""
        self.threshold = threshold

        # Load TFLite model
        self.interpreter = Interpreter(model_path=model_path)
        self.interpreter.allocate_tensors()

        # Get input and output details
        self.input_details = self.interpreter.get_input_details()
        self.output_details = self.interpreter.get_output_details()

        # Get input shape
        self.input_shape = self.input_details[0]['shape']
        self.height = self.input_shape[1]
        self.width = self.input_shape[2]

        # Load labels
        with open(label_path, 'r') as f:
            self.labels = [line.strip() for line in f.readlines()]

    def detect_objects(self, frame):
        """Run object detection on a frame"""
        # Preprocess frame
        frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        frame_resized = cv2.resize(frame_rgb, (self.width, self.height))
        input_data = np.expand_dims(frame_resized, axis=0)

        # Run inference
        self.interpreter.set_tensor(self.input_details[0]['index'], input_data)
        self.interpreter.invoke()

        # Get results
        boxes = self.interpreter.get_tensor(self.output_details[0]['index'])[0]
        classes = self.interpreter.get_tensor(self.output_details[1]['index'])[0]
        scores = self.interpreter.get_tensor(self.output_details[2]['index'])[0]

        return boxes, classes, scores

    def draw_detections(self, frame, boxes, classes, scores):
        """Draw bounding boxes and labels on frame"""
        h, w = frame.shape[:2]

        for i in range(len(scores)):
            if scores[i] > self.threshold:
                # Get box coordinates
                ymin, xmin, ymax, xmax = boxes[i]
                left = int(xmin * w)
                top = int(ymin * h)
                right = int(xmax * w)
                bottom = int(ymax * h)

                # Get label and score
                label = self.labels[int(classes[i])]
                score = scores[i]

                # Draw box
                cv2.rectangle(frame, (left, top), (right, bottom), (0, 255, 0), 2)

                # Draw label background
                label_text = f"{label}: {score:.2f}"
                (text_width, text_height), _ = cv2.getTextSize(
                    label_text, cv2.FONT_HERSHEY_SIMPLEX, 0.5, 1
                )
                cv2.rectangle(
                    frame, (left, top - text_height - 5), 
                    (left + text_width, top), (0, 255, 0), -1
                )

                # Draw label text
                cv2.putText(
                    frame, label_text, (left, top - 5),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 0), 1
                )

        return frame

def main():
    """Main function for real-time object detection"""
    # Paths to model and labels
    MODEL_PATH = "models/detect.tflite"
    LABEL_PATH = "models/coco_labels.txt"

    # Initialize detector
    print("Loading TensorFlow Lite model...")
    detector = ObjectDetector(MODEL_PATH, LABEL_PATH, threshold=0.5)
    print("Model loaded successfully!")

    # Initialize camera
    print("Initializing camera...")
    picam2 = Picamera2()
    config = picam2.create_preview_configuration(
        main={"size": (640, 480), "format": "RGB888"}
    )
    picam2.configure(config)
    picam2.start()
    print("Camera ready!")

    # FPS calculation
    frame_count = 0
    start_time = time.time()
    fps = 0

    try:
        while True:
            # Capture frame
            frame = picam2.capture_array()
            frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)

            # Run detection
            boxes, classes, scores = detector.detect_objects(frame)

            # Draw results
            frame = detector.draw_detections(frame, boxes, classes, scores)

            # Calculate FPS
            frame_count += 1
            if frame_count % 30 == 0:
                end_time = time.time()
                fps = 30 / (end_time - start_time)
                start_time = time.time()

            # Display FPS
            cv2.putText(
                frame, f"FPS: {fps:.1f}", (10, 30),
                cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 255, 0), 2
            )

            # Show frame
            cv2.imshow("Object Detection", frame)

            # Exit on 'q' press
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break

    finally:
        picam2.stop()
        cv2.destroyAllWindows()
        print("\nDetection stopped.")

if __name__ == "__main__":
    main()

Running the Detection Script

1
2
3
4
5
# Make script executable
chmod +x object_detection.py

# Run detection
python3 object_detection.py

Expected Output:

The script will: 1. Load the TensorFlow Lite model 2. Initialize the camera 3. Display real-time video with bounding boxes around detected objects 4. Show object labels and confidence scores 5. Display FPS in the top-left corner

Press 'q' to quit the application.

Performance Optimization

1. Using Coral USB Accelerator (Edge TPU)

The Coral USB Accelerator provides 10x faster inference:

1
2
3
4
5
6
7
8
# Install Edge TPU runtime
echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo apt update
sudo apt install -y libedgetpu1-std python3-pycoral

# Download Edge TPU optimized model
wget https://github.com/google-coral/test_data/raw/master/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite

Modify the detector initialization:

1
2
3
4
5
from pycoral.utils import edgetpu
from pycoral.adapters import common

# Use Edge TPU delegate
interpreter = edgetpu.make_interpreter(model_path)

2. Model Quantization

Use quantized models (INT8) instead of floating-point for 4x speedup:

1
2
3
# Quantized models have "_quant" in filename
detect_quant.tflite  # INT8 quantized
detect.tflite        # FP32 floating-point

3. Resolution and Frame Rate Tuning

1
2
3
4
5
6
7
8
# Lower resolution = faster inference
config = picam2.create_preview_configuration(
    main={"size": (320, 240)}  # Down from 640x480
)

# Process every N frames
if frame_count % 2 == 0:  # Process every other frame
    boxes, classes, scores = detector.detect_objects(frame)

4. CPU Governor Settings

Set CPU to performance mode for consistent speed:

1
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

Use Cases and Applications

1. Smart Security Camera

Detect people entering restricted areas:

1
2
3
if 'person' in detected_labels:
    send_alert("Person detected!")
    save_snapshot(frame, timestamp)

2. Wildlife Monitoring

Log animal sightings:

1
2
3
wildlife_classes = ['bird', 'cat', 'dog', 'horse', 'sheep', 'cow']
if any(animal in detected_labels for animal in wildlife_classes):
    log_sighting(detected_labels, timestamp, location)

3. Package Delivery Notification

Detect packages at your door:

1
2
if 'suitcase' in detected_labels or 'box' in detected_labels:
    notify_owner("Package detected at door")

4. Traffic Monitoring

Count vehicles:

1
2
vehicle_classes = ['car', 'truck', 'bus', 'motorcycle']
vehicle_count = sum(1 for label in detected_labels if label in vehicle_classes)

Troubleshooting

Low FPS / Performance Issues

1
2
3
4
5
6
7
8
9
# Check CPU frequency
vcgencmd measure_clock arm

# Check temperature (throttling occurs at 80°C)
vcgencmd measure_temp

# Ensure swap is enabled
sudo dphys-swapfile swapoff
sudo dphys-swapfile swapon

Camera Not Detected

1
2
3
4
5
6
7
# Verify camera connection
vcgencmd get_camera

# Should output: supported=1 detected=1

# Check libcamera
libcamera-hello --list-cameras

Import Errors

1
2
3
4
5
6
# Reinstall TensorFlow Lite
pip3 uninstall tflite-runtime
pip3 install --upgrade tflite-runtime

# Check Python version (3.7+ required)
python3 --version

Memory Errors

1
2
3
4
5
# Increase GPU memory in /boot/firmware/config.txt
gpu_mem=256

# Close unnecessary applications
sudo systemctl stop lightdm  # Disable GUI

Conclusion

TensorFlow Lite brings powerful AI capabilities to Raspberry Pi, enabling real-time object detection for countless applications. While the base performance may not match desktop GPUs, proper optimization and hardware acceleration with Coral USB Accelerator can achieve impressive results suitable for most edge AI projects.

Key takeaways: - TensorFlow Lite provides an excellent balance of performance and accuracy on Raspberry Pi - Pre-trained models like MobileNet SSD enable quick prototyping without ML expertise - Hardware acceleration (Edge TPU) dramatically improves inference speed - Optimization techniques (quantization, resolution tuning) are crucial for real-time performance

For production deployments, consider combining this setup with other Raspberry Pi optimization techniques like CPU frequency scaling, swap optimization, and proper thermal management to ensure reliable long-term operation.

Further Reading