Skip to content

inotify - Filesystem Monitoring and Event-Driven Automation for Raspberry Pi

Introduction

inotify is a powerful Linux kernel subsystem that monitors filesystem events in real-time, enabling responsive, event-driven automation. Instead of polling directories repeatedly to check for changes, inotify notifies your application instantly when files are created, modified, deleted, or accessed.

This technology transforms how you can interact with filesystems:

  • Instant Response: React to file changes within milliseconds
  • Efficient Monitoring: No CPU-wasting polling loops
  • Scalable: Monitor thousands of files with minimal overhead
  • Versatile: Track creation, modification, deletion, access, and metadata changes
  • Kernel-Level: Built into the Linux kernel for reliability and performance

Common use cases for Raspberry Pi:

  • Automatic File Synchronization: Sync files to cloud/NAS immediately after changes
  • Hot Reload Development: Automatically restart services when code changes
  • Security Monitoring: Detect unauthorized file modifications in real-time
  • Media Processing: Auto-convert/compress photos and videos on upload
  • Backup Triggers: Incremental backups triggered by file changes
  • Log Monitoring: Alert on specific log file patterns
  • Configuration Management: Reload services when config files change
  • Document Processing: OCR/index documents as they arrive
  • Smart Home Automation: Trigger actions based on file events

This comprehensive guide covers:

  • inotify Basics: Understanding kernel events and watch descriptors
  • Command-Line Tools: inotifywait and inotifywatch for quick automation
  • Python Programming: Building robust file monitoring applications
  • Bash Scripting: Shell-based automation with inotify-tools
  • Advanced Patterns: Recursive monitoring, filtering, and event batching
  • Production Deployment: Systemd integration and resource management
  • Performance Tuning: Optimizing inotify for Raspberry Pi
  • Real-World Projects: Cloud sync, hot reload, security monitoring, media processing

Perfect for:

  • DevOps Engineers: Automated deployment and configuration management
  • System Administrators: File integrity monitoring and log analysis
  • Developers: Hot reload workflows and build automation
  • Media Enthusiasts: Automatic photo/video processing pipelines
  • Security Professionals: Intrusion detection and audit logging
  • Backup Administrators: Event-driven incremental backups
  • IoT Developers: Responsive file-based integrations
  • Home Automation: Smart home file-triggered actions

Understanding inotify

How inotify Works

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
┌─────────────────────────────────────────────────────────┐
│                    User Application                      │
│                                                          │
│  1. Create inotify instance                             │
│  2. Add watches to files/directories                    │
│  3. Read events from inotify file descriptor            │
│  4. Process events and take action                      │
└────────────────────┬────────────────────────────────────┘
                     │ syscalls: inotify_init, inotify_add_watch, read
                     ▼
┌─────────────────────────────────────────────────────────┐
│                   Linux Kernel                          │
│                                                          │
│  ┌──────────────────────────────────────────┐           │
│  │         inotify Subsystem                │           │
│  │                                          │           │
│  │  Watch List:                             │           │
│  │    Watch 1: /var/log/syslog              │           │
│  │    Watch 2: /home/pi/uploads/            │           │
│  │    Watch 3: /etc/nginx/nginx.conf        │           │
│  └──────────────────┬───────────────────────┘           │
│                     │                                    │
│                     │ Event notifications                │
│                     ▼                                    │
│  ┌──────────────────────────────────────────┐           │
│  │       Filesystem Layer (VFS)             │           │
│  │                                          │           │
│  │  File operations trigger events:         │           │
│  │    - open(), close()                     │           │
│  │    - read(), write()                     │           │
│  │    - create(), delete()                  │           │
│  │    - rename(), chmod()                   │           │
│  └──────────────────┬───────────────────────┘           │
└────────────────────┼────────────────────────────────────┘
                     │
                     ▼
              Actual Filesystem
           (ext4, btrfs, tmpfs, etc.)

inotify Events

Event Description Use Case
IN_ACCESS File was read Log access tracking
IN_MODIFY File content changed Trigger reprocessing
IN_ATTRIB Metadata changed (permissions, timestamps) Security monitoring
IN_CLOSE_WRITE File opened for writing was closed Safe file processing
IN_CLOSE_NOWRITE File opened read-only was closed Read tracking
IN_OPEN File was opened Access logging
IN_MOVED_FROM File moved out of watched directory Track relocations
IN_MOVED_TO File moved into watched directory Process new arrivals
IN_CREATE File/directory created Process new files
IN_DELETE File/directory deleted Cleanup actions
IN_DELETE_SELF Watched file/directory deleted Remove watch
IN_MOVE_SELF Watched file/directory moved Update watch

Key Concepts

Watch Descriptor (wd): - Unique integer identifying each watch - Returned when you add a watch - Used to identify which file/directory triggered event

Event Queue: - Kernel buffers events in queue - Application reads events from queue - Queue can overflow if not read fast enough

Limitations: - inotify watches files/directories, not paths - Moving watched file removes watch - Only monitors one level (not recursive by default) - Limited number of watches per user (configurable)

Installing inotify-tools

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# Update package list
sudo apt update

# Install inotify-tools (provides inotifywait and inotifywatch)
sudo apt install -y inotify-tools

# Verify installation
inotifywait --version
# Should show: inotifywait 3.x.x

inotifywatch --version
# Should show: inotifywatch 3.x.x

# Check current limits
cat /proc/sys/fs/inotify/max_user_watches
# Default: 8192 (can be increased)

cat /proc/sys/fs/inotify/max_user_instances
# Default: 128

cat /proc/sys/fs/inotify/max_queued_events
# Default: 16384

Increasing inotify Limits

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# For systems monitoring many files (e.g., IDE, sync tools)

# Temporarily increase limits
sudo sysctl fs.inotify.max_user_watches=524288
sudo sysctl fs.inotify.max_user_instances=256
sudo sysctl fs.inotify.max_queued_events=32768

# Make permanent
sudo tee -a /etc/sysctl.conf <<EOF
fs.inotify.max_user_watches=524288
fs.inotify.max_user_instances=256
fs.inotify.max_queued_events=32768
EOF

# Apply changes
sudo sysctl -p

inotifywait: Real-Time File Monitoring

Basic Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# Watch a single file
inotifywait /var/log/syslog

# Output when file changes:
# /var/log/syslog MODIFY

# Watch a directory
inotifywait /home/pi/uploads/

# Output when file created:
# /home/pi/uploads/ CREATE newfile.txt

# Monitor continuously (loop after each event)
inotifywait -m /home/pi/uploads/

# This runs indefinitely, reporting each event

Event Filtering

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Watch specific events only
inotifywait -m -e create /home/pi/uploads/
# Only reports file creation

# Multiple events
inotifywait -m -e create,modify,delete /home/pi/uploads/

# All write-related events
inotifywait -m -e close_write /home/pi/uploads/
# Triggers when file writing is complete (safe to process)

# Exclude events
inotifywait -m --exclude '.*\.tmp$' /home/pi/uploads/
# Ignore .tmp files

Output Formatting

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# Custom format
inotifywait -m --format '%T %w %f %e' --timefmt '%Y-%m-%d %H:%M:%S' /home/pi/uploads/

# Output example:
# 2025-11-23 14:30:45 /home/pi/uploads/ newfile.txt CREATE
# 2025-11-23 14:30:46 /home/pi/uploads/ newfile.txt MODIFY
# 2025-11-23 14:30:47 /home/pi/uploads/ newfile.txt CLOSE_WRITE,CLOSE

# Format specifiers:
# %w - Watched directory
# %f - Filename
# %e - Event name
# %T - Time (with --timefmt)

# CSV format for logging
inotifywait -m --csv --format '%w,%f,%e,%T' --timefmt '%s' /home/pi/uploads/

Recursive Monitoring

1
2
3
4
5
6
7
8
9
# Monitor directory and all subdirectories
inotifywait -m -r /home/pi/projects/

# Recursively with specific events
inotifywait -m -r -e modify /home/pi/projects/

# Exclude patterns
inotifywait -m -r --exclude '(\.git|node_modules|__pycache__)' /home/pi/projects/
# Ignore version control and build directories

Practical Automation Scripts

Auto-Sync to Cloud Storage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#!/bin/bash
# auto-sync-rclone.sh - Automatically sync changed files to cloud

WATCH_DIR="/home/pi/Documents"
REMOTE="gdrive:Backup/Documents"
LOG_FILE="/var/log/auto-sync.log"

log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}

log "Starting auto-sync for $WATCH_DIR -> $REMOTE"

inotifywait -m -r -e close_write,moved_to,create "$WATCH_DIR" --format '%w%f' |
while read -r CHANGED_FILE; do
    # Wait a moment to let file finish writing
    sleep 1

    # Calculate relative path
    REL_PATH="${CHANGED_FILE#$WATCH_DIR/}"

    log "Change detected: $REL_PATH"

    # Sync the specific file
    if rclone copy "$CHANGED_FILE" "$REMOTE/$(dirname "$REL_PATH")" -v >> "$LOG_FILE" 2>&1; then
        log "✓ Synced: $REL_PATH"
    else
        log "✗ Failed to sync: $REL_PATH"
    fi
done

Run as systemd service: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# /etc/systemd/system/auto-sync.service
[Unit]
Description=Auto-sync files to cloud storage
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=pi
ExecStart=/usr/local/bin/auto-sync-rclone.sh
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Hot Reload for Development

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
#!/bin/bash
# hot-reload-webserver.sh - Restart web server when files change

WATCH_DIR="/home/pi/website"
SERVICE_NAME="nginx"

echo "Watching $WATCH_DIR for changes..."
echo "Will reload $SERVICE_NAME on modifications"

inotifywait -m -r -e close_write --format '%w%f' \
    --exclude '(\.git|\.swp|\.tmp)' \
    "$WATCH_DIR" |
while read -r CHANGED_FILE; do
    echo "[$(date '+%H:%M:%S')] Changed: $CHANGED_FILE"

    # Debounce - wait for multiple rapid changes
    sleep 0.5

    # Test configuration before reloading
    if sudo nginx -t 2>/dev/null; then
        echo "Configuration valid, reloading..."
        sudo systemctl reload nginx
        echo "✓ Reloaded $SERVICE_NAME"
    else
        echo "✗ Configuration error, not reloading"
        sudo nginx -t
    fi
done

Automatic Media Processing

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
#!/bin/bash
# auto-process-photos.sh - Convert and optimize photos on upload

WATCH_DIR="/home/pi/camera-uploads"
OUTPUT_DIR="/home/pi/processed-photos"
ARCHIVE_DIR="/home/pi/archive"

mkdir -p "$OUTPUT_DIR" "$ARCHIVE_DIR"

echo "Watching for new photos in $WATCH_DIR"

inotifywait -m -e close_write --format '%w%f' \
    "$WATCH_DIR" |
while read -r FILE; do
    # Only process image files
    if [[ $FILE =~ \.(jpg|jpeg|png|raw)$ ]]; then
        FILENAME=$(basename "$FILE")
        BASENAME="${FILENAME%.*}"

        echo "Processing: $FILENAME"

        # 1. Resize to 1920px wide (preserve aspect ratio)
        convert "$FILE" -resize 1920x\> "$OUTPUT_DIR/${BASENAME}_resized.jpg"

        # 2. Create thumbnail (300px wide)
        convert "$FILE" -resize 300x\> "$OUTPUT_DIR/${BASENAME}_thumb.jpg"

        # 3. Extract EXIF data
        exiftool "$FILE" > "$OUTPUT_DIR/${BASENAME}_exif.txt"

        # 4. Optimize JPEGs
        jpegoptim --max=85 --strip-all "$OUTPUT_DIR/${BASENAME}_resized.jpg"

        # 5. Move original to archive
        mv "$FILE" "$ARCHIVE_DIR/"

        echo "✓ Processed: $FILENAME"
        echo "  - Resized: ${BASENAME}_resized.jpg"
        echo "  - Thumbnail: ${BASENAME}_thumb.jpg"
        echo "  - Archived: $ARCHIVE_DIR/$FILENAME"
    fi
done

Security File Integrity Monitor

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
#!/bin/bash
# file-integrity-monitor.sh - Detect unauthorized file changes

WATCH_DIRS=(
    "/etc"
    "/usr/bin"
    "/usr/sbin"
    "/boot"
    "/root"
)

LOG_FILE="/var/log/file-integrity.log"
ALERT_EMAIL="admin@example.com"

log_alert() {
    local SEVERITY=$1
    local MESSAGE=$2
    local TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')

    echo "[$TIMESTAMP] [$SEVERITY] $MESSAGE" | tee -a "$LOG_FILE"

    # Send email alert for critical events
    if [ "$SEVERITY" == "CRITICAL" ]; then
        echo "$MESSAGE" | mail -s "CRITICAL: File Integrity Alert" "$ALERT_EMAIL"
    fi
}

# Calculate file hash
get_hash() {
    sha256sum "$1" 2>/dev/null | awk '{print $1}'
}

log_alert "INFO" "File integrity monitoring started"

# Monitor all specified directories
inotifywait -m -r -e modify,attrib,delete,create,move \
    --format '%e %w%f' \
    "${WATCH_DIRS[@]}" |
while read -r EVENT FILE; do
    case $EVENT in
        MODIFY)
            HASH=$(get_hash "$FILE")
            log_alert "WARNING" "File modified: $FILE (SHA256: $HASH)"
            ;;
        ATTRIB)
            PERMS=$(stat -c '%a' "$FILE" 2>/dev/null)
            OWNER=$(stat -c '%U:%G' "$FILE" 2>/dev/null)
            log_alert "WARNING" "Attributes changed: $FILE (Perms: $PERMS, Owner: $OWNER)"
            ;;
        DELETE)
            log_alert "CRITICAL" "File deleted: $FILE"
            ;;
        CREATE)
            log_alert "WARNING" "File created: $FILE"
            ;;
        MOVED_TO|MOVED_FROM)
            log_alert "WARNING" "File moved: $FILE"
            ;;
    esac
done

Automatic Backup Trigger

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
#!/bin/bash
# incremental-backup-trigger.sh - Trigger backups on file changes

WATCH_DIR="/home/pi/important-data"
BACKUP_SCRIPT="/usr/local/bin/backup.sh"
MIN_INTERVAL=300  # Minimum 5 minutes between backups

LAST_BACKUP=0

echo "Monitoring $WATCH_DIR for changes"
echo "Will trigger backup after modifications (max every $MIN_INTERVAL seconds)"

inotifywait -m -r -e close_write,delete,moved_to \
    --exclude '(\.tmp|\.swp|~)$' \
    "$WATCH_DIR" |
while read -r DIR EVENT FILE; do
    CURRENT_TIME=$(date +%s)
    TIME_SINCE_LAST=$((CURRENT_TIME - LAST_BACKUP))

    if [ $TIME_SINCE_LAST -ge $MIN_INTERVAL ]; then
        echo "[$(date '+%Y-%m-%d %H:%M:%S')] Change detected: $FILE ($EVENT)"
        echo "Triggering backup..."

        if $BACKUP_SCRIPT; then
            LAST_BACKUP=$CURRENT_TIME
            echo "✓ Backup completed"
        else
            echo "✗ Backup failed"
        fi
    else
        REMAINING=$((MIN_INTERVAL - TIME_SINCE_LAST))
        echo "Change detected but waiting ${REMAINING}s before next backup"
    fi
done

Python Programming with inotify

Using pyinotify Library

1
2
# Install pyinotify
pip3 install pyinotify

Basic Example: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
#!/usr/bin/env python3
# basic-file-monitor.py

import pyinotify

class EventHandler(pyinotify.ProcessEvent):
    def process_IN_CREATE(self, event):
        print(f"Created: {event.pathname}")

    def process_IN_DELETE(self, event):
        print(f"Deleted: {event.pathname}")

    def process_IN_MODIFY(self, event):
        print(f"Modified: {event.pathname}")

    def process_IN_CLOSE_WRITE(self, event):
        print(f"File closed after writing: {event.pathname}")

# Watch manager
wm = pyinotify.WatchManager()

# Event mask
mask = pyinotify.IN_CREATE | pyinotify.IN_DELETE | pyinotify.IN_MODIFY | pyinotify.IN_CLOSE_WRITE

# Event handler
handler = EventHandler()

# Notifier
notifier = pyinotify.Notifier(wm, handler)

# Add watch
wm.add_watch('/home/pi/watched', mask, rec=True, auto_add=True)

print("Monitoring /home/pi/watched (Ctrl+C to stop)")

# Start monitoring
notifier.loop()

Advanced: Multi-Directory Monitor with Filtering

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
#!/usr/bin/env python3
# advanced-monitor.py

import pyinotify
import os
import logging
from pathlib import Path

# Setup logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('/var/log/file-monitor.log'),
        logging.StreamHandler()
    ]
)

class SmartFileHandler(pyinotify.ProcessEvent):
    def __init__(self, config):
        self.config = config
        self.ignored_extensions = config.get('ignored_extensions', [])
        self.ignored_patterns = config.get('ignored_patterns', [])

    def should_ignore(self, path):
        """Check if file should be ignored"""
        # Ignore hidden files
        if os.path.basename(path).startswith('.'):
            return True

        # Ignore by extension
        for ext in self.ignored_extensions:
            if path.endswith(ext):
                return True

        # Ignore by pattern
        for pattern in self.ignored_patterns:
            if pattern in path:
                return True

        return False

    def process_IN_CLOSE_WRITE(self, event):
        if self.should_ignore(event.pathname):
            return

        logging.info(f"File saved: {event.pathname}")

        # Trigger custom actions based on file type
        if event.pathname.endswith('.py'):
            self.handle_python_file(event.pathname)
        elif event.pathname.endswith(('.jpg', '.png', '.gif')):
            self.handle_image_file(event.pathname)
        elif event.pathname.endswith('.log'):
            self.handle_log_file(event.pathname)

    def process_IN_CREATE(self, event):
        if event.is_dir:
            logging.info(f"Directory created: {event.pathname}")
        elif not self.should_ignore(event.pathname):
            logging.info(f"File created: {event.pathname}")

    def process_IN_DELETE(self, event):
        if not event.is_dir and not self.should_ignore(event.pathname):
            logging.warning(f"File deleted: {event.pathname}")

    def handle_python_file(self, path):
        """Handle Python file changes"""
        logging.info(f"Python file detected: {path}")
        # Could run linter, tests, etc.
        # os.system(f"pylint {path}")

    def handle_image_file(self, path):
        """Handle image file changes"""
        logging.info(f"Image file detected: {path}")
        # Could trigger image processing
        # subprocess.run(['convert', path, '-resize', '1920x', output])

    def handle_log_file(self, path):
        """Handle log file changes"""
        # Could parse logs, send alerts, etc.
        pass

# Configuration
config = {
    'watched_directories': [
        '/home/pi/projects',
        '/var/log/apps',
        '/home/pi/uploads'
    ],
    'ignored_extensions': ['.tmp', '.swp', '.pyc', '~'],
    'ignored_patterns': ['__pycache__', '.git', 'node_modules']
}

# Setup inotify
wm = pyinotify.WatchManager()
handler = SmartFileHandler(config)
notifier = pyinotify.Notifier(wm, handler)

# Event mask
mask = (pyinotify.IN_CLOSE_WRITE | 
        pyinotify.IN_CREATE | 
        pyinotify.IN_DELETE)

# Add watches
for directory in config['watched_directories']:
    if os.path.exists(directory):
        wm.add_watch(directory, mask, rec=True, auto_add=True)
        logging.info(f"Watching: {directory}")
    else:
        logging.warning(f"Directory not found: {directory}")

logging.info("File monitoring started")

try:
    notifier.loop()
except KeyboardInterrupt:
    logging.info("File monitoring stopped")

Production-Ready Monitor with Error Handling

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
#!/usr/bin/env python3
# production-file-monitor.py

import pyinotify
import sys
import signal
import logging
import queue
import threading
from datetime import datetime

class FileEventQueue:
    """Thread-safe event queue for batch processing"""
    def __init__(self, batch_size=10, batch_timeout=5):
        self.queue = queue.Queue()
        self.batch_size = batch_size
        self.batch_timeout = batch_timeout
        self.running = True

    def add_event(self, event):
        self.queue.put(event)

    def process_worker(self):
        """Background worker to process events in batches"""
        batch = []

        while self.running:
            try:
                # Wait for event with timeout
                event = self.queue.get(timeout=self.batch_timeout)
                batch.append(event)

                # Process when batch is full
                if len(batch) >= self.batch_size:
                    self.process_batch(batch)
                    batch = []

            except queue.Empty:
                # Timeout - process accumulated events
                if batch:
                    self.process_batch(batch)
                    batch = []

    def process_batch(self, events):
        """Process a batch of events"""
        logging.info(f"Processing batch of {len(events)} events")

        for event in events:
            logging.debug(f"  - {event['type']}: {event['path']}")

        # Here you could:
        # - Batch upload to cloud
        # - Aggregate statistics
        # - Send batch notifications
        # - Update database in one transaction

    def stop(self):
        self.running = False

class RobustEventHandler(pyinotify.ProcessEvent):
    def __init__(self, event_queue):
        self.event_queue = event_queue
        self.stats = {
            'created': 0,
            'modified': 0,
            'deleted': 0,
            'errors': 0
        }

    def process_default(self, event):
        """Catch-all for all events"""
        try:
            event_data = {
                'type': event.maskname,
                'path': event.pathname,
                'is_dir': event.dir,
                'timestamp': datetime.now().isoformat()
            }

            self.event_queue.add_event(event_data)

            # Update statistics
            if 'CREATE' in event.maskname:
                self.stats['created'] += 1
            elif 'MODIFY' in event.maskname or 'CLOSE_WRITE' in event.maskname:
                self.stats['modified'] += 1
            elif 'DELETE' in event.maskname:
                self.stats['deleted'] += 1

        except Exception as e:
            logging.error(f"Error processing event: {e}")
            self.stats['errors'] += 1

    def get_stats(self):
        return self.stats.copy()

# Setup
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)

# Event queue
event_queue = FileEventQueue(batch_size=10, batch_timeout=5)

# Start processing worker thread
worker_thread = threading.Thread(target=event_queue.process_worker, daemon=True)
worker_thread.start()

# inotify setup
wm = pyinotify.WatchManager()
handler = RobustEventHandler(event_queue)
notifier = pyinotify.Notifier(wm, handler)

# Graceful shutdown
def signal_handler(signum, frame):
    logging.info("Shutting down...")
    event_queue.stop()
    notifier.stop()

    stats = handler.get_stats()
    logging.info(f"Final statistics: {stats}")
    sys.exit(0)

signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)

# Add watches
mask = pyinotify.ALL_EVENTS
wm.add_watch('/home/pi/monitored', mask, rec=True, auto_add=True)

logging.info("Production file monitor started")

# Main loop
notifier.loop()

inotifywatch: Statistics Collection

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# Collect statistics on file access
inotifywatch -v -t 60 -r /home/pi/projects

# Output after 60 seconds:
# Establishing watches...
# Finished establishing watches, now collecting statistics.
# 
# total  modify  close_write  create  delete  filename
# 45     23      15           5       2       /home/pi/projects

# Monitor specific events
inotifywatch -e modify,delete -t 30 /var/log/

# CSV output
inotifywatch -c -t 60 /home/pi/

Performance Optimization

Efficient Recursive Monitoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
#!/bin/bash
# efficient-recursive-watch.sh

WATCH_ROOT="/home/pi/large-directory"

# Problem: Adding thousands of watches can be slow and memory-intensive
# Solution: Use excludes to skip unnecessary directories

inotifywait -m -r \
    --exclude '(\.git|node_modules|__pycache__|\.cache|build|dist|\.venv)' \
    --format '%w%f %e' \
    "$WATCH_ROOT"

Event Batching to Reduce CPU

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
#!/bin/bash
# batch-processing.sh - Process events in batches

WATCH_DIR="/home/pi/uploads"
BATCH_SIZE=10
BATCH_TIMEOUT=5

declare -a EVENT_BATCH=()

process_batch() {
    # shellcheck disable=SC2128
    if [ "$EVENT_BATCH" = "" ]; then
        return
    fi

    # shellcheck disable=SC2128
    echo "Processing batch of files"

    # Process all files at once (more efficient)
    for file in "${EVENT_BATCH[@]}"; do
        echo "  - $file"
        # Your processing here
    done

    # Clear batch
    EVENT_BATCH=()
}

# Timeout handler
trap 'process_batch' ALRM

inotifywait -m -e close_write --format '%w%f' "$WATCH_DIR" |
while read -r FILE; do
    EVENT_BATCH+=("$FILE")

    # Process when batch is full (check array length differently)
    if [ "${EVENT_BATCH[*]}" != "" ]; then
        COUNT=0
        for _ in "${EVENT_BATCH[@]}"; do
            COUNT=$((COUNT + 1))
        done

        if [ $COUNT -ge $BATCH_SIZE ]; then
            process_batch
        fi
    fi

    # Also set timeout to process smaller batches
    ( sleep $BATCH_TIMEOUT && kill -ALRM $$ ) &
done

Memory-Efficient Large Directory Monitoring

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#!/usr/bin/env python3
# memory-efficient-monitor.py

import pyinotify
import os

class MemoryEfficientHandler(pyinotify.ProcessEvent):
    """Process events immediately without buffering"""

    def process_default(self, event):
        # Process immediately and discard
        print(f"{event.maskname}: {event.pathname}")

        # Don't store event data
        # Don't accumulate statistics (unless essential)

# Use minimal event mask
mask = pyinotify.IN_CLOSE_WRITE  # Only what you need

wm = pyinotify.WatchManager()
handler = MemoryEfficientHandler()

# Smaller event queue
notifier = pyinotify.Notifier(wm, handler, default_proc_fun=handler, max_queued_events=1024)

wm.add_watch('/path/to/large/directory', mask, rec=True, auto_add=True)
notifier.loop()

Troubleshooting

Check Current Watches

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# List all watches for all processes
sudo cat /proc/*/fd/* 2>/dev/null | grep inotify | wc -l

# Find processes using inotify
for pid in /proc/[0-9]*; do
    if grep -q inotify "$pid/fd/"* 2>/dev/null; then
        echo "PID $(basename $pid): $(cat $pid/comm 2>/dev/null)"
        ls -l $pid/fd/ 2>/dev/null | grep inotify | wc -l
    fi
done

# Check limit reached
dmesg | grep -i inotify

"Too many open files" Error

1
2
3
4
5
6
7
8
9
# Increase limits temporarily
sudo sysctl fs.inotify.max_user_watches=524288

# Make permanent
echo "fs.inotify.max_user_watches=524288" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

# Check current usage
find /proc/*/fd -lname anon_inode:inotify 2>/dev/null | wc -l

Events Not Triggering

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Test if inotify works on filesystem
cd /tmp
touch test.txt
inotifywait -e modify test.txt &
echo "test" >> test.txt
# Should show event

# Some filesystems don't support inotify:
# - NFS (older versions)
# - FUSE (some implementations)
# - Network mounts (SMB, CIFS)

# Check filesystem type
df -T /path/to/watch

High CPU Usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Reduce event frequency
# Use IN_CLOSE_WRITE instead of IN_MODIFY
# (IN_MODIFY fires on every write(), IN_CLOSE_WRITE only when file closed)

# Bad (too many events):
inotifywait -m -e modify /var/log/syslog

# Good (fewer events):
inotifywait -m -e close_write /var/log/syslog

# Add exclusions
inotifywait -m --exclude '\.tmp$' /path

# Use event batching in scripts

Real-World Project: Cloud Photo Sync

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
#!/bin/bash
# cloud-photo-sync.sh - Complete photo management pipeline

set -euo pipefail

# Configuration
CAMERA_IMPORT="/home/pi/camera-import"
PROCESSED_DIR="/home/pi/photos/processed"
CLOUD_REMOTE="gdrive:Photos"
ARCHIVE_DIR="/mnt/nas/photo-archive"
LOG_FILE="/var/log/photo-sync.log"

# Ensure directories exist
mkdir -p "$CAMERA_IMPORT" "$PROCESSED_DIR" "$ARCHIVE_DIR"

log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
}

process_photo() {
    local SRC=$1
    local FILENAME=$(basename "$SRC")
    local BASENAME="${FILENAME%.*}"
    local EXT="${FILENAME##*.}"

    # Extract date from EXIF
    local PHOTO_DATE=$(exiftool -DateTimeOriginal -d "%Y-%m-%d" "$SRC" 2>/dev/null | awk -F': ' '{print $2}')

    if [ -z "$PHOTO_DATE" ]; then
        PHOTO_DATE=$(date -r "$SRC" '+%Y-%m-%d')
    fi

    # Create date-based directory
    local YEAR=$(echo $PHOTO_DATE | cut -d'-' -f1)
    local MONTH=$(echo $PHOTO_DATE | cut -d'-' -f2)
    local DEST_DIR="$PROCESSED_DIR/$YEAR/$MONTH"

    mkdir -p "$DEST_DIR"

    # Process image
    log "Processing: $FILENAME (Date: $PHOTO_DATE)"

    # 1. Copy original
    cp "$SRC" "$DEST_DIR/"

    # 2. Create web-optimized version
    convert "$SRC" \
        -auto-orient \
        -resize 2048x2048\> \
        -quality 85 \
        -strip \
        "$DEST_DIR/${BASENAME}_web.jpg"

    # 3. Create thumbnail
    convert "$SRC" \
        -auto-orient \
        -resize 400x400^ \
        -gravity center \
        -extent 400x400 \
        -quality 80 \
        "$DEST_DIR/${BASENAME}_thumb.jpg"

    # 4. Upload to cloud
    log "Uploading to cloud: $YEAR/$MONTH/$FILENAME"
    rclone copy "$DEST_DIR/" "$CLOUD_REMOTE/$YEAR/$MONTH/" -v --transfers 1

    # 5. Archive original to NAS
    ARCHIVE_PATH="$ARCHIVE_DIR/$YEAR/$MONTH"
    mkdir -p "$ARCHIVE_PATH"
    mv "$SRC" "$ARCHIVE_PATH/"

    log "✓ Completed: $FILENAME"
}

# Start monitoring
log "Photo sync service started"
log "Watching: $CAMERA_IMPORT"

inotifywait -m -e close_write \
    --format '%w%f' \
    "$CAMERA_IMPORT" |
while read -r PHOTO; do
    # Only process image files
    if [[ $PHOTO =~ \.(jpg|jpeg|png|raw|cr2|nef|arw)$ ]]; then
        # Process in background to avoid blocking
        process_photo "$PHOTO" &
    else
        log "Ignored non-image file: $(basename $PHOTO)"
    fi
done

Systemd service: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# /etc/systemd/system/photo-sync.service
[Unit]
Description=Automatic Photo Processing and Cloud Sync
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=pi
Environment="PATH=/usr/local/bin:/usr/bin:/bin"
ExecStart=/usr/local/bin/cloud-photo-sync.sh
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Summary

This comprehensive guide covered inotify filesystem monitoring on Raspberry Pi:

✅ Core Concepts

  • inotify kernel subsystem architecture
  • Event types and watch descriptors
  • Event queue management
  • Filesystem limitations and considerations

✅ Command-Line Tools

  • inotifywait: Real-time event monitoring with filtering
  • inotifywatch: Statistical analysis of file access
  • Custom output formatting and CSV export
  • Recursive monitoring with exclusions

✅ Practical Automation

  • Auto-sync: Cloud storage synchronization on file changes
  • Hot reload: Development server restart on code changes
  • Media processing: Automatic photo/video conversion pipeline
  • Security monitoring: File integrity detection
  • Backup triggers: Event-driven incremental backups

✅ Python Programming

  • pyinotify library integration
  • Advanced filtering and event handling
  • Multi-directory monitoring
  • Thread-safe event queuing
  • Production-ready error handling

✅ Performance Optimization

  • Efficient recursive monitoring with exclusions
  • Event batching to reduce CPU usage
  • Memory-efficient large directory handling
  • Smart event filtering (IN_CLOSE_WRITE vs IN_MODIFY)

✅ Troubleshooting

  • Watch limit management (max_user_watches)
  • Filesystem compatibility checks
  • CPU usage optimization
  • Event debugging techniques

✅ Production Deployment

  • Systemd service integration
  • Logging and error handling
  • Graceful shutdown handling
  • Resource monitoring

✅ Real-World Projects

  • Complete cloud photo sync pipeline with EXIF processing
  • Multi-stage media conversion workflows
  • Security monitoring with alerting
  • Automated backup systems

Next Steps

Advanced Topics:

  1. fanotify: More advanced filesystem monitoring (requires root)
  2. auditd Integration: Combine with Linux audit framework
  3. eBPF Filesystem Monitoring: Modern kernel-level tracing
  4. Distributed Monitoring: Aggregate events across multiple Pis
  5. AI-Powered Classification: Use ML to auto-categorize files

Related Guides:

With inotify, your Raspberry Pi can respond instantly to filesystem changes, enabling sophisticated automation workflows that would be impossible with traditional polling approaches!