SMART OBJECTS

OAK-D + Raspberry Pi 5

Template for Discord bots with computer vision

SVA MFA Interaction Design · Winter 2026

OVERVIEW

What's Included

👁️ Person detection (YOLO) 😴 Fatigue detection (EAR) 👀 Gaze estimation 📝 Whiteboard OCR 🤖 Discord bot commands 📢 Webhook notifications 🎯 Temporal smoothing 🖼️ Live screenshots

Pre-configured: Your instructor set up the Pis — just connect and start!

HARDWARE

Three Camera Stations

Camera	Hostname	RAM	Access
Orbit	`orbit`	16GB	SSH + VNC
Gravity	`gravity`	16GB	SSH + VNC
Horizon	`horizon`	16GB	SSH + VNC

VNC: one desktop user at a time · SSH: multiple users OK

IMPORTANT

Repo Stays on Your Laptop

Your Laptop

Clone the GitHub repo
Edit code locally
Copy files via scp

Raspberry Pi

Receives Python files
Has .env with secrets
Runs the detectors

See WORKFLOW.md for details.

CONNECTING

Three Ways to Access

Method	Best For	Command
SSH	Terminal access	`ssh orbit`
VNC	Desktop GUI	RealVNC Viewer → `orbit`
VS Code	Full dev environment	Remote-SSH extension

Recommended: VS Code Remote-SSH gives you the best editing experience.

RUNNING CODE

Activate Environment First

$ ssh orbit
$ activate-oak
$ cd ~/oak-projects

Then run any detector:

# Person detection
python3 person_detector.py --display

# Fatigue detection with DM notifications
python3 fatigue_detector.py --dm

# Gaze estimation
python3 gaze_detector.py --display --log

OPTIONS

Common Flags

Flag	What It Does
`--display`	Show live video window (VNC/X11)
`--log`	Save events to timestamped log file
`--discord`	Send notifications to class channel
`--dm`	Send personal DM notifications
`--threshold 0.7`	Set detection sensitivity (0.0–1.0)

Combine flags: python3 fatigue_detector.py --display --log --dm

PROJECT

File Structure on Pi

/opt/oak-shared/
└── venv/                    # Shared Python environment

~/oak-projects/
├── person_detector.py       # Person detection (YOLO)
├── fatigue_detector.py      # Fatigue detection (EAR)
├── gaze_detector.py         # Gaze estimation
├── whiteboard_reader.py     # OCR text detection
├── discord_bot.py           # Bot for commands
├── utils/                   # Helper modules
├── depthai_models/          # Model YAML configs
├── .env                     # Your Discord tokens
└── *.log                    # Detection logs

REFERENCE

Quick Commands

# Activate environment
activate-oak

# Check if camera is connected
python3 -c "import depthai as dai; \
  print(dai.Device.getAllAvailableDevices())"

# View logs
tail -f ~/oak-projects/*.log

# Stop a running script
Ctrl+C

VS CODE

Remote SSH Setup

Install VS Code + "Remote - SSH" extension
macOS: Grant Local Network permission in System Settings
Ctrl+Shift+P → "Remote-SSH: Connect to Host"
Select orbit (or gravity, horizon)
Open folder: /home/[username]/oak-projects
Select Python interpreter: /opt/oak-shared/venv/bin/python

DISCORD

Two Integration Options

Webhooks (Simple)

One-way notifications. Run with --discord flag.

→ discord-integration.md

Bot (Advanced)

Two-way commands: !status, !screenshot, etc.

→ DISCORD_BOT_PLAN.md

Camera bots already configured! OrbitBot, GravityBot, HorizonBot are ready to use.

EXTEND

Next Steps

Different models — Browse models.luxonis.com
Depth features — Use the OAK-D's stereo cameras
Multi-camera — Coordinate all three Pis
Cross-detector fusion — Combine fatigue + gaze + person count

See NEXT_IDEAS.md for detailed project ideas.

RESOURCES

Learn More

Quick Start	Student Quickstart Slides
Commands	Cheatsheet Slides
Versions	Working Versions
DepthAI Docs	docs.luxonis.com
GitHub Repo	kandizzy/smart-objects-cameras

SMART OBJECTS

Start Building!

Clone the repo, connect to a Pi, and run your first detector.

$ ssh orbit && activate-oak && python3 person_detector.py --display