SMART OBJECTS: THE SMART CLASSROOM

Student Quick Start

OAK-D Cameras · Raspberry Pi · Discord Bots

SVA MFA Interaction Design · Winter 2026
~20–30 min first-time setup

OVERVIEW

6-Step Setup Checklist

1 Clone the GitHub repo to your laptop

2 Set up SSH access to your Pi

3 Copy Python files from laptop to Pi (scp)

4 Check / create your .env file on the Pi

5 Run your first detector!

6 Optional: Set up personal DM bot

Recommended starting points: Fatigue Detection or Gaze Estimation

MENTAL MODEL

Where Am I? IMPORTANT

Your Laptop LOCAL

yourname@MacBook-Pro ~ %

GitHub repo lives here.
Edit code here.

Raspberry Pi REMOTE

yourname@orbit ~ $

Camera + venv live here.
Run scripts here.

Pro tip: Type hostname and press Enter to see which machine you're on.

Full details →

MENTAL MODEL

The Workflow

Edit code
on your laptop

→

scp to Pi
copy changed files

→

Run on Pi
python3 script.py

Bottom line: Edit on laptop → copy to Pi → run on Pi. Repeat.

Full details →

MENTAL MODEL

What's Done vs. What You Do

Already configured on each Pi

Camera bot tokens (OrbitBot, etc.)
Discord webhooks
Shared Python venv at /opt/oak-shared/venv/
All required libraries installed

What you need to do

› Copy Python files from laptop → Pi
› (Optional) Add your DM bot token to .env
› Run the scripts!

SETUP

Step 2: SSH Into a Pi LOCAL

Test your SSH config from your local terminal:

$ ssh orbit # or: ssh gravity | ssh horizon

If it connects, you're good! If not:

"Host not found" — Your SSH config needs the Pi IP addresses (10.1.x.x). See the main README or ask your instructor.

Full details →

SETUP

Step 3: Copy Files to Pi LOCAL

From your laptop terminal (not the Pi!):

# Navigate to your local repo
cd ~/path/to/smart-objects-cameras

# Fatigue detection (recommended)
scp fatigue_detector.py orbit:~/oak-projects/
scp -r utils orbit:~/oak-projects/
scp -r depthai_models orbit:~/oak-projects/
scp discord_notifier.py orbit:~/oak-projects/
scp discord_dm_notifier.py orbit:~/oak-projects/

Replace orbit with gravity or horizon as needed.

Full details →

SETUP

Alt: VS Code Remote SSH

If you have VS Code with the Remote-SSH extension:

1 Connect to the Pi via Remote SSH

2 Open your local repo in a separate VS Code window

3 Drag & drop files from local to remote window

Advantage: No scp commands needed. You can also edit files directly on the Pi.

SETUP

Step 4: The .env File ON THE PI

SSH in, then check if .env already exists:

$ ls ~/oak-projects/.env

If it exists

Camera bot tokens are set. Optionally add your DM bot token:

nano ~/oak-projects/.env
# Add at the bottom:
DISCORD_USER_ID=your_id
DISCORD_DM_BOT_TOKEN=your_token

If it doesn't exist

Your instructor will share tokens via 1Password. Create the file:

nano ~/oak-projects/.env
# Paste tokens from instructor
chmod 600 ~/oak-projects/.env

Never commit .env to GitHub!

Full details →

SETUP

Step 5: Activate & Run ON THE PI

Every time you start working on the Pi:

# Activate the shared virtual environment
activate-oak

# Navigate to your project folder
cd ~/oak-projects

# Run a detector!
python3 fatigue_detector.py --display

You should see (venv) appear before your prompt after activate-oak.

PYTHON FLAGS

Flag Reference

Flags are optional settings that change how scripts run:

Flag	What It Does	Example
`--display`	Show live video window	`python3 gaze_detector.py --display`
`--log`	Save events to a log file	`python3 fatigue_detector.py --log`
`--discord`	Post to Discord class channel	`python3 person_detector*.py --discord`
`--threshold`	Detection sensitivity (0.0–1.0)	`... --threshold 0.7`

Combine them: python3 fatigue_detector.py --display --log

Full details →

PYTHON FLAGS

When to Use `--display`

Use --display when:

Connected via VNC
Using a physical monitor
At the Pi's desktop

Don't use when:

SSH'd from terminal only
Running remotely without VNC

Instead: run without --display, use Discord or logs to monitor.

RUNNING DETECTORS

Fatigue Detection RECOMMENDED

Monitors eye aspect ratio (EAR) to detect drowsiness. Uses your personal DM bot.

With display (VNC/monitor):

activate-oak
cd ~/oak-projects
python3 fatigue_detector.py --display

Press 'q' to quit.

Without display (SSH only):

activate-oak
cd ~/oak-projects
python3 fatigue_detector.py

Press Ctrl+C to stop. Check Discord DMs.

Full details →

RUNNING DETECTORS

Gaze Estimation RECOMMENDED

Detects where someone is looking. Outputs gaze direction to the terminal.

activate-oak
cd ~/oak-projects
python3 gaze_detector.py --display        # with video window
python3 gaze_detector.py                   # terminal only

Bonus challenge: Use Claude Code to add Discord notifications to gaze detection! What gaze info is worth sending? How do you avoid spamming?

Full details →

RUNNING DETECTORS

Person Detection (optional)

Uses the class camera bots (OrbitBot, GravityBot, HorizonBot) — not your personal DM bot.

activate-oak && cd ~/oak-projects

python3 person_detector_with_display.py --display   # with video
python3 person_detector_with_display.py             # headless
python3 person_detector_with_display.py --discord   # + class channel

Bot commands in Discord: !ping !status !screenshot !detect

Full details →

RUNNING DETECTORS

Quick Reference

What You Want	Window 1	Window 2
Fatigue + personal DMs	`python3 fatigue_detector.py --display`	`python3 discord_dm_notifier.py`
Gaze estimation	`python3 gaze_detector.py --display`	not needed
Person + personal DMs	`python3 person_detector*.py`	`python3 discord_dm_notifier.py`
Person + class channel	`python3 person_detector*.py --discord`	not needed

MULTIPLE WINDOWS

Why Two Terminal Windows?

A running script blocks its terminal. If you need a detector and your DM bot at the same time, you need two windows — each SSH'd into the Pi.

Window 1 — Detector

$ ssh orbit
$ activate-oak
$ cd ~/oak-projects
$ python3 fatigue_detector.py --display

Window 2 — DM Bot

$ ssh orbit
$ activate-oak
$ cd ~/oak-projects
$ python3 discord_dm_notifier.py

Stop each with Ctrl+C in its own window.

Full details →

MULTIPLE WINDOWS

How to Open a Second Terminal

Mac

Cmd+T new tab
Cmd+N new window

Windows

Click + for a new tab
or open another window

VS Code

Click + in terminal panel
or click the split icon

VS Code tip: Split terminal lets you see both side-by-side — detector on left, bot on right.

Full details →

TROUBLESHOOTING

Common Errors

"No module named 'depthai'"
Run activate-oak first.

"Command not found: activate-oak"
Run source ~/.bashrc then try again.

"Permission denied"
You don't have sudo. Ask instructor.

Camera not found
Check USB 3.0 (blue) port. Unplug/replug. Anyone else using it?

Black screen in display
Wait ~10 sec for frames. If still black, Ctrl+C and retry.

VNC grey screen
Someone else is logged in at the monitor. Use SSH instead.

Full details →

TROUBLESHOOTING

"File Not Found" Checklist

1 Are you on the Pi? (Run hostname to check)

2 Did you copy the file? (ls ~/oak-projects/)

3 Any spaces in your path? Paths have no spaces!

4 Did you copy the utils/ folder? (-r flag for folders)

5 Did you copy depthai_models/?

Wrong

cd ~ / oak-projects

Correct

cd ~/oak-projects

PROJECT DIRECTORY

Fatigue Detection Layout

After copying files, your ~/oak-projects/ on the Pi:

~/oak-projects/
├── fatigue_detector.py            # Main detector script
├── utils/                         # Helper modules
│   └── face_landmarks.py
├── depthai_models/                # Model YAML configs
│   ├── yunet.RVC2.yaml
│   └── mediapipe_face_landmarker.RVC2.yaml
├── discord_notifier.py            # Webhook helper
├── discord_dm_notifier.py         # Your personal DM bot
└── .env                           # Your tokens (never commit!)

Python packages live in /opt/oak-shared/venv/ (shared by everyone).

Full details →

PROJECT DIRECTORY

What Files Do I Need?

Detector	Files to Copy	Notes
Fatigue	`fatigue_detector.py`, `utils/`, `depthai_models/`, `discord_notifier.py`, `discord_dm_notifier.py`	Recommended! Uses DM bot
Gaze	`gaze_detector.py`, `utils/`, `depthai_models/`	Terminal output only
Person	`person_detector_with_display.py`, `discord_notifier.py`	Uses class camera bots
DM Bot	`discord_dm_notifier.py`	Add token to `.env`

DEV TIPS

The Edit-Copy-Run Loop

Edit
on your laptop

→

scp
to the Pi

→

Run
with --display

→

Quit
press 'q'

↺

Changed a file? You must scp it to the Pi again! The Pi has the old version until you copy the new one.

Full details →

DEV TIPS

Adjusting Sensitivity & Logging

Detection threshold

# More sensitive (more false positives)
python3 person_detector*.py --threshold 0.3

# Less sensitive (fewer false positives)
python3 person_detector*.py --threshold 0.7

# Default is 0.5

Logging & headless

# Save events to a log file
python3 fatigue_detector.py --log

# Run headless + monitor via Discord
python3 person_detector*.py --discord
# Then use !screenshot in Discord

CLAUDE CODE

Install & Launch LOCAL

Install once on your laptop:

# Install Claude Code
curl -fsSL https://cli.anthropic.com/install.sh | sh

# Verify
claude --version

Then use it from your project directory:

$ cd ~/path/to/smart-objects-cameras
$ claude

Full details →

CLAUDE CODE

What It Can Do

› Read & explain your code
› Propose & apply code changes
› SSH into Pis & run commands
› Copy files to/from Pis
› Debug errors you paste in

Example:

You: How does the person detection
     pipeline work?

You: Modify the detector to also
     detect cars. Create a new file
     called person_detector_alice.py

You: I'm getting this error: [paste]

CLAUDE CODE

Safety Rules IMPORTANT

Read proposed changes carefully
Make sure you understand what it's changing
Ask it to create new files with your name
Ask questions if unsure

Don't

Auto-accept everything ("y y y")
Let it make changes you don't understand
Accept sudo commands blindly
Install packages you didn't ask for

Best practice: Ask Claude to create person_detector_yourname.py instead of editing the original. You can always compare your version to the reference!

Full details →

SMART OBJECTS

You're Ready!

Full README STUDENT_QUICKSTART.md

Workflow guide WORKFLOW.md

Version info WORKING_VERSIONS.md

Luxonis docs docs.luxonis.com

Stuck? Ask your instructor or use claude from your project directory.

SMART OBJECTS: THE SMART CLASSROOM

Student Quick Start

OVERVIEW

6-Step Setup Checklist

MENTAL MODEL

Where Am I? IMPORTANT

MENTAL MODEL

The Workflow

MENTAL MODEL

What's Done vs. What You Do

SETUP

Step 2: SSH Into a Pi LOCAL

SETUP

Step 3: Copy Files to Pi LOCAL

SETUP

Alt: VS Code Remote SSH

SETUP

Step 4: The .env File ON THE PI

SETUP

Step 5: Activate & Run ON THE PI

PYTHON FLAGS

Flag Reference

PYTHON FLAGS

When to Use --display

RUNNING DETECTORS

Fatigue Detection RECOMMENDED

RUNNING DETECTORS

Gaze Estimation RECOMMENDED

RUNNING DETECTORS

Person Detection (optional)

RUNNING DETECTORS

Quick Reference

MULTIPLE WINDOWS

Why Two Terminal Windows?

MULTIPLE WINDOWS

How to Open a Second Terminal

TROUBLESHOOTING

Common Errors

TROUBLESHOOTING

"File Not Found" Checklist

PROJECT DIRECTORY

Fatigue Detection Layout

PROJECT DIRECTORY

What Files Do I Need?

DEV TIPS

The Edit-Copy-Run Loop

DEV TIPS

Adjusting Sensitivity & Logging

CLAUDE CODE

Install & Launch LOCAL

CLAUDE CODE

What It Can Do

CLAUDE CODE

Safety Rules IMPORTANT

SMART OBJECTS

You're Ready!

When to Use `--display`