From 4a19da8b34a3aa0d98273c8699e9a6203462c770 Mon Sep 17 00:00:00 2001 From: jason Date: Sat, 28 Mar 2026 00:28:00 -0500 Subject: [PATCH] COMPLETE --- PROJECT.md | 217 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 217 insertions(+) create mode 100644 PROJECT.md diff --git a/PROJECT.md b/PROJECT.md new file mode 100644 index 0000000..c3de8a9 --- /dev/null +++ b/PROJECT.md @@ -0,0 +1,217 @@ +# UI Stock Tracker — Project Documentation + +**Status:** ✅ Complete and running +**Last updated:** 2026-03-28 + +--- + +## Purpose + +Monitors product pages on [store.ui.com](https://store.ui.com/us/en) for stock availability and sends a Telegram alert the moment a watched item comes back in stock. Runs as a single persistent Docker container on Unraid with a clean web UI for managing tracked products. + +--- + +## Stack + +| Layer | Technology | +|---|---| +| Frontend | React 18 + TypeScript, Vite, Lucide icons | +| Backend | Node.js 20 + TypeScript, Express | +| Database | SQLite via `better-sqlite3` (WAL mode) | +| Scraper | Puppeteer-core + system Chromium | +| Alerts | Telegram Bot API (direct HTTP) | +| Container | Single Docker container — nginx + Node.js managed by supervisord | +| Web server | nginx — serves frontend, proxies `/api/` to Node.js on port 3001 | + +--- + +## Container Architecture + +One container runs three processes via **supervisord**: + +``` +supervisord +├── nginx → serves React build on :8080, proxies /api/ → localhost:3001 +└── node → Express API, SQLite, Puppeteer scheduler, Telegram sender +``` + +The SQLite database is stored on a mounted volume at `/app/data/tracker.db` so it persists across rebuilds. + +--- + +## Key Features + +### Stock Detection +Puppeteer navigates to each product URL and waits for React hydration (2.5s delay), then scans all `` elements for exact text matches: +- `"Add to Cart"` → `in_stock` +- `"Sold Out"` → `sold_out` +- Neither found → `unknown` + +Product name is pulled from the `og:title` meta tag (with Ubiquiti store suffix stripped). Thumbnail is pulled from `og:image`. + +### Alert Logic +- Alert fires **once** when an item transitions to `in_stock` and `alert_sent = 0` +- `alert_sent` is set to `1` after firing — no repeat alerts for the same in-stock event +- User clicks **Re-arm** in the UI to reset `alert_sent = 0`, enabling a new alert next time it comes back in stock +- This covers the "missed the buy window" scenario without spamming + +### Scheduler +- Each watched item runs its own `setInterval` timer in memory +- Minimum interval enforced at **30 seconds** (backend + frontend) +- On startup, all active items are loaded from the DB and timers are started automatically +- Pause/Resume/Edit immediately updates the in-memory timer map and the DB +- A semaphore limits concurrent Puppeteer instances to **2** to prevent OOM on Unraid + +### Frontend Polling +The UI polls `GET /api/items` every **10 seconds** to keep check counts and statuses current without requiring a page refresh. + +--- + +## Database Schema + +```sql +watched_items ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + url TEXT NOT NULL UNIQUE, + name TEXT, -- populated after first scrape + thumbnail_url TEXT, -- populated after first scrape + check_interval INTEGER NOT NULL DEFAULT 60, -- seconds, min 30 + is_active INTEGER NOT NULL DEFAULT 1, -- 0 = paused + last_status TEXT NOT NULL DEFAULT 'unknown', -- in_stock | sold_out | unknown + alert_sent INTEGER NOT NULL DEFAULT 0, -- 1 = alert fired, awaiting re-arm + check_count INTEGER NOT NULL DEFAULT 0, + last_checked_at TEXT, + created_at TEXT NOT NULL DEFAULT (datetime('now')) +) + +settings ( + key TEXT PRIMARY KEY, -- telegram_bot_token | telegram_chat_id + value TEXT NOT NULL DEFAULT '' +) +``` + +--- + +## API Endpoints + +| Method | Path | Description | +|---|---|---| +| GET | `/api/items` | List all watched items | +| POST | `/api/items` | Add new item (url, check_interval) | +| PUT | `/api/items/:id` | Update interval or active state | +| DELETE | `/api/items/:id` | Remove item | +| POST | `/api/items/:id/pause` | Stop checking | +| POST | `/api/items/:id/resume` | Resume checking | +| POST | `/api/items/:id/reset` | Clear alert_sent (re-arm) | +| GET | `/api/settings` | Get Telegram credentials | +| PUT | `/api/settings` | Save Telegram credentials | +| POST | `/api/settings/test-telegram` | Send a test Telegram message | +| GET | `/api/health` | Health check | + +--- + +## Telegram Configuration + +| Field | Value | +|---|---| +| Bot Token | `8769097441:AAFBqPlSTcTIi3I-F5ZIN9EEpwbNDzHg8hM` | +| Chat ID | `8435449432` | + +Configured via **Settings** in the UI. Use **Test Alert** to verify the connection after saving. + +--- + +## Dockerfile Build Stages + +``` +Stage 1 — frontend-builder + node:20-slim → npm install → vite build → /build/frontend/dist + +Stage 2 — backend-builder + node:20-slim + python3/make/g++ → npm install → tsc → npm prune --production + +Stage 3 — runtime + node:20-slim + nginx + chromium + supervisor + ├── Copies /build/frontend/dist → /usr/share/nginx/html + ├── Copies /build/backend/dist → /app/backend/dist + ├── Copies /build/backend/node_modules → /app/backend/node_modules + ├── nginx.conf → /etc/nginx/conf.d/default.conf + └── supervisord.conf → /etc/supervisor/conf.d/supervisord.conf +``` + +--- + +## Build & Deploy + +```bash +# Initial build and start +cd /mnt/user/appdata/ui-tracker +docker build -t ui-tracker . +# Then add container via Unraid Docker GUI — see UNRAID.md + +# Rebuild after code changes +docker stop ui-tracker && docker rm ui-tracker +docker build -t ui-tracker . +# Re-add in Unraid GUI or: docker run ... (see UNRAID.md) +``` + +--- + +## Fixes Applied During Development + +### 1. `npm ci` → `npm install` (Dockerfile) +`npm ci` requires a `package-lock.json` to exist. Since lockfiles were not committed, the build failed immediately. Switched both frontend and backend build stages to `npm install`. + +### 2. TypeScript DOM lib missing (backend) +`scraper.ts` uses `page.evaluate()` which runs a callback in the browser context. TypeScript flagged `document`, `navigator`, and `HTMLMetaElement` as unknown because the backend `tsconfig.json` only included `"lib": ["ES2020"]`. Fixed by adding `"DOM"` and `"DOM.Iterable"`: +```json +"lib": ["ES2020", "DOM", "DOM.Iterable"] +``` +`DOM.Iterable` was specifically required to allow `for...of` iteration over `NodeListOf`. + +### 3. Two containers → one container +Original design used separate `frontend` and `backend` Docker services in docker-compose. Consolidated into a single container using **supervisord** to run nginx and Node.js side by side, with nginx proxying `/api/` to `localhost:3001`. Removed `backend/Dockerfile`, `frontend/Dockerfile`, and `frontend/nginx.conf`. Root-level `Dockerfile`, `nginx.conf`, and `supervisord.conf` replaced them. + +--- + +## File Structure + +``` +ui-tracker/ +├── Dockerfile # 3-stage single-container build +├── docker-compose.yml # Single service, port 8080, ./data volume +├── nginx.conf # Serves frontend, proxies /api/ to Node +├── supervisord.conf # Keeps nginx + node alive +├── PROJECT.md # This file +├── UNRAID.md # Installation guide +├── data/ # SQLite DB lives here (volume mount) +├── backend/ +│ ├── package.json +│ ├── tsconfig.json +│ └── src/ +│ ├── index.ts # Express entry + scheduler init +│ ├── database.ts # Schema, getDb(), WatchedItem type +│ ├── scraper.ts # Puppeteer stock check + semaphore +│ ├── scheduler.ts # Per-item interval timers + alert logic +│ ├── telegram.ts # Bot API HTTP calls +│ └── routes/ +│ ├── items.ts # Item CRUD + pause/resume/reset +│ └── settings.ts # Telegram config + test endpoint +└── frontend/ + ├── package.json + ├── tsconfig.json + ├── tsconfig.node.json + ├── vite.config.ts + ├── index.html + └── src/ + ├── main.tsx + ├── App.tsx # Root, 10s polling, connection indicator + ├── index.css # Dark theme, CSS variables, responsive grid + ├── types/index.ts + ├── api/client.ts # Typed fetch wrappers + └── components/ + ├── ItemCard.tsx # Thumbnail, badges, Re-arm, actions + ├── AddItemModal.tsx + ├── EditItemModal.tsx + └── SettingsModal.tsx # Token + chat ID + test alert +```