COMPLETE
This commit is contained in:
217
PROJECT.md
Normal file
217
PROJECT.md
Normal file
@@ -0,0 +1,217 @@
|
||||
# UI Stock Tracker — Project Documentation
|
||||
|
||||
**Status:** ✅ Complete and running
|
||||
**Last updated:** 2026-03-28
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Monitors product pages on [store.ui.com](https://store.ui.com/us/en) for stock availability and sends a Telegram alert the moment a watched item comes back in stock. Runs as a single persistent Docker container on Unraid with a clean web UI for managing tracked products.
|
||||
|
||||
---
|
||||
|
||||
## Stack
|
||||
|
||||
| Layer | Technology |
|
||||
|---|---|
|
||||
| Frontend | React 18 + TypeScript, Vite, Lucide icons |
|
||||
| Backend | Node.js 20 + TypeScript, Express |
|
||||
| Database | SQLite via `better-sqlite3` (WAL mode) |
|
||||
| Scraper | Puppeteer-core + system Chromium |
|
||||
| Alerts | Telegram Bot API (direct HTTP) |
|
||||
| Container | Single Docker container — nginx + Node.js managed by supervisord |
|
||||
| Web server | nginx — serves frontend, proxies `/api/` to Node.js on port 3001 |
|
||||
|
||||
---
|
||||
|
||||
## Container Architecture
|
||||
|
||||
One container runs three processes via **supervisord**:
|
||||
|
||||
```
|
||||
supervisord
|
||||
├── nginx → serves React build on :8080, proxies /api/ → localhost:3001
|
||||
└── node → Express API, SQLite, Puppeteer scheduler, Telegram sender
|
||||
```
|
||||
|
||||
The SQLite database is stored on a mounted volume at `/app/data/tracker.db` so it persists across rebuilds.
|
||||
|
||||
---
|
||||
|
||||
## Key Features
|
||||
|
||||
### Stock Detection
|
||||
Puppeteer navigates to each product URL and waits for React hydration (2.5s delay), then scans all `<span>` elements for exact text matches:
|
||||
- `"Add to Cart"` → `in_stock`
|
||||
- `"Sold Out"` → `sold_out`
|
||||
- Neither found → `unknown`
|
||||
|
||||
Product name is pulled from the `og:title` meta tag (with Ubiquiti store suffix stripped). Thumbnail is pulled from `og:image`.
|
||||
|
||||
### Alert Logic
|
||||
- Alert fires **once** when an item transitions to `in_stock` and `alert_sent = 0`
|
||||
- `alert_sent` is set to `1` after firing — no repeat alerts for the same in-stock event
|
||||
- User clicks **Re-arm** in the UI to reset `alert_sent = 0`, enabling a new alert next time it comes back in stock
|
||||
- This covers the "missed the buy window" scenario without spamming
|
||||
|
||||
### Scheduler
|
||||
- Each watched item runs its own `setInterval` timer in memory
|
||||
- Minimum interval enforced at **30 seconds** (backend + frontend)
|
||||
- On startup, all active items are loaded from the DB and timers are started automatically
|
||||
- Pause/Resume/Edit immediately updates the in-memory timer map and the DB
|
||||
- A semaphore limits concurrent Puppeteer instances to **2** to prevent OOM on Unraid
|
||||
|
||||
### Frontend Polling
|
||||
The UI polls `GET /api/items` every **10 seconds** to keep check counts and statuses current without requiring a page refresh.
|
||||
|
||||
---
|
||||
|
||||
## Database Schema
|
||||
|
||||
```sql
|
||||
watched_items (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
url TEXT NOT NULL UNIQUE,
|
||||
name TEXT, -- populated after first scrape
|
||||
thumbnail_url TEXT, -- populated after first scrape
|
||||
check_interval INTEGER NOT NULL DEFAULT 60, -- seconds, min 30
|
||||
is_active INTEGER NOT NULL DEFAULT 1, -- 0 = paused
|
||||
last_status TEXT NOT NULL DEFAULT 'unknown', -- in_stock | sold_out | unknown
|
||||
alert_sent INTEGER NOT NULL DEFAULT 0, -- 1 = alert fired, awaiting re-arm
|
||||
check_count INTEGER NOT NULL DEFAULT 0,
|
||||
last_checked_at TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
)
|
||||
|
||||
settings (
|
||||
key TEXT PRIMARY KEY, -- telegram_bot_token | telegram_chat_id
|
||||
value TEXT NOT NULL DEFAULT ''
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
| Method | Path | Description |
|
||||
|---|---|---|
|
||||
| GET | `/api/items` | List all watched items |
|
||||
| POST | `/api/items` | Add new item (url, check_interval) |
|
||||
| PUT | `/api/items/:id` | Update interval or active state |
|
||||
| DELETE | `/api/items/:id` | Remove item |
|
||||
| POST | `/api/items/:id/pause` | Stop checking |
|
||||
| POST | `/api/items/:id/resume` | Resume checking |
|
||||
| POST | `/api/items/:id/reset` | Clear alert_sent (re-arm) |
|
||||
| GET | `/api/settings` | Get Telegram credentials |
|
||||
| PUT | `/api/settings` | Save Telegram credentials |
|
||||
| POST | `/api/settings/test-telegram` | Send a test Telegram message |
|
||||
| GET | `/api/health` | Health check |
|
||||
|
||||
---
|
||||
|
||||
## Telegram Configuration
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Bot Token | `8769097441:AAFBqPlSTcTIi3I-F5ZIN9EEpwbNDzHg8hM` |
|
||||
| Chat ID | `8435449432` |
|
||||
|
||||
Configured via **Settings** in the UI. Use **Test Alert** to verify the connection after saving.
|
||||
|
||||
---
|
||||
|
||||
## Dockerfile Build Stages
|
||||
|
||||
```
|
||||
Stage 1 — frontend-builder
|
||||
node:20-slim → npm install → vite build → /build/frontend/dist
|
||||
|
||||
Stage 2 — backend-builder
|
||||
node:20-slim + python3/make/g++ → npm install → tsc → npm prune --production
|
||||
|
||||
Stage 3 — runtime
|
||||
node:20-slim + nginx + chromium + supervisor
|
||||
├── Copies /build/frontend/dist → /usr/share/nginx/html
|
||||
├── Copies /build/backend/dist → /app/backend/dist
|
||||
├── Copies /build/backend/node_modules → /app/backend/node_modules
|
||||
├── nginx.conf → /etc/nginx/conf.d/default.conf
|
||||
└── supervisord.conf → /etc/supervisor/conf.d/supervisord.conf
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Build & Deploy
|
||||
|
||||
```bash
|
||||
# Initial build and start
|
||||
cd /mnt/user/appdata/ui-tracker
|
||||
docker build -t ui-tracker .
|
||||
# Then add container via Unraid Docker GUI — see UNRAID.md
|
||||
|
||||
# Rebuild after code changes
|
||||
docker stop ui-tracker && docker rm ui-tracker
|
||||
docker build -t ui-tracker .
|
||||
# Re-add in Unraid GUI or: docker run ... (see UNRAID.md)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Fixes Applied During Development
|
||||
|
||||
### 1. `npm ci` → `npm install` (Dockerfile)
|
||||
`npm ci` requires a `package-lock.json` to exist. Since lockfiles were not committed, the build failed immediately. Switched both frontend and backend build stages to `npm install`.
|
||||
|
||||
### 2. TypeScript DOM lib missing (backend)
|
||||
`scraper.ts` uses `page.evaluate()` which runs a callback in the browser context. TypeScript flagged `document`, `navigator`, and `HTMLMetaElement` as unknown because the backend `tsconfig.json` only included `"lib": ["ES2020"]`. Fixed by adding `"DOM"` and `"DOM.Iterable"`:
|
||||
```json
|
||||
"lib": ["ES2020", "DOM", "DOM.Iterable"]
|
||||
```
|
||||
`DOM.Iterable` was specifically required to allow `for...of` iteration over `NodeListOf<HTMLSpanElement>`.
|
||||
|
||||
### 3. Two containers → one container
|
||||
Original design used separate `frontend` and `backend` Docker services in docker-compose. Consolidated into a single container using **supervisord** to run nginx and Node.js side by side, with nginx proxying `/api/` to `localhost:3001`. Removed `backend/Dockerfile`, `frontend/Dockerfile`, and `frontend/nginx.conf`. Root-level `Dockerfile`, `nginx.conf`, and `supervisord.conf` replaced them.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
ui-tracker/
|
||||
├── Dockerfile # 3-stage single-container build
|
||||
├── docker-compose.yml # Single service, port 8080, ./data volume
|
||||
├── nginx.conf # Serves frontend, proxies /api/ to Node
|
||||
├── supervisord.conf # Keeps nginx + node alive
|
||||
├── PROJECT.md # This file
|
||||
├── UNRAID.md # Installation guide
|
||||
├── data/ # SQLite DB lives here (volume mount)
|
||||
├── backend/
|
||||
│ ├── package.json
|
||||
│ ├── tsconfig.json
|
||||
│ └── src/
|
||||
│ ├── index.ts # Express entry + scheduler init
|
||||
│ ├── database.ts # Schema, getDb(), WatchedItem type
|
||||
│ ├── scraper.ts # Puppeteer stock check + semaphore
|
||||
│ ├── scheduler.ts # Per-item interval timers + alert logic
|
||||
│ ├── telegram.ts # Bot API HTTP calls
|
||||
│ └── routes/
|
||||
│ ├── items.ts # Item CRUD + pause/resume/reset
|
||||
│ └── settings.ts # Telegram config + test endpoint
|
||||
└── frontend/
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
├── tsconfig.node.json
|
||||
├── vite.config.ts
|
||||
├── index.html
|
||||
└── src/
|
||||
├── main.tsx
|
||||
├── App.tsx # Root, 10s polling, connection indicator
|
||||
├── index.css # Dark theme, CSS variables, responsive grid
|
||||
├── types/index.ts
|
||||
├── api/client.ts # Typed fetch wrappers
|
||||
└── components/
|
||||
├── ItemCard.tsx # Thumbnail, badges, Re-arm, actions
|
||||
├── AddItemModal.tsx
|
||||
├── EditItemModal.tsx
|
||||
└── SettingsModal.tsx # Token + chat ID + test alert
|
||||
```
|
||||
Reference in New Issue
Block a user