Files
ui-tracker/README.md
T

293 lines
13 KiB
Markdown
Raw Normal View History

2026-06-22 19:51:24 -05:00
# UI Stock Tracker
Monitors product pages on [store.ui.com](https://store.ui.com/us/en) for stock availability and fires a Telegram alert the moment a watched item comes back in stock. Runs as a single self-contained Docker container on Unraid with a web UI for managing tracked products.
**Status:** Complete and running.
## How It Works
A Puppeteer scraper visits each watched product URL on a per-item timer, waits for the React page to hydrate, and classifies the page as `in_stock`, `sold_out`, or `unknown` by inspecting the Add-to-Cart / notify buttons. When an item transitions to `in_stock`, the backend sends a one-time Telegram alert. You re-arm the item in the UI to enable the next alert, so you get notified without being spammed.
Stock detection rules:
- A `<button>` with `label="Add to Cart"`, or whose text contains `"add to cart"``in_stock`.
- A `<button>` whose text contains `"login for notifications"`, `"login for updates"`, `"notify me when available"`, `"notify me"`, `"sold out"`, `"out of stock"`, `"currently unavailable"`, or `"coming soon"``sold_out`.
- Neither found → `unknown` (logs a debug payload with page title and button texts).
The scraper runs logged out, so Ubiquiti shows `"Login for Notifications"` on sold-out items — that string is in the sold-out list by design. Product name comes from the `og:title` meta tag (store suffix stripped) and the thumbnail from `og:image`.
## Stack
| Layer | Technology |
|---|---|
| Frontend | React 18 + TypeScript, Vite, Lucide icons |
| Backend | Node.js 20 + TypeScript, Express |
| Database | SQLite via `better-sqlite3` (WAL mode) |
| Scraper | `puppeteer-core` + system Chromium |
| Alerts | Telegram Bot API (direct HTTP) |
| Container | Single Docker image — nginx + Node managed by supervisord |
## Architecture
One container runs two processes under **supervisord**:
```
supervisord
├── nginx → serves the React build on :8080, proxies /api/ → 127.0.0.1:3001
└── node → Express API, SQLite, Puppeteer scheduler, Telegram sender (:3001)
```
The SQLite database lives on a mounted volume at `/app/data/tracker.db`, so it persists across rebuilds.
Backend runtime behavior:
- **Scheduler** — each active item runs its own in-memory `setInterval`; timers start automatically on boot from the DB. Minimum interval is **30 seconds** (enforced backend and frontend). Pause / Resume / Edit update both the timer map and the DB immediately.
- **Concurrency** — a semaphore caps concurrent Puppeteer instances at **2** to avoid OOM on Unraid.
- **Alert logic** — an alert fires once when an item goes `in_stock` with `alert_sent = 0`, then sets `alert_sent = 1`. **Re-arm** in the UI resets it to `0`.
- **Frontend polling** — the UI polls `GET /api/items` every **10 seconds**.
## Repository Layout
```
ui-tracker/
├── Dockerfile # 3-stage single-container build
├── docker-compose.yml # Single service, port 8080, ./data volume
├── nginx.conf # Serves frontend, proxies /api/ to Node
├── supervisord.conf # Keeps nginx + node alive
├── .gitea/workflows/ # CI: build + push image on push to main
├── data/ # SQLite DB (volume mount)
├── backend/
│ └── src/
│ ├── index.ts # Express entry + scheduler init
│ ├── database.ts # Schema, getDb(), WatchedItem type
│ ├── scraper.ts # Puppeteer stock check + semaphore
│ ├── scheduler.ts # Per-item timers + alert logic
│ ├── telegram.ts # Bot API HTTP calls
│ └── routes/ # items.ts, settings.ts
└── frontend/
└── src/
├── App.tsx # Root, 10s polling, connection indicator
├── api/client.ts # Typed fetch wrappers
├── types/index.ts
└── components/ # ItemCard, AddItemModal, EditItemModal, SettingsModal
```
## API
| Method | Path | Description |
|---|---|---|
| GET | `/api/items` | List all watched items |
| POST | `/api/items` | Add new item (`url`, `check_interval`) |
| PUT | `/api/items/:id` | Update interval or active state |
| DELETE | `/api/items/:id` | Remove item |
| POST | `/api/items/:id/pause` | Stop checking |
| POST | `/api/items/:id/resume` | Resume checking |
| POST | `/api/items/:id/reset` | Clear `alert_sent` (re-arm) |
| GET | `/api/settings` | Get Telegram credentials |
| PUT | `/api/settings` | Save Telegram credentials |
| POST | `/api/settings/test-telegram` | Send a test Telegram message |
| GET | `/api/health` | Health check |
## Database Schema
```sql
watched_items (
id INTEGER PRIMARY KEY AUTOINCREMENT,
url TEXT NOT NULL UNIQUE,
name TEXT, -- populated after first scrape
thumbnail_url TEXT, -- populated after first scrape
check_interval INTEGER NOT NULL DEFAULT 60, -- seconds, min 30
is_active INTEGER NOT NULL DEFAULT 1, -- 0 = paused
last_status TEXT NOT NULL DEFAULT 'unknown', -- in_stock | sold_out | unknown
alert_sent INTEGER NOT NULL DEFAULT 0, -- 1 = alert fired, awaiting re-arm
check_count INTEGER NOT NULL DEFAULT 0,
last_checked_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
)
settings (
key TEXT PRIMARY KEY, -- telegram_bot_token | telegram_chat_id
value TEXT NOT NULL DEFAULT ''
)
```
## Local Development
The backend and frontend run independently in dev.
```bash
# Backend (http://localhost:3001)
cd backend
npm install
npm run dev # ts-node-dev, respawns on change
# Frontend (http://localhost:5173, Vite)
cd frontend
npm install
npm run dev
```
Backend env vars (set automatically in the container via `supervisord.conf`): `PORT` (default `3001`), `DATABASE_PATH` (default `/app/data/tracker.db`), `PUPPETEER_EXECUTABLE_PATH` (default `/usr/bin/chromium`).
> Lockfiles are not committed, so the Docker build uses `npm install` rather than `npm ci`.
## Build & Run (Docker)
```bash
docker build -t ui-tracker .
docker compose up -d # serves on http://localhost:8080
```
The image builds in three stages: Vite builds the frontend, `tsc` compiles the backend (then `npm prune --production`), and the runtime stage assembles `node:20-slim` + nginx + Chromium + supervisor.
2026-06-25 22:39:23 -05:00
Full Unraid install instructions are in [Deploy on Unraid](#deploy-on-unraid) below.
2026-06-22 19:51:24 -05:00
## CI/CD
`.gitea/workflows/docker-build.yml` builds and pushes the image on every push to `main`:
1. Checks out the repo.
2. Logs in to the private registry at `registry.alwisp.com` (`secrets.REGISTRY_USER` / `secrets.REGISTRY_TOKEN`).
3. Builds and pushes `registry.alwisp.com/{owner}/{repo}:latest`.
Pull the updated image on Unraid afterward:
```bash
docker pull registry.alwisp.com/{owner}/{repo}:latest
```
## Telegram Configuration
Configure via **Settings** in the UI, then use **Test Alert** to verify.
| Field | Value |
|---|---|
| Bot Token | `8769097441:AAFBqPlSTcTIi3I-F5ZIN9EEpwbNDzHg8hM` |
| Chat ID | `8435449432` |
> ⚠️ These are live credentials committed to the repo. Rotate the bot token (via @BotFather) if this repository is ever made public or shared.
2026-06-25 22:39:23 -05:00
## Deploy on Unraid
2026-06-22 19:51:24 -05:00
2026-06-25 22:39:23 -05:00
### 1. Clone and build the image (CLI)
SSH into your Unraid server and run:
```bash
mkdir -p /mnt/user/appdata/ui-tracker
cd /mnt/user/appdata/ui-tracker
git clone https://git.alwisp.com/jason/ui-tracker.git .
docker build -t ui-tracker .
```
The first build takes a few minutes — it compiles the frontend, compiles the backend, and installs Chromium inside the image. The result is available locally as `ui-tracker`.
### 2. Add the container (Unraid GUI)
In the Unraid web UI go to **Docker → Add Container** and fill in each section:
**Basic**
| Field | Value |
|---|---|
| Name | `ui-tracker` |
| Repository | `ui-tracker` |
| Icon URL | *(leave blank)* |
**Network**
| Field | Value |
|---|---|
| Network Type | `br0` |
| Fixed IP Address | Your chosen static LAN IP — e.g. `192.168.1.50` |
> With `br0`, the container gets its own IP on your LAN, so you reach the UI directly at `http://<fixed-ip>:8080` with no port conflict against the Unraid host.
**Port***Add another Path, Port, Variable, Label or Device → Port*
| Field | Value |
|---|---|
| Name | `Web UI` |
| Container Port | `8080` |
| Host Port | `8080` |
| Protocol | `TCP` |
**Path (persistent data)***Add another Path, Port, Variable, Label or Device → Path*
| Field | Value |
|---|---|
| Name | `Data` |
| Container Path | `/app/data` |
| Host Path | `/mnt/user/appdata/ui-tracker/data` |
| Access Mode | `Read/Write` |
> This is where the SQLite database lives — tracked items and Telegram settings persist here across restarts and rebuilds.
**Variables** — none required; all configuration is baked into the image.
Click **Apply** to create and start the container.
### 3. First-time setup
Open the UI at `http://<fixed-ip>:8080` and:
1. Click **Settings** (top right).
2. Enter your **Bot Token** and **Chat ID** (see [Telegram Configuration](#telegram-configuration)).
3. Click **Test Alert** — a Telegram message should arrive within seconds.
4. Click **Save**.
5. Click **Add Item**, paste a `store.ui.com` product URL, set the check interval, and click **Start Tracking**.
### Rebuilding after an update
```bash
cd /mnt/user/appdata/ui-tracker
git pull
docker stop ui-tracker
docker rm ui-tracker
docker build -t ui-tracker .
```
Then re-add the container from **Docker → Add Container** with the same settings, or start it from the Docker tab if Unraid retained the template.
### Troubleshooting
**UI not loading**
- Confirm the container is running in the Docker tab.
- Check logs: click the container icon → **Logs**.
- Make sure the fixed IP isn't already in use on the network.
**Telegram test fails**
- Verify the bot token and chat ID in Settings.
- Send `/start` to your bot in Telegram at least once to open the conversation.
- Confirm Unraid has outbound HTTPS (port 443) access.
**Items stuck on "Unknown" status**
- Open container logs and look for Puppeteer errors.
- Restart the container — Chromium occasionally needs a clean start.
## Design Notes & Development History
### Stock detection rationale
The scraper classifies each product page using case- and whitespace-insensitive substring matching against the page's buttons. Two non-obvious details drove the matching rules:
**Auth-state difference.** The scraper runs logged out. On a sold-out item, Ubiquiti shows `"Login for Notifications"` to logged-out visitors instead of the logged-in `"Notify me when available"`. Both strings must be in the sold-out list, or logged-out checks would fall through to `unknown`.
**The `label` attribute exists only on the in-stock button.** The production DOM carries `label="Add to Cart"` on the primary Add-to-Cart button (confirmed in DevTools), but the sold-out "Login for Notifications" button has no `label` attribute. So in-stock has two redundant signals (the attribute *and* the button text) while sold-out relies on text alone — this asymmetry is by design.
When neither signal matches, the scraper logs a `[Scraper] UNKNOWN debug` payload (page title, button labels, button texts) so a misclassification can be diagnosed from container logs without reproducing locally. To avoid reading the page before React finishes rendering, the scraper waits 2.5s plus an active wait for a `<button>` to appear (up to 8s) before evaluating.
### Fixes applied during development
**`npm ci``npm install` (Dockerfile).** `npm ci` requires a committed `package-lock.json`. Lockfiles were never committed, so the build failed immediately. Both build stages use `npm install` instead — which is why no lockfiles are present in the repo.
**TypeScript DOM lib missing (backend).** `scraper.ts` uses `page.evaluate()`, whose callback runs in the browser context. TypeScript flagged `document`, `navigator`, and `HTMLMetaElement` as unknown because the backend `tsconfig.json` only included `"lib": ["ES2020"]`. Fixed by adding the DOM libs:
```json
"lib": ["ES2020", "DOM", "DOM.Iterable"]
```
`DOM.Iterable` was specifically required to allow `for...of` iteration over `NodeListOf<HTMLSpanElement>`.
**Two containers → one container.** The original design used separate `frontend` and `backend` Docker services in docker-compose. These were consolidated into a single container running nginx and Node side by side under supervisord, with nginx proxying `/api/` to `localhost:3001`. Removed `backend/Dockerfile`, `frontend/Dockerfile`, and `frontend/nginx.conf`; the root-level `Dockerfile`, `nginx.conf`, and `supervisord.conf` replaced them.