Jason Stedwell
95 lines
3.6 KiB
Markdown
95 lines
3.6 KiB
Markdown
# nyaa-crawler
|
|
|
|
A dockerized torrent crawler and downloader for [Nyaa.si](https://nyaa.si). Track anime shows, poll for new episodes via RSS, and automatically download `.torrent` files to a host-mounted directory.
|
|
|
|
## Features
|
|
|
|
- Search Nyaa.si and add shows to a watch list
|
|
- Automatic polling for new episodes (configurable interval, default 15 min)
|
|
- Auto-downloads `.torrent` files to a mapped host directory
|
|
- Backfills existing episodes (oldest first) when a show is added
|
|
- Tracks episode status: pending, auto-downloaded, manually marked, or failed
|
|
- Automatically retries failed downloads on each poll (up to 5 attempts), including ones that have scrolled off the RSS feed
|
|
- Surfaces failed downloads with their error message, plus a "Poll Now" manual trigger
|
|
- Bulk-mark episodes as already downloaded
|
|
- Health check that warns when the torrent output directory isn't writable
|
|
- Minimal dark-themed web UI
|
|
- SQLite persistence — easy to back up and migrate
|
|
- Unraid-friendly Docker container
|
|
|
|
## Stack
|
|
|
|
- **Backend**: Node.js + TypeScript + Express
|
|
- **Frontend**: React + Vite
|
|
- **Database**: SQLite via Node's built-in `node:sqlite` (requires Node 22+)
|
|
- **Scheduler**: `node-cron`
|
|
- **Nyaa integration**: RSS via `fast-xml-parser`
|
|
|
|
## Quick Start (Docker)
|
|
|
|
```bash
|
|
docker compose up -d
|
|
```
|
|
|
|
Open `http://localhost:8082` in your browser.
|
|
|
|
## Quick Start (Development)
|
|
|
|
Requires **Node.js 22+** (the app uses the built-in `node:sqlite` module).
|
|
|
|
```bash
|
|
npm install
|
|
npm run dev
|
|
```
|
|
|
|
- Client dev server: `http://localhost:5173` (proxies `/api` to `:3000`)
|
|
- API server: `http://localhost:3000`
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
npm test
|
|
```
|
|
|
|
Unit tests (via Node's built-in test runner) cover the pure logic most prone to
|
|
edge cases: episode-code parsing, batch-release detection, the poll-interval →
|
|
cron conversion, filename slugification, and release-title parsing.
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Default | Description |
|
|
|---|---|---|
|
|
| `PORT` | `3000` | HTTP port |
|
|
| `POLL_INTERVAL_SECONDS` | `900` | Polling frequency (min 60) |
|
|
| `TORRENT_OUTPUT_DIR` | `./data/torrents` | Where `.torrent` files are saved |
|
|
| `DATABASE_PATH` | `./data/db.sqlite` | SQLite database path |
|
|
|
|
## Unraid Deployment
|
|
|
|
```yaml
|
|
services:
|
|
nyaa-watcher:
|
|
image: your-registry/nyaa-watcher:latest
|
|
container_name: nyaa-watcher
|
|
restart: unless-stopped
|
|
environment:
|
|
- PORT=3000
|
|
- POLL_INTERVAL_SECONDS=900
|
|
- TORRENT_OUTPUT_DIR=/data/torrents
|
|
- DATABASE_PATH=/data/db.sqlite
|
|
volumes:
|
|
- /mnt/user/appdata/nyaa-watcher:/data
|
|
- /mnt/user/downloads/torrents/nyaa:/data/torrents
|
|
ports:
|
|
- "8082:3000"
|
|
```
|
|
|
|
## Notes
|
|
|
|
- Only `.torrent` files are downloaded — media is handled by your existing torrent client watching the output directory.
|
|
- Batch releases (titles containing "Batch", "Complete", "Vol.", or episode ranges) are automatically skipped.
|
|
- **Pause** stops further polling for a show but keeps its episodes. **Delete** permanently removes the show and its episode history (already-downloaded `.torrent` files on disk are left untouched).
|
|
- Failed downloads are retried automatically on each poll, up to 5 attempts; after that they stay `failed` (mark them downloaded manually, or delete and re-add the show). Failures are deduplicated against the live feed and retried even once they've scrolled off it.
|
|
- If the torrent output directory isn't writable, a warning banner appears on every page showing the failing path — fix the volume mapping or permissions and the next poll will recover.
|
|
- The SQLite schema migrates automatically on startup; existing databases are upgraded in place.
|