notifier/service-royalroad-chapters
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Feature/add or remove #7

Merged
florian merged 6 commits from feature/add-or-remove into main 2026-11-01 08:49:48 +00:00
Conversation 0 Commits 6 Files Changed 6
+175 -29
1 changed files with 175 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 73fcd3b53c - Show all commits
README.md
+175 -29
View File
@@ -3,20 +3,18 @@
## Overview ## Overview
`service-royalroad-chapters` is a FastAPI-based webserver that monitors Royalroad stories for new chapters and sends notifications data to `backend-api-internal`. `service-royalroad-chapters` is a FastAPI-based webserver that monitors RSS feeds for new entries and sends a notification to every subscribed user through `backend-api-internal`. It supports two feed types:
## Features - **Royal Road**: new chapters of a story
- **Gitea**: new releases of a repository on `gitea.com/gitea/`
- **RSS Feed Monitoring**: Parses Royal Road RSS feeds to detect new chapter releases Feeds are shared between users. A feed is stored once and polled once, and each user has their own subscription with an optional custom name.
- **Automatic Updates**: Updates database with latest chapter numbers
- **Retry Logic**: Built-in retry mechanisms for network failures with exponential backoff
- **Prometheus Metrics**: Request counters and performance monitoring on port 9000
## Prerequisites ## Prerequisites
- Python 3.10+ - Python 3.11+
- MySQL database - MySQL database with the tables described under [Database](database.sql), including the `users` table
- Backend notification service (for sending push notifications) - `backend-api-internal` (receives the notifications)
- Royal Road story IDs - Royal Road story IDs
## Installation ## Installation
@@ -25,6 +23,32 @@
pip install -r requirements.txt pip install -r requirements.txt
``` ```
Internal libraries (`lib-db-module`, `lib-secret-manager`, `lib-service-auth`) come from the Gitea package registry. Point pip at it first:
```bash
export PIP_EXTRA_INDEX_URL=https://git.gansejunge.com/api/packages/notifier/pypi/simple/
```
## Configuration
Secrets are read from files under `/etc/secrets/` (Kubernetes secrets) and fall back to environment variables for development. The files are deleted after startup.
| Secret | File | Dev environment variable |
|---|---|---|
| Gateway key | `/etc/secrets/gateway_key` | `GATEWAY_KEY` |
| Scheduler key | `/etc/secrets/scheduler_key` | `SCHEDULER_KEY` |
| Gitea token | `/etc/secrets/gitea_token` | `GITEA_TOKEN` |
| Notification API key | `/etc/secrets/api_key` | (see `lib-secret-manager`) |
| Database user | `/etc/secrets/db_username` | (see `lib-db-module`) |
| Database password | `/etc/secrets/db_password` | (see `lib-db-module`) |
Caller keys must be at least 32 characters and must differ from each other. Generate them with:
```bash
python3 -c "import secrets; print(secrets.token_urlsafe(32))"
```
## Running the Service ## Running the Service
```bash ```bash
@@ -35,36 +59,162 @@ This starts two servers:
- **Main API**: `http://0.0.0.0:5000` - Handles chapter check requests - **Main API**: `http://0.0.0.0:5000` - Handles chapter check requests
- **Metrics Server**: `http://0.0.0.0:9000/metrics` - Prometheus metrics endpoint - **Metrics Server**: `http://0.0.0.0:9000/metrics` - Prometheus metrics endpoint
### API Endpoint ## Authentication
#### Check for New Chapters Every route on the main API requires a bearer key, and each key belongs to exactly one caller:
```http ```http
GET /royalroad Authorization: Bearer <key>
``` ```
**Response:** | Caller | Key | May call |
|---|---|---|
| Gateway (`backend-api`) | `gateway_key` | `/add/feed`, `/delete/feed` |
| Scheduler (CronJob) | `scheduler_key` | `/royalroad`, `/gitea` |
A missing or unknown key returns `401`. A valid key used on a route it is not allowed to call returns `403`.
User routes also require the user the gateway is acting for:
```http
X-User-Id: 1
```
The value must be a positive integer.
## API
### Subscribe to a feed
```http
POST /add/feed
```
**Headers:** `Authorization: Bearer <gateway_key>`, `X-User-Id: <user id>`
**Body:**
```json ```json
{ {
"status": "checked" "type": "royalroad",
"rss_feed_url": "royalroad.com/fiction/118891",
"custom_name": "My favourite story"
}
```
| Field | Required | Description |
|---|---|---|
| `type` | yes | `royalroad` or `gitea` |
| `rss_feed_url` | yes | Feed or page URL, see [Accepted feed URLs](#accepted-feed-urls) |
| `custom_name` | no | Name used as the notification title. If empty, the title from the feed is used |
**Response:**
```json
{
"status": "ok",
"detail": "Subscribed to feed",
"feed_url": "https://www.royalroad.com/fiction/syndication/65058"
} }
``` ```
**Behavior:** **Behavior:**
1. Fetches all active stories from the database
2. Queries Royal Road RSS feed for each story
3. Compares latest chapter with stored chapter number
4. Updates database if new chapter is detected
5. Sends notification to backend service
**Error Responses:** 1. Validates the URL and converts it to the canonical feed URL
- `404`: Feed not found or has no entries 2. If the feed is not stored yet, checks that it exists and records its latest entry, so only entries published after subscribing trigger a notification
- `500`: Internal server error during processing 3. Stores the feed, or reactivates it if it was inactive
- `503`: Notification service unavailable after retries 4. Creates the user's subscription
Sending the same request again renames the subscription and reactivates it if the user had unsubscribed.
### Unsubscribe from a feed
```http
POST /delete/feed
```
**Headers:** `Authorization: Bearer <gateway_key>`, `X-User-Id: <user id>`
**Body:** same as `/add/feed`. `custom_name` is ignored.
**Response:**
```json
{
"status": "ok",
"detail": "Unsubscribed from feed",
"feed_url": "https://www.royalroad.com/fiction/syndication/118891"
}
```
**Behavior:** deactivates the user's subscription. If no active subscribers are left, the feed itself is deactivated and is no longer polled. An unknown feed also returns `ok`.
### Check for new chapters
```http
POST /royalroad
```
**Headers:** `Authorization: Bearer <scheduler_key>`
### Check for new Gitea releases
```http
POST /gitea
```
**Headers:** `Authorization: Bearer <scheduler_key>`
**Response (both):**
```json
{
"checked": 12,
"new": 2,
"errors": 0
}
```
If any feed failed, the same counts are returned as the `detail` of a `500` response.
**Notification failures:**
- If every notification for a feed fails, `last_entry` is not updated, the feed counts as an error and it is retried on the next run
- If only some fail, `last_entry` is updated, so the users who were notified are not notified again
- If the notification backend is unavailable, the remaining users of that feed are skipped
When a new release of `gitea/runner` is detected, the service also triggers the runner build workflow on the Gitea instance, using `gitea_token`.
## Accepted feed URLs
The scheme and `www.` are optional. Whatever is submitted is converted to the canonical feed URL.
| Type | Accepted input | Stored as |
|---|---|---|
| `royalroad` | `royalroad.com/fiction/118891`<br>`https://www.royalroad.com/fiction/118891/story-title` | `https://www.royalroad.com/fiction/syndication/118891` |
| `gitea` | `gitea.com/gitea/runner`<br>`https://gitea.com/gitea/runner/releases`<br>`https://gitea.com/gitea/runner/releases.rss` | `https://gitea.com/gitea/runner/releases.rss` |
Only repositories under `gitea.com/gitea/` are supported.
## Error Responses
Errors are returned as `{"detail": "<message>"}`.
| Status | Cause | Example `detail` |
|---|---|---|
| `401` | Missing or invalid bearer key | `Unauthorized` |
| `403` | Key is valid but not allowed on this route | `Forbidden` |
| `422` | Invalid `X-User-Id` or body | `type: Input should be 'royalroad' or 'gitea'` |
| `422` | URL does not match the feed type | `A gitea feed must be on gitea.com` |
| `422` | URL has the wrong path | `Unrecognised feed path (expected e.g. royalroad.com/fiction/118891)` |
| `422` | The feed does not exist | `The feed does not exist: <url>` |
| `502` | The feed could not be checked right now | `Could not verify the feed right now, try again later: <url>` |
| `500` | Internal error, or a check run with failed feeds | |
## Notification Format ## Notification Format
Messages sent to the backend service: One message is sent to `backend-api-internal` per subscriber:
```json ```json
{ {
@@ -79,11 +229,7 @@ Messages sent to the backend service:
} }
``` ```
**Headers:** Gitea releases use the category `utility`. When a subscription has no custom name, the title is the feed title for Royal Road and the repository path (for example `gitea.com/gitea/runner`) for Gitea.
```http
X-API-Key-Internal: <api_key>
Content-Type: application/json
```
## Monitoring ## Monitoring