notifier/service-royalroad-chapters
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Added path to unsubscribe from RSS feeds and exception handler to propagate FeedErrors

Browse Source
This commit is contained in:
florian committed 2026-11-01 09:49:02 +01:00
1 parent 6f1f024e90
commit 73fcd3b53c
1 file changed
+182 -36
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+182 -36
View File
@@ -3,20 +3,18 @@
## Overview
`service-royalroad-chapters` is a FastAPI-based webserver that monitors Royalroad stories for new chapters and sends notifications data to `backend-api-internal`.
`service-royalroad-chapters` is a FastAPI-based webserver that monitors RSS feeds for new entries and sends a notification to every subscribed user through `backend-api-internal`. It supports two feed types:
## Features
- **Royal Road**: new chapters of a story
- **Gitea**: new releases of a repository on `gitea.com/gitea/`
- **RSS Feed Monitoring**: Parses Royal Road RSS feeds to detect new chapter releases
- **Automatic Updates**: Updates database with latest chapter numbers
- **Retry Logic**: Built-in retry mechanisms for network failures with exponential backoff
- **Prometheus Metrics**: Request counters and performance monitoring on port 9000
Feeds are shared between users. A feed is stored once and polled once, and each user has their own subscription with an optional custom name.
## Prerequisites
- Python 3.10+
- MySQL database
- Backend notification service (for sending push notifications)
- Python 3.11+
- MySQL database with the tables described under [Database](database.sql), including the `users` table
- `backend-api-internal` (receives the notifications)
- Royal Road story IDs
## Installation
@@ -25,6 +23,32 @@
pip install -r requirements.txt
```
Internal libraries (`lib-db-module`, `lib-secret-manager`, `lib-service-auth`) come from the Gitea package registry. Point pip at it first:
```bash
export PIP_EXTRA_INDEX_URL=https://git.gansejunge.com/api/packages/notifier/pypi/simple/
```
## Configuration
Secrets are read from files under `/etc/secrets/` (Kubernetes secrets) and fall back to environment variables for development. The files are deleted after startup.
| Secret | File | Dev environment variable |
|---|---|---|
| Gateway key | `/etc/secrets/gateway_key` | `GATEWAY_KEY` |
| Scheduler key | `/etc/secrets/scheduler_key` | `SCHEDULER_KEY` |
| Gitea token | `/etc/secrets/gitea_token` | `GITEA_TOKEN` |
| Notification API key | `/etc/secrets/api_key` | (see `lib-secret-manager`) |
| Database user | `/etc/secrets/db_username` | (see `lib-db-module`) |
| Database password | `/etc/secrets/db_password` | (see `lib-db-module`) |
Caller keys must be at least 32 characters and must differ from each other. Generate them with:
```bash
python3 -c "import secrets; print(secrets.token_urlsafe(32))"
```
## Running the Service
```bash
@@ -35,56 +59,178 @@ This starts two servers:
- **Main API**: `http://0.0.0.0:5000` - Handles chapter check requests
- **Metrics Server**: `http://0.0.0.0:9000/metrics` - Prometheus metrics endpoint
### API Endpoint
## Authentication
#### Check for New Chapters
Every route on the main API requires a bearer key, and each key belongs to exactly one caller:
```http
GET /royalroad
Authorization: Bearer <key>
```
**Response:**
| Caller | Key | May call |
|---|---|---|
| Gateway (`backend-api`) | `gateway_key` | `/add/feed`, `/delete/feed` |
| Scheduler (CronJob) | `scheduler_key` | `/royalroad`, `/gitea` |
A missing or unknown key returns `401`. A valid key used on a route it is not allowed to call returns `403`.
User routes also require the user the gateway is acting for:
```http
X-User-Id: 1
```
The value must be a positive integer.
## API
### Subscribe to a feed
```http
POST /add/feed
```
**Headers:** `Authorization: Bearer <gateway_key>`, `X-User-Id: <user id>`
**Body:**
```json
{
"status": "checked"
"type": "royalroad",
"rss_feed_url": "royalroad.com/fiction/118891",
"custom_name": "My favourite story"
}
```
| Field | Required | Description |
|---|---|---|
| `type` | yes | `royalroad` or `gitea` |
| `rss_feed_url` | yes | Feed or page URL, see [Accepted feed URLs](#accepted-feed-urls) |
| `custom_name` | no | Name used as the notification title. If empty, the title from the feed is used |
**Response:**
```json
{
"status": "ok",
"detail": "Subscribed to feed",
"feed_url": "https://www.royalroad.com/fiction/syndication/65058"
}
```
**Behavior:**
1. Fetches all active stories from the database
2. Queries Royal Road RSS feed for each story
3. Compares latest chapter with stored chapter number
4. Updates database if new chapter is detected
5. Sends notification to backend service
**Error Responses:**
- `404`: Feed not found or has no entries
- `500`: Internal server error during processing
- `503`: Notification service unavailable after retries
1. Validates the URL and converts it to the canonical feed URL
2. If the feed is not stored yet, checks that it exists and records its latest entry, so only entries published after subscribing trigger a notification
3. Stores the feed, or reactivates it if it was inactive
4. Creates the user's subscription
## Notification Format
Sending the same request again renames the subscription and reactivates it if the user had unsubscribed.
Messages sent to the backend service:
### Unsubscribe from a feed
```http
POST /delete/feed
```
**Headers:** `Authorization: Bearer <gateway_key>`, `X-User-Id: <user id>`
**Body:** same as `/add/feed`. `custom_name` is ignored.
**Response:**
```json
{
"user_id": 1,
"message": {
"title": "New Life As A Max Level Archmage",
"body": "Chapter 68 has been released",
"link": "https://www.royalroad.com/fiction/118891/new-life-as-a-max-level-archmage/chapter/2683797/68-carve",
"category": "royal-road",
"timestamp": 1729339200
}
"status": "ok",
"detail": "Unsubscribed from feed",
"feed_url": "https://www.royalroad.com/fiction/syndication/118891"
}
```
**Headers:**
**Behavior:** deactivates the user's subscription. If no active subscribers are left, the feed itself is deactivated and is no longer polled. An unknown feed also returns `ok`.
### Check for new chapters
```http
X-API-Key-Internal: <api_key>
Content-Type: application/json
POST /royalroad
```
**Headers:** `Authorization: Bearer <scheduler_key>`
### Check for new Gitea releases
```http
POST /gitea
```
**Headers:** `Authorization: Bearer <scheduler_key>`
**Response (both):**
```json
{
"checked": 12,
"new": 2,
"errors": 0
}
```
If any feed failed, the same counts are returned as the `detail` of a `500` response.
**Notification failures:**
- If every notification for a feed fails, `last_entry` is not updated, the feed counts as an error and it is retried on the next run
- If only some fail, `last_entry` is updated, so the users who were notified are not notified again
- If the notification backend is unavailable, the remaining users of that feed are skipped
When a new release of `gitea/runner` is detected, the service also triggers the runner build workflow on the Gitea instance, using `gitea_token`.
## Accepted feed URLs
The scheme and `www.` are optional. Whatever is submitted is converted to the canonical feed URL.
| Type | Accepted input | Stored as |
|---|---|---|
| `royalroad` | `royalroad.com/fiction/118891`<br>`https://www.royalroad.com/fiction/118891/story-title` | `https://www.royalroad.com/fiction/syndication/118891` |
| `gitea` | `gitea.com/gitea/runner`<br>`https://gitea.com/gitea/runner/releases`<br>`https://gitea.com/gitea/runner/releases.rss` | `https://gitea.com/gitea/runner/releases.rss` |
Only repositories under `gitea.com/gitea/` are supported.
## Error Responses
Errors are returned as `{"detail": "<message>"}`.
| Status | Cause | Example `detail` |
|---|---|---|
| `401` | Missing or invalid bearer key | `Unauthorized` |
| `403` | Key is valid but not allowed on this route | `Forbidden` |
| `422` | Invalid `X-User-Id` or body | `type: Input should be 'royalroad' or 'gitea'` |
| `422` | URL does not match the feed type | `A gitea feed must be on gitea.com` |
| `422` | URL has the wrong path | `Unrecognised feed path (expected e.g. royalroad.com/fiction/118891)` |
| `422` | The feed does not exist | `The feed does not exist: <url>` |
| `502` | The feed could not be checked right now | `Could not verify the feed right now, try again later: <url>` |
| `500` | Internal error, or a check run with failed feeds | |
## Notification Format
One message is sent to `backend-api-internal` per subscriber:
```json
{
"user_id": 1,
"message": {
"title": "New Life As A Max Level Archmage",
"body": "Chapter 68 has been released",
"link": "https://www.royalroad.com/fiction/118891/new-life-as-a-max-level-archmage/chapter/2683797/68-carve",
"category": "royal-road",
"timestamp": 1729339200
}
}
```
Gitea releases use the category `utility`. When a subscription has no custom name, the title is the feed title for Royal Road and the repository path (for example `gitea.com/gitea/runner`) for Gitea.
## Monitoring
### Prometheus Metrics