First version

This commit is contained in:
florian committed 2026-11-02 19:55:04 +01:00
commit a1072dcb97
6 files changed
+348

No files matched your search

+114
View File
@@ -0,0 +1,114 @@
# Getting Started
## 1. Create a Git repository
Start by creating a repository on GitHub or another Git hosting platform.
- Give it a descriptive name.
- Add a `README.md` explaining what the project does.
- Add a `.gitignore` to prevent unnecessary files from being committed. Github has a Python template you should use, which prevents cache or build data to be commited
- Commit changes regularly with descriptive commit messages.
## 2. Recommended project structure
```text
my-project/
├── src/
│ ├── main.py
│ ├── utils.py
│ └── ...
├── .venv/
├── .gitignore
├── requirements.txt
└── README.md
```
## 3. requirements.txt
Use `requirements.txt` to keep track of external Python packages your project depends on.
Example:
```text
requests==2.32.5
beautifulsoup4==4.14.2
```
- One package per line.
- Pin versions to make installations reproducible.
- Update the file whenever you add or remove dependencies.
Install all dependencies with:
```bash
pip install -r requirements.txt
```
# Next step: Set up a virtual environment
A virtual environment (`venv`) creates an isolated Python environment for your project.
This means:
- Each project can have its own dependencies.
- Installing packages won't affect your system Python installation.
- Different projects can use different package versions.
## 1. Check your Python installation
```bash
python3 --version
```
## 2. Create the virtual environment
Navigate to your project directory:
```bash
cd my-project
```
Create the environment:
```bash
python3 -m venv .venv
```
This creates a `.venv/` directory containing the isolated Python environment.
## 3. Activate the virtual environment
```bash
source .venv/bin/activate
```
Once activated, your terminal should show something similar to:
```text
(.venv) user@computer:~/my-project$
```
## 4. Install dependencies
With the virtual environment activated:
```bash
pip install -r requirements.txt
```
You can now install additional packages:
```bash
pip install requests
```
If you add a new dependency, update `requirements.txt` by running `pip freeze` which lists all installed packages.:
```bash
pip freeze > requirements.txt
```
## 5. Deactivate the environment
When you're finished working or want to change to a different virtual environment:
```bash
deactivate
```
+161
View File
@@ -0,0 +1,161 @@
# Backend API – Getting Started
## 1. Project overview
The `backend-api` is a small REST API responsible for receiving, storing and delivering notifications.
It uses:
- **FastAPI:** Python web framework for building REST APIs.
- **Uvicorn:** Server used to run the FastAPI application.
- **SQLite:** Lightweight database for storing notifications.
- **Pydantic:** Data validation for incoming API requests (included with FastAPI).
## 2. API endpoints
The application requires two POST endpoints. POST is used when the application call changes something, GET when it just retrieves information.
| Method | Endpoint | Description |
|---|---|---|
| POST | `/notifications` | Receive and store a notification. |
| POST | `/notifications/deliver` | Retrieve pending notifications and mark them as delivered. |
### Receiving notifications
`POST /notifications`
Accepts a JSON payload containing the notification information.
Example request:
```json
{
"title": "New notification",
"body": "Something happened!",
"link": "https://example.com",
"timestamp": 1791626400
}
```
The API validates the payload and saves it into the `notifications` database.
Each notification should have a `delivered` flag, initially set to `false`.
### Getting notifications
`POST /notifications/deliver`
This endpoint retrieves all notifications where `delivered = false`.
The application should:
1. Query the database for undelivered notifications.
2. Set their `delivered` flag to `true`.
3. Commit the database transaction.
4. Return the retrieved notifications as JSON.
Example response:
```json
[
{
"id": 1,
"title": "New notification",
"body": "Something happened!",
"link": "https://example.com",
"timestamp": 1791626400,
"delivered": true
}
]
```
If there are no pending notifications, return an empty array.
**Important:** The database update should happen before returning the notifications. This prevents the same notifications from being delivered again on subsequent requests.
## 3. Database structure
For the initial implementation, use SQLite.
The database will be stored in a local file called `notifications.db`.
The `notifications` table should contain:
| Column | Type | Description |
|---|---|---|
| `id` | INTEGER | Unique notification identifier (primary key). |
| `title` | TEXT | Notification title. |
| `body` | TEXT | Notification content. |
| `link` | TEXT | Optional URL associated with the notification. |
| `timestamp` | INTEGER | Unix timestamp of the notification. |
| `delivered` | BOOLEAN | Indicates whether the notification was delivered. |
---
# 4. Minimal working implementation
The following implementation gives you a baseline for starting the server
## requirements.txt
```text
fastapi
uvicorn
```
## src/main.py
```python
from contextlib import asynccontextmanager
from fastapi import FastAPI
from pydantic import BaseModel #use this to create the Notification class
from db_module import initialize_database, get_database #Implement these yourself with SqLite3
@asynccontextmanager
async def lifespan(app: FastAPI):
initialize_database()
yield
app = FastAPI(lifespan=lifespan)
@app.post("/notifications")
def receive_notification(notification: Notification):
@app.post("/notifications/deliver")
def deliver_notifications():
```
## 6. Run the application
Make sure the virtual environment is activated and the dependencies are installed.
Start the application from the project root:
```bash
uvicorn src.main:app --host 0.0.0.0 --port 8100 --reload
```
Explanation:
| Argument | Description |
|---|---|
| `src.main:app` | Imports the FastAPI instance called `app` from `src/main.py`. |
| `--host 0.0.0.0` | Makes the server accessible outside localhost. |
| `--port 8100` | Port on which the API listens. |
| `--reload` | Automatically restarts the server when source files change. |
The `--reload` option should only be used during development.
## 7. Test the application
FastAPI automatically generates interactive API documentation.
Open:
http://localhost:8100/docs
You can use the Swagger UI to execute both POST endpoints without installing additional tools.
+25
View File
@@ -0,0 +1,25 @@
# YouTube Notification Service – Overview
You need to use an [external Website like this one](https://channelsurfer.tv/tools/youtube-rss-feed/) to get the RSS feed ID for a channel.
## 1. Order of events
1. **Connect to the database:** Establish a connection to the database containing the subscribed YouTube channels.
2. **Retrieve subscriptions:** Fetch all subscribed YouTube RSS feed URLs from the database.
3. **Fetch RSS feeds:** Use `feedparser` to query each subscribed feed.
4. **Check for updates:** Compare the latest feed entries against the last known entry stored in the database.
5. **Send notifications:** If a new video is found, send its information (`title`, `body`, `link`, `timestamp`) to the `backend-api` using a POST request.
6. **Update database:** After successfully sending the notification, update the stored latest video information to prevent duplicate notifications.
7. **Close database connection:** Close the database connection once all subscriptions have been processed.
## 2. Dependencies
- **feedparser:** Parse YouTube RSS feeds and extract video information.
- **requests:** Send HTTP POST requests to the `backend-api`.
- **Database driver:** Connect to and query the database.
## 3. Things to improve once it works
- Handle unavailable RSS feeds without stopping the entire service.
- Handle failed API requests so notifications aren't accidentally marked as processed.
- Log errors and processing information using Python's `logging` module. Use a LOG_LEVEL to set it to either debugging or production(info)
+28
View File
@@ -0,0 +1,28 @@
# AsuraScans Monitoring – Overview
## 1. Order of events
1. **Fetch comic URL:** Use `requests.get(baseurl)` to retrieve the current comic page URL. The URL may change frequently, so avoid hardcoding the chapter page URL.
2. **Parse website:** Use `BeautifulSoup` to parse the HTML and extract the chapter information.
3. **Extract chapter data:** The chapter information is stored inside the HTML element's `props` attribute as JSON.
```python
soup = BeautifulSoup(response.text, "html.parser")
element = soup.select_one("[props*='chapters']")
props = json.loads(element.get("props"))
```
4. **Check for updates:** Compare the latest chapter against the last known chapter stored in the database.
5. **Create notification:** If a new chapter is found, extract its title, URL and timestamp.
6. **Send notification:** Convert the chapter information into the notification payload (`title`, `body`, `link`, `timestamp`) and send it to the `backend-api` using a POST request.
7. **Update database:** After successfully sending the notification, update the stored latest chapter information to prevent duplicate notifications.
## 2. Dependencies
- **requests:** Fetch the comic website and send HTTP POST requests to the `backend-api`.
- **beautifulsoup4:** Parse the website HTML and extract chapter information.
- **Database driver:** Connect to and query the database.
- **json:** Parse the chapter data stored in the HTML `props` attribute (included in Python's standard library).
- **datetime:** Convert chapter timestamps if necessary (included in Python's standard library).
+10
View File
@@ -0,0 +1,10 @@
# 8. Future improvements
Once the basic setup is working, consider implementing the following:
- **MySQL:** Replace SQLite with MySQL for persistent production database storage.
- **Authentication:** Protect endpoints using API keys or token-based authentication. The [backend-api](https://git.gansejunge.com/notifier/backend-api/src/branch/main/src/main.py) has an example
- **Logging:** Replace `print()` statements with Python's `logging` module.
- **Error handling:** Handle database connection failures and invalid requests.
- **Configuration:** Move database paths, ports and other settings into environment variables.
- **Deployment** - Use Docker with docker-compose to run self hosted images from
+10
View File
@@ -0,0 +1,10 @@
# Guide for Kaash
- [1. Getting started](1.getting-started.md)
- [2. The backbone: Backend-API](2.backend-api.md)
- [3. First Service: Get Youtube updates](3.youtube-service.md)
- [4. AsuraScans monitoring](4.asura-scans-monitoring.md)
- [5. Improvements for the future](5.Improvements.md)