Florian
d2586defda
All checks were successful
Build & Publish to GHCR / build (push) Successful in 44s
140 lines
3.9 KiB
Markdown
140 lines
3.9 KiB
Markdown
# Backend Push Notifications
|
|
|
|
|
|
## Overview
|
|
|
|
This service consumes messages from RabbitMQ, retrieves encrypted push tokens from a MySQL database, decrypts them, and sends notifications via the Expo Push API. It includes comprehensive error handling, automatic retries, dead-letter queuing, and Prometheus metrics.
|
|
|
|
|
|
## Features
|
|
|
|
- **Asynchronous Architecture**: Built with `asyncio`, `aio_pika`, and `aiomysql` for high-performance concurrent operations
|
|
- **Automatic Retry Logic**: Failed notifications are automatically retried up to 5 times with exponential backoff
|
|
- **Dead Letter Queue (DLQ)**: Messages that exceed retry limits are moved to a DLQ for manual inspection
|
|
- **Health Monitoring**: Database connection pool health checks every 60 seconds
|
|
- **Prometheus Metrics**: Real-time monitoring of queue depths and message processing stats
|
|
- **Token Management**: Automatically expires invalid or unregistered device tokens
|
|
|
|
## Architecture
|
|
|
|
```
|
|
RabbitMQ → Consumer → Database Lookup → Token Decryption → Expo Push API
|
|
↓ ↓
|
|
Retry Queue ←─────────── Delivery Validation ──────┘
|
|
↓
|
|
Dead Letter Queue
|
|
```
|
|
|
|
## Prerequisites
|
|
|
|
- Python 3.10+
|
|
- MySQL/MariaDB database
|
|
- RabbitMQ server
|
|
- Expo Push Notification service account
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## Running the Service
|
|
|
|
```bash
|
|
python rabbitmq_handler.py
|
|
```
|
|
|
|
The service will:
|
|
1. Connect to the MySQL database
|
|
2. Establish RabbitMQ connection and declare queues
|
|
3. Start consuming messages
|
|
4. Begin Prometheus metrics server on port 9000
|
|
|
|
|
|
## Configuration
|
|
|
|
### Database Manager (`db.py`)
|
|
|
|
- **Pool Size**: 5 connections (configurable)
|
|
- **Health Check Interval**: 60 seconds
|
|
- **Auto-commit**: Enabled
|
|
- **Cursor Type**: DictCursor for convenient row access
|
|
|
|
### RabbitMQ Consumer (`rabbitmq_handler.py`)
|
|
|
|
- **Queue Name**: `notifications`
|
|
- **Retry Queue**: `notifications_retry`
|
|
- **Dead Letter Queue**: `notifications_dlq`
|
|
- **Max Retries**: 5 attempts
|
|
- **Retry TTL**: 30 seconds
|
|
- **Routing Key**: `notify.user.*` (standard), `notify.user.retry` (retries)
|
|
|
|
### Notification Sender (`send_notification.py`)
|
|
|
|
- **Endpoint**: `https://exp.host/--/api/v2/push/send`
|
|
- **Max Retries**: 5 attempts
|
|
- **Timeout**: 5 seconds per request
|
|
- **Backoff Strategy**: Exponential (2^attempt seconds)
|
|
|
|
### Message Format
|
|
|
|
Messages should be published to RabbitMQ with the following structure:
|
|
|
|
```json
|
|
{
|
|
"title": "New Message",
|
|
"body": "You have a new notification",
|
|
"data": {
|
|
"category": "utility",
|
|
"link": "https://example.com",
|
|
"timestamp": "1760734800"
|
|
}
|
|
```
|
|
|
|
**Routing Key**: `notify.user.<user_id>` or `notify.user.retry` for retries
|
|
|
|
### Retry Messages
|
|
|
|
Retry messages include additional metadata:
|
|
|
|
```json
|
|
{
|
|
"title": "Notification Title",
|
|
"body": "Notification message body",
|
|
"uuid": "device-uuid-here",
|
|
"retry_count": 2
|
|
}
|
|
```
|
|
|
|
## Delivery Validation
|
|
|
|
The service validates delivery results from the Expo Push API:
|
|
|
|
| Status | API Status | Action |
|
|
|--------|-----------|--------|
|
|
| `ok` | `ok` | Success - metric incremented |
|
|
| `ok` | `error` (DeviceNotRegistered) | Token expired in database |
|
|
| `ok` | `error` (other) | Sent to DLQ |
|
|
| `error` | - | Sent to DLQ |
|
|
| `failure` | - | Requeued for retry |
|
|
|
|
## Monitoring
|
|
|
|
### Prometheus Metrics
|
|
|
|
Available at `http://localhost:9000/metrics`:
|
|
|
|
- `msg_published_total`: Successfully delivered notifications
|
|
- `msg_retry_total{queue_name, uuid, retry_count}`: Retry attempts
|
|
- `msg_failed_total`: Failed deliveries
|
|
- `queue_depth{queue_name}`: Current queue message counts
|
|
|
|
## Troubleshooting
|
|
|
|
### Connection Issues
|
|
- `aiomysql` uses `PyMySQL` which requires the `mysql_native_password` plugin to be used. Can be checked and changed via:
|
|
```sql
|
|
SELECT user, host, plugin FROM mysql.user WHERE user='backend-push-notifications';
|
|
ALTER USER 'backend-push-notifications'@'%' IDENTIFIED WITH mysql_native_password BY 'db_password';
|
|
```
|