28 lines
1.6 KiB
Markdown
28 lines
1.6 KiB
Markdown
|
||
# AsuraScans Monitoring – Overview
|
||
|
||
## 1. Order of events
|
||
|
||
1. **Fetch comic URL:** Use `requests.get(baseurl)` to retrieve the current comic page URL. The URL may change frequently, so avoid hardcoding the chapter page URL.
|
||
2. **Parse website:** Use `BeautifulSoup` to parse the HTML and extract the chapter information.
|
||
3. **Extract chapter data:** The chapter information is stored inside the HTML element's `props` attribute as JSON.
|
||
|
||
```python
|
||
soup = BeautifulSoup(response.text, "html.parser")
|
||
|
||
element = soup.select_one("[props*='chapters']")
|
||
props = json.loads(element.get("props"))
|
||
```
|
||
|
||
4. **Check for updates:** Compare the latest chapter against the last known chapter stored in the database.
|
||
5. **Create notification:** If a new chapter is found, extract its title, URL and timestamp.
|
||
6. **Send notification:** Convert the chapter information into the notification payload (`title`, `body`, `link`, `timestamp`) and send it to the `backend-api` using a POST request.
|
||
7. **Update database:** After successfully sending the notification, update the stored latest chapter information to prevent duplicate notifications.
|
||
|
||
## 2. Dependencies
|
||
|
||
- **requests:** Fetch the comic website and send HTTP POST requests to the `backend-api`.
|
||
- **beautifulsoup4:** Parse the website HTML and extract chapter information.
|
||
- **Database driver:** Connect to and query the database.
|
||
- **json:** Parse the chapter data stored in the HTML `props` attribute (included in Python's standard library).
|
||
- **datetime:** Convert chapter timestamps if necessary (included in Python's standard library). |