cuz-guide/4.asura-scans-monitoring.md

AsuraScans Monitoring – Overview

1. Order of events

1. Fetch comic URL: Use requests.get(baseurl) to retrieve the current comic page URL. The URL may change frequently, so avoid hardcoding the chapter page URL.

2. Parse website: Use BeautifulSoup to parse the HTML and extract the chapter information.

3. Extract chapter data: The chapter information is stored inside the HTML element's props attribute as JSON.

   soup = BeautifulSoup(response.text, "html.parser")
   
   element = soup.select_one("[props*='chapters']")
   props = json.loads(element.get("props"))
   

4. Check for updates: Compare the latest chapter against the last known chapter stored in the database.

5. Create notification: If a new chapter is found, extract its title, URL and timestamp.

6. Send notification: Convert the chapter information into the notification payload (title, body, link, timestamp) and send it to the backend-api using a POST request.

7. Update database: After successfully sending the notification, update the stored latest chapter information to prevent duplicate notifications.

2. Dependencies

• requests: Fetch the comic website and send HTTP POST requests to the backend-api.
• beautifulsoup4: Parse the website HTML and extract chapter information.
• Database driver: Connect to and query the database.
• json: Parse the chapter data stored in the HTML props attribute (included in Python's standard library).
• datetime: Convert chapter timestamps if necessary (included in Python's standard library).