- This file contains the specific dependencies required for running the application in Windows Subsystem for Linux (WSL).
- Adding this file ensures that developers working on WSL can easily set up their environment and install all necessary packages.
- This improves consistency across development environments and reduces setup time for new contributors or team members.
- Modified verify_target method to skip updating the target if the NamespaceRequest is already verified and the last verification was within the past day.
- This change reduces unnecessary modifications to the target, improving efficiency and maintaining data integrity.
- Added logging to indicate when target verification is skipped due to verification status.
hopefully this:
* lowers cpu usage, although it can't hurt.
* reduces database churn and locking.
modified: remarkbox/models/namespace_request.py
- Added logic to skip scraping if a NamespaceRequest is already verified and the last scrape was within the past day.
- This change reduces unnecessary scraping, improving efficiency and reducing load on target servers.
- Added logging to indicate when a scrape is skipped due to verification status.
modified: remarkbox/models/namespace_request.py
- Implemented logic to follow HTTP redirects up to a specified maximum depth (max_redirects) in the scrape_target method.
- Added logging to track each redirect step, providing better visibility into the redirect chain.
- Set a limit on the number of redirects to prevent infinite loops in case of circular redirects.
- This change ensures that the UUID is checked on the final destination page, improving the accuracy of the scraping process.
modified: remarkbox/models/namespace_request.py
