@here
to those who would like to assist in backing up the website:
Here’s a step-by-step guide formatted in Markdown to share with others for using Replit to run the spn.sh script, setting up an Internet Archive account, and distributing tasks among team members.
Step-by-Step Guide for Archiving URLs Using Replit and spn.sh
1. Setting Up on Replit
-
Create an Account or Log In:
- Visit replit.com and sign up for an account or log in if you already have one.
-
Create a New Replit Project:
- Click on the New Repl button.
- Select Bash as the language for your project.
- Name your project (e.g.,
url-archiver) and click Create Repl.
2. Create an Internet Archive Account and Obtain an API Key
-
Sign Up for an Account:
- Go to archive.org and create an account or log in if you already have one.
-
Get Your API Key:
- Navigate to archive.org/account/s3.php.
- Generate an API key. It will be in the format
access_key:secret_key.
3. Download the Provided Text File
-
Download the URL File:
- Ensure you have the text file (
urls.txt) containing the list of URLs. - Upload the
urls.txtfile to your Replit project by dragging and dropping it or by using the Upload File button.
urls.txt: https://files.catbox.moe/pryse8.txt
- Ensure you have the text file (
4. Run the spn.sh Script on Replit
-
Clone the Script from GitHub:
- In your Replit shell, run:
git clone https://github.com/overcast07/wayback-machine-spn-scripts.git cd wayback-machine-spn-scripts chmod a+x spn.sh
- In your Replit shell, run:
-
Run the Script:
- Execute the script using your API key:
./spn.sh -a "your_access_key:your_secret_key" ~/urls.txt - Replace
your_access_key:your_secret_keywith the API key you obtained earlier.
- Execute the script using your API key:
output should start looking like:
5. Distribute Tasks Among Team Members
- The total number of links is approximately 24,600, and we need to split this among 20 people.
- Each person should take approximately 1,230 links. Use the ranges below to avoid overlap:
| Person | Link Range |
|---|---|
| 1 | 1–1,230 |
| 2 | 1,231–2,460 |
| 3 | 2,461–3,690 |
| 4 | 3,691–4,920 |
| 5 | 4,921–6,150 |
| 6 | 6,151–7,380 |
| 7 | 7,381–8,610 |
| 8 | 8,611–9,840 |
| 9 | 9,841–11,070 |
| 10 | 11,071–12,300 |
| 11 | 12,301–13,530 |
| 12 | 13,531–14,760 |
| 13 | 14,761–15,990 |
| 14 | 15,991–17,220 |
| 15 | 17,221–18,450 |
| 16 | 18,451–19,680 |
| 17 | 19,681–20,910 |
| 18 | 20,911–22,140 |
| 19 | 22,141–23,370 |
| 20 | 23,371–24,600 |
- Each person should edit the
urls.txtfile to keep only their assigned range of URLs.
this is the best method i can think of right now that would avoid having to pay for a service. (not taking into account we might get limited by Repilt but if you know how and can you can run this on your local machine as well.
thank you thank you thank you thank you thank you thank you thank you thank you thank you thank you x 1million times.