Beginner: Trying to create a new csv every hour

Replit Help
1

Question:

  • I am trying to create a csv file every hour that contains the headlines from CNN
  • I am not able to deploy it. The deployment fails before there are any logs
  • I am a total beginner, so I am sure I am missing something very basic

Repl link:
https://replit.com/@makanamini/Project-Swift?v=1

import os
import requests
from bs4 import BeautifulSoup
import csv
from datetime import datetime
import time

url = "https://www.cnn.com/"


def scrape_headlines():
    now = datetime.now()
    # Format date and time with leading zeros
    date = "{:02d}-{:02d}-{:02d}".format(now.year, now.month, now.day)
    time = "{:02d}{:02d}".format(now.hour, now.minute)
    folder_name = "cnn"
    if not os.path.exists(folder_name):
        os.makedirs(folder_name)
    filename = os.path.join(folder_name, "cnn.com-{}-{}.csv".format(date, time))
    response = requests.get(url)
    soup = BeautifulSoup(response.text, "html.parser")
    headlines = []
    for headline in soup.find_all("span", {"data-editable": "headline"}):
        headlines.append(headline.text.strip())
    with open(filename, "w", newline="") as csvfile:
        writer = csv.writer(csvfile)
        writer.writerow(["Headline", "Rank"])
        rank = 1
        for headline in headlines:
            writer.writerow([headline, rank])
            rank += 1


# Add code to run at the top of the hour
while True:
    current_time = datetime.now().strftime("%M")
    if current_time == "00":
        scrape_headlines()
    time.sleep(60)  # Sleep for 1 minute
2

Sorry for the horrible formatting!

3

Hi @makanamini , welcome to the forums!
Can you format it like this:

```py
```
1 Like
4

Hi there. Did you mean to deploy this via on-demand deployments? It should work if you deploy to a reserved VM. Select the “Background Worker” option.

5

One limitation of our current product is changes you make to the deployed VM’s filesystem will not persist with your development Repl.

In other words, even if you are successful in deploying the repl, the csv files you want won’t save back to the original development Repl.

We’ll have a solution for this soon.

2 Likes