I previously posted this on Python code help, but realized that this may be a Replit bug. I’m kind of new to replit, and I was trying to gather data from this website: https://hadith.inoor.ir/ar/home via BeautifulSoup, however I get the following error: “Connection to hadith.inoor.ir timed out.”
At first I assumed that this was an issue with the JavaScript, so I ran it using Selenium but received the same error. Then I ran it using VS Code and it worked exactly how I wanted it to. Which led me to the conclusion that Replit was the problem.
Currently my hypothesis is that iranian websites have a hard time working with Replit, I say this because I’ve tried multiple other iranain websites and received the same error. I personally don’t really care about the iranian thing, I just need the data on the website. Thanks!
BeautifulSoup:
import requests
from bs4 import BeautifulSoup
def sol(link):
r = requests.get(link)
soup = BeautifulSoup(r.content, "html.parser")
title = soup.find("h2")
title = title.get_text()
print(title)
print(sol("https://hadith.inoor.ir/fa/home"))
Selenium:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--disable-blink-features=AutomationControlled')
driver = webdriver.Chrome(options=chrome_options)
driver.get("https://hadith.inoor.ir/ar/home")
text = driver.find_element(By.CLASS_NAME, "hadith-title ng-tns-c261-4 ng-star-inserted").text
print(text)
Expected Behavior:
ارائه پرونده جامع حدیث
None
Current Behavior:
I would attach images, but I’m getting an error “Sorry, you can’t embed media items in a post.”
In output I’m getting “This site can’t be reached”. And in the console I’m getting the following:
Traceback (most recent call last):
File "main.py", line 12, in <module>
driver.get("https://hadith.inoor.ir/ar/home")
File "/home/runner/dynamic/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 449, in get
self.execute(Command.GET, {"url": url})
File "/home/runner/dynamic/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
self.error_handler.check_response(response)
File "/home/runner/dynamic/venv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 300.000
(Session info: chrome=108.0.5359.94)
Stacktrace:
#0 0x559f995802a3 <unknown>
#1 0x559f9933ef77 <unknown>
#2 0x559f993294e9 <unknown>
#3 0x559f99329225 <unknown>
#4 0x559f99327dc6 <unknown>
#5 0x559f99328244 <unknown>
#6 0x559f99340e88 <unknown>
#7 0x559f993b3b05 <unknown>
#8 0x559f9939bb32 <unknown>
#9 0x559f993b347c <unknown>
#10 0x559f9939b903 <unknown>
#11 0x559f9936eece <unknown>
#12 0x559f9936ffde <unknown>
#13 0x559f995d063e <unknown>
#14 0x559f995d3b79 <unknown>
#15 0x559f995b689e <unknown>
#16 0x559f995d4a83 <unknown>
#17 0x559f995a9505 <unknown>
#18 0x559f995f5ca8 <unknown>
#19 0x559f995f5e36 <unknown>
#20 0x559f99611333 <unknown>
#21 0x7f2a9f50be86 start_thread