Hi I’ve been exposed to python for a long time and I had never track any site before. This time I decide to give https://deevid.ai/ a try. But before that, I want to know if it is legal to crawl a site like this?
I’ve created the codes in the following, will they work?
import requests
url = “https://deevid.ai/”
response = requests.get(url)
if response.status_code == 200:
print(“Page successfully fetched!”)
soup = BeautifulSoup(response.content, ‘html.parser’)
print(“Page Title:”, soup.title.string)
#Find all links on the page
links = soup.find_all('a', href=True)
print("\nLinks found on the page:")
for link in links:
print(link['href'])
# Find all paragraph texts on the page
paragraphs = soup.find_all('p')
print("\nParagraphs found on the page:")
for paragraph in paragraphs:
print(paragraph.get_text())
else:
print(f"Failed to retrieve the webpage. Status code: {response.status_code}")