Custom Web Scraping Playbook Using Python

• Open the script linked in the playbook document.
• Copy the full Python code.
• Save it as a .py file.

Example:

• scraper.py

‍

• Open the .py file in VS Code or Cursor.
• Locate the target URL variable:
  
  • url = "YOUR_TARGET_WEBSITE_URL_HERE"

• Replace it with the actual website URL.
• Save the file.

‍

• Open Command Line (Windows) or Terminal (Mac).
• Navigate to the folder containing the script.
• Run:
  
  • python scraper.py‍
• The script will:
  
  • Open Chrome
  • Load the website
  • Extract the defined elements
  • Save the results locally

‍

• Locate the output files in the same folder:
  • .json file for structured data
  • .csv file for spreadsheet use
• Open the CSV file in Excel, Google Sheets or Clay.
• Review a sample of rows for accuracy and completeness.

‍

• Modify:
  • Target URL
  • XPath or CSS selectors
  • Data fields to extract (names, links, emails, phone numbers)
• Use browser inspection tools to identify stable selectors.
• Share page HTML or selector hints with Claude to improve accuracy.

‍

• Increase wait times if elements load slowly.
• Update selectors if the site layout changes.
• Add delays to reduce bot-detection risk.
• Re-run the script after each change.

‍