Custom Web Scraping Playbook Using Python

January 13, 2026

Tools used

Download playbook

Enter your email

By clicking download you subscribe to our marketing communications.

A workflow that uses Python-based scraping to extract clean, structured data from public websites with full control.

Elena Mahmutaj

How it works

Step 1: Set up the python environment

Install Python and ensure it is added to your system PATH.
Verify installation by running:
- python --version
Install required libraries:
- selenium
- webdriver-manager
- beautifulsoup4
- requests
- pandas

Step 2: Save the scraping script locally

Open the script linked in the playbook document.
Copy the full Python code.
Save it as a .py file.

Example:

scraper.py

‍

Step 3: Configure the target website

Open the .py file in VS Code or Cursor.
Locate the target URL variable:
- url = "YOUR_TARGET_WEBSITE_URL_HERE"
Replace it with the actual website URL.
Save the file.

‍

Step 4: Run the scraper

Open Command Line (Windows) or Terminal (Mac).
Navigate to the folder containing the script.
Run:
- python scraper.py‍
The script will:
- Open Chrome
- Load the website
- Extract the defined elements
- Save the results locally

‍

Step 5: Review the scraped output

Locate the output files in the same folder:
- .json file for structured data
- .csv file for spreadsheet use
Open the CSV file in Excel, Google Sheets or Clay.
Review a sample of rows for accuracy and completeness.

‍

Step 6: Customize scraping logic as needed (Optional)

Modify:
- Target URL
- XPath or CSS selectors
- Data fields to extract (names, links, emails, phone numbers)
Use browser inspection tools to identify stable selectors.
Share page HTML or selector hints with Claude to improve accuracy.

‍

Step 7: Handle failures and iterate (Optional)

Increase wait times if elements load slowly.
Update selectors if the site layout changes.
Add delays to reduce bot-detection risk.
Re-run the script after each change.

‍

Custom Web Scraping Playbook Using Python

Scroll to zoom in and out, drag to navigateUse 2 fingers to drag, zoom in and out

The Content RAG Agent Playbook

A workflow that feeds data into a database, connects it to a RAG agent built in n8n, and automatically produces content and video drafts from your knowledge base.

CCoonntteennttAAII RReevvOOppss

Angelee Carandang

May 15, 2026

Automated News Publishing Playbook

A workflow that sources daily news on any topic, stores it in Notion for manual approval, and auto-publishes approved stories to Webflow using Claude Cowork and MCP-triggered webhooks.

AAII RReevvOOppss

Pari Tomar

April 22, 2026

Automated LinkedIn-to-Blog Publishing Playbook

Automated LinkedIn-to-Blog Publishing Playbook

A workflow that monitors top LinkedIn creators, converts their best posts into brand-voiced blogs with Claude, stores them in Notion for manual approval, and auto-publishes approved blogs to Webflow.