Prompts/Coding/Python Web Scraping & Data Pipeline Automation

Python Web Scraping & Data Pipeline Automation

Build asynchronous Python scrapers with Playwright, BeautifulSoup, and Pandas with automatic retry and rate-limiting.

AAdmin
about 3 hours ago
5251 views
1739 copies
Python Web Scraping & Data Pipeline Automation

The Prompt

You are a Senior Data Engineer. Write a production-grade, asynchronous Python script to scrape data from: "[Target Site / Domain Type, e.g. E-commerce product listings]". Data Fields Required: - [Field 1, e.g. Title] - [Field 2, e.g. Price] - [Field 3, e.g. Rating & Review Count] - [Field 4, e.g. Image URL] Requirements: - Use Playwright / aiohttp / BeautifulSoup for reliable modern web rendering. - Implement exponential backoff, rotating User-Agents, and polite rate limiting. - Clean and normalize extracted data using Pandas. - Export results to both CSV and JSON formats with comprehensive error logging.

Use this prompt with Gemini.

Frequently Asked Questions

How do I use this prompt in Gemini?

Click 'Copy Prompt', paste it into Gemini, and adjust any bracketed placeholders to match your desired topic or context.

Is this prompt free for commercial work?

Yes, 100% free to copy, modify, and utilize for personal, client, and commercial projects without royalties.

Can I use this prompt in ChatGPT or Gemini?

Yes! While tailored for Gemini, this prompt adapts smoothly across OpenAI ChatGPT, Google Gemini, Anthropic Claude, and Midjourney.

Sponsored Content
Google AdSense Slot #9012
High-converting responsive display ad

How to use this prompt

  • Click the Copy Prompt button.
  • Open Gemini.
  • Paste into the chat or image prompt input.
  • Adjust brackets like [Insert Topic].
  • Press generate to create your content!
Advertisement
Google AdSense Slot #0123
High-converting responsive display ad
Advertisement
Google AdSense Slot #1234
High-converting responsive display ad

Related Prompts

Awesome Prompt for task 2

Awesome Prompt for task 2

120
45
Awesome Prompt for task 12

Awesome Prompt for task 12

120
45
Awesome Prompt for task 22

Awesome Prompt for task 22

121
45