Simple Scraping API with zenscrape (Sponsored)

Celý článek

So many developers believe that scraping a URL’s content is easy — just a simple curl and you’re done, right? Nope. CAPTCHAs, rate limitations, origin-specific limitations — they make getting content off the web a nightmare. Luckily we now have services like zenscrape which do the heavy lifting for you — the perfect middle man between you and the desired content!

Quick Hits

• You can sign up for free
• Request a webpage from any location around the world
• Bypass CAPTCHAs and other listing limits
• Get access to rendered JavaScript contents; perfect for Vue and React driven websites
• Satisfies important use cases: downloading reviews, feedback, pricing, and other data
• Generate code for Node.js, PHP, cURL, and other utilities to use zenscrape’s API
• Use the visual Web Scraper tool to visually select the data you want

After signing up for free, you’re ready to take advantage of zenscrape’s easy to use API. Armed with an API key, the easiest use of zenscrape is:

curl "https://app.zenscrape.com/api/v1/get?apikey=MY_API_KEY&url=https://davidwalsh.name"

Aside from providing the obvious URL, there are other parameters you can pass with your request:

• location: The country location code from which the request should be made
• render: Use a headless browser to render the page
• premium: Allows you to use residential addresses to make scraper detection more difficult

These options can be appended to the API request as follows:

curl "https://app.zenscrape.com/api/v1/get\
?apikey=MY_API_KEY\
&url=https://davidwalsh.name\
&location=au\
render=1"

The render option is especially useful, allowing you to download not only the raw source code but the source code after the page has been modified by its hosted JavaScript files — especially since so many are based on React, Vue, Angular, jQuery and other frameworks.

Another really nice feature that zenscrape provides is a code snippet builder that allows you to save time:

Web Scraper Tool

APIs are great for using zenscrape but visual tools are sometimes more helpful. zenscrape provides a visual point and click tool to select, store, and export data from URLs of your choosing:

You can schedule recurring scrapes so you always have the most up to date information.

Imagine always knowing competitor pricing and being able to beat them at all times!

zenscrape is a really cool service for those that want to scrape content reliably. Building tools to try to deal with the hassle of CAPTCHAs and other limitations isn’t worth the time. Have a look at zenscrape and th eir other tool, zenserp!

The post Simple Scraping API with zenscrape (Sponsored) appeared first on David Walsh Blog.