What web scraping actually costs, and the lever that moves the bill 6x

The cost of a scraping job is mostly proxy bandwidth, and two things you control move it far more than which provider you sign with: how defended the target is, which sets your proxy tier, and how heavy each page is, which sets how many gigabytes you burn. Get those two right and the provider choice is a rounding error.

The three proxy tiers, cheapest first

• Datacenter: the cheapest, and fine for sites that barely fight back. It is also the only tier worth self-hosting.
• Residential: billed per GB and the usual default for defended sites. Oxylabs pay-as-you-go starts at 6 dollars per GB and falls to about 2.50 dollars per GB at terabyte volume (list rates, checked June 2026; they move). Here the bandwidth is the whole bill.
• Managed unblocker: handles fingerprints, CAPTCHAs and retries for you and usually bills per successful request. The priciest per unit, but often the only thing that reliably gets through a hard target.

The lever nobody prices in: payload size

On residential proxies you pay for bytes, so page weight is the cost. Fetch the raw HTML, often around 0.3 MB, and you spend a fraction of what a full browser render at roughly 2 MB costs. For the very same records that is about 6 to 7 times the bandwidth, and therefore 6 to 7 times the bill. Most scrapers reach for a headless browser by reflex when the data is already sitting in the markup.

A worked example

Say you scrape 100,000 pages a month off a defended site, so residential, with a 1.4x retry overhead because some attempts fail and still spend bandwidth. As raw HTML at 0.3 MB a page that is about 42 GB, so roughly 252 dollars a month at 6 dollars per GB. Render the same pages in a full browser at 2 MB each and it is about 280 GB, roughly 1,680 dollars a month, for identical data. Same job, about 1,400 dollars of avoidable cost, decided entirely by how you fetch.

The other line the bandwidth math skips: CAPTCHAs

Payload size is one cost the per-GB view hides; the CAPTCHA is the other. On a defended target some requests come back as a challenge, a reCAPTCHA, an hCaptcha or a Cloudflare Turnstile, that a proxy alone will not clear, and solving it is priced per solve rather than per byte, so a bandwidth estimate never sees it. The solving services bill without a subscription at a rate that floats with their load: roughly 0.50 to 1 dollar per 1,000 simple image captchas and about 1 to 3 dollars per 1,000 for reCAPTCHA v2, with reCAPTCHA v3 and Cloudflare Turnstile in the same one-to-three-dollar range (list prices, checked July 2026, they move).

The reason this line is easy to miss is that it scales with how often the target challenges you, not with page weight. Take the same 100,000 pages a month: if the site throws a challenge at one request in five, that is 20,000 solves, roughly 20 to 60 dollars on top of the proxy bill at reCAPTCHA v2 rates. Small next to the bandwidth here, but it climbs fast on a target that challenges most requests, and every failed solve is also a retry that spends bandwidth twice. It is also part of why a managed unblocker or scraping API can be cheaper than its per-request sticker looks: that price already folds the solving in, so you are not stacking a separate solver bill on top. One caveat that ties back to the legal section below: routinely clearing an anti-bot challenge looks less like fetching a public page and more like bypassing a technical block, so it nudges a job toward the higher-risk lane, worth weighing, not only costing.

The line the bandwidth math leaves out: your own time

The proxy bill is the infrastructure cost, and it is the one a calculator can price. It is not the only cost. A self-built scraper is software, and software has to be kept alive: sites redesign their markup, add or rotate defenses, change pagination and rate limits, and any one of those can break a parser or a login flow overnight. So a scraper is not write-once, it is maintained, and for a small or mid-size job the developer hours to build it and keep it running often cost more than the bandwidth it burns. A proxy bill that sits in the low hundreds of dollars a month or less is easy to beat with a few engineering hours, so the time is the line to watch, not the gigabytes.

That is what makes the managed route a real option, not just a lazy one. A managed scraping API bills per successful request and folds the proxy pool, the unblocker and the retries into one price, so you carry no proxy contract and no block-handling code. Zyte API starts around 0.13 dollars per 1,000 simple HTML requests and runs up to a few dollars per 1,000 for browser-rendered or well-defended pages; ScraperAPI floors at about 3 dollars per 1,000 unrendered requests and 7 dollars rendered (list prices, checked June 2026, they move). You usually pay only for successful responses, so the failed-request bandwidth that quietly inflates a self-hosted bill is on the provider, not you. The trade is a higher cost per record in exchange for near-zero maintenance, which is the right trade when your volume is modest or the target fights hard, and the wrong one when volume is high and steady on a target that rarely changes.

Is web scraping even legal?

This is the question the bandwidth math never touches, and it is not one question. The useful split is between the US computer-fraud law and everything else. On the first, US courts have repeatedly found that collecting data which is publicly visible without logging in does not, on its own, break the Computer Fraud and Abuse Act, the anti-hacking statute. hiQ Labs v. LinkedIn set the direction, and in the 2024 Meta v. Bright Data ruling a federal court held that Facebook and Instagram terms do not bar logged-off scraping of public data. Public and logged out is the safest lane there is.

The anti-hacking law is only one door, though, and three others stay open even on public data:

• Terms of service. If you agreed to a site's terms, usually by holding an account or logging in, scraping against them can be a breach of contract. hiQ itself was later found to have breached LinkedIn's user agreement. Scraping public pages while logged out is a much weaker basis for that kind of claim, which is exactly the line the Bright Data ruling drew.
• Copyright. The raw facts in a listing are generally not protected, but copying and republishing protected text, images or a substantial creative arrangement can infringe.
• Data-protection law. Personal data does not stop being personal because it is public. Scraping names, emails and profiles pulls you under the GDPR in the EU, and similar regimes elsewhere, regardless of the computer-fraud question, and for an EU operator that is often the larger exposure.

So the low-risk lane is consistent: public pages, logged out, without bypassing a login or a technical block, without hammering the server, not scraping personal data, and not republishing protected content verbatim. The higher-risk moves are simply the opposite of each. None of this is legal advice, it varies by jurisdiction and by your exact facts, and anything material is worth running past a lawyer. The calculator prices the infrastructure; this is the line that sits outside any price.

When to roll your own

Self-hosting a proxy pool only pays for datacenter-class work on weakly defended targets. The moment you need residential IPs or a managed unblocker you cannot realistically build it, and the upkeep eats the saving. Put your real page count, page weight and target difficulty in and the tool estimates your monthly GB, picks the proxy tier that actually gets through, and prices it across the big providers, so you see the bill before you write the scraper.