Guides

How to Scrape eBay with Python: API, Listings, CSV and Proxies

Learn how to scrape eBay listings with Python using an approved data source, validate prices and variants, export CSV, and configure a SOCKS5 proxy when needed.

Gideon ReedGideon Reed19 min read

For developers and cross-border sellers researching offers, the practical way to scrape eBay data is to retrieve listings from an approved source, retain product and delivery context, and export records that separate usable amounts from offers needing review. This tutorial takes you through application authentication, listing retrieval, pagination, CSV output and optional SOCKS5 route checks.

Start here: save the complete Python program below, run --self-test, then run --demo to produce a CSV and a dated run log. The displayed run results use the program’s local API-shaped fixtures. Switch to --live with an approved production application for API responses; the same normalization and export functions handle both inputs. Authorized saved HTML is a separate second path.

In our local run, two fixture pages containing four records produced three unique rows, one skipped duplicate, one numeric-ready offer and two review rows. Sixteen normalization and pagination checks passed. The following steps show the commands, output and why those rows were handled differently.

execution-results

1. Choose the source before writing a crawler

eBay’s User Agreement requires prior express permission for automated access. API access has separate conditions. Choose the route that covers your intended use rather than treating a public page or proxy subscription as permission.

Source What it can support Setup and operating trade-off
Approved Browse API Structured current-offer discovery and supported detail fields Application access, OAuth, quota and schema handling
Authorized saved HTML Parsing the fields present in a permitted capture Selector maintenance; preserve original capture context
A maintained data service Contracted fields and freshness Check provenance, scope, validation cost and delivery terms
Proxy connection A configured network exit for an authorized workflow Connection, geography and session checks; not a dataset by itself

Which eBay data source should I use for price monitoring?

Use a structured approved source when it supplies the required fields and delivery context. The Browse API search reference documents q, limit, offset and listing-summary fields. The page size can be up to 200; one result set is limited to 10,000 items. The tutorial deliberately caps its own run at three pages rather than trying to exhaust a catalogue.

Data need Search summaries Item detail / another approved source
Title, item ID, displayed price Extract supported returned fields Request more detail only when necessary
Condition and seller business identifier Retain available fields and missing-value flags Resolve additional offer details through approved methods
Exact selected variant Preserve the returned item/variation identity A summary is not proof that every variant was enumerated
Shipping / delivery terms Provide destination context; review absent or ambiguous options Confirm the applicable offer and destination
Completed sale, realized revenue or review history Do not infer from an asking price or bid A separate approved dataset and use case

2. Install and prepare the application

For live access, obtain developer credentials and confirm Buy API production eligibility for the application. Sandbox access and production access are different. The program below uses production endpoints and the application-token flow described in eBay’s OAuth client-credentials guide.

On Windows PowerShell:

powershellpowershell
py -3 -m venv .venv
.\.venv\Scripts\python.exe -m pip install "requests[socks]" beautifulsoup4

If the Python Launcher is absent but python --version identifies your intended Python 3 installation, use python -m venv .venv for the first command. After that, keep using the explicit virtual-environment interpreter.

On macOS or Linux:

bashbash
python3 -m venv .venv
.venv/bin/python -m pip install 'requests[socks]' beautifulsoup4

Save the complete program in section 4 as ebay_browse.py. Environment variables for a live run are:

Variable Meaning
EBAY_CLIENT_ID Approved production application’s client ID
EBAY_CLIENT_SECRET Its client secret, supplied through your local secret-management process
EBAY_PROXY_URL Optional generated proxy URL; omit for a direct API connection
IPINFO_TOKEN IPinfo Lite lookup token, required by this program only for a configured proxy-route check

The program mints a token with the client-credentials grant and keeps it in memory. It does not write the token or secret to CSV or logs. Authentication uses a separate direct HTTPS session; the optional proxy applies to the research session, not the OAuth credential exchange.

3. Follow four key operations

These excerpts explain the main flow; the complete program follows them.

Request: the live branch sends query, page size and offset, plus marketplace and destination headers.

pythonpython
params = dict(q=args.query, limit=args.limit, offset=offset)
headers = {
    'Authorization': 'Bearer ' + access,
    'X-EBAY-C-MARKETPLACE-ID': args.marketplace,
    'X-EBAY-C-ENDUSERCTX': 'contextualLocation=' + quote(
        f'country={args.country},zip={args.postal}', safe=''),
}

Parse: read each returned summary and record the time after that page has been obtained. Do not assign one timestamp before all requests.

pythonpython
observed = now()
items = payload.get('itemSummaries', [])
row = normalize(item, context)

Validate: use Decimal amounts; keep auctions, missing shipping and currency conflicts out of the numeric-ready set.

pythonpython
pv, sv = amount(price.get('value')), amount(shipping.get('value'))
subtotal = pv + sv if not reasons else None

Export: retain the first successful appearance of each item/variation, preserve its original page and timestamp, and write a new CSV rather than overwrite an earlier run.

pythonpython
key = (row['item_id'], row['variation_id'])
if key in seen:
    duplicates += 1
    continue

The live loop stops when next is absent, the item list is empty, an error occurs, or the chosen page cap is reached. It constructs the next offset itself instead of blindly following a returned URL. Search pagination is not a transactionally consistent snapshot, so deduplication does not establish that every eligible offer was captured.

4. Complete API-to-CSV program

The demo uses local JSON-shaped records embedded in the script. The live branch obtains an application token and calls Browse search. Run neither branch against a filename you want to preserve.

pythonpython
import argparse, csv, ipaddress, json, os, sys
from datetime import datetime, timezone
from decimal import Decimal, InvalidOperation
from pathlib import Path
from urllib.parse import quote, urlsplit
import requests

BASE = 'https://api.ebay.com'
SEARCH = BASE + '/buy/browse/v1/item_summary/search'

class Stop(Exception):
    pass

def now():
    return datetime.now(timezone.utc).isoformat()

def amount(value):
    try:
        n = Decimal(str(value))
        return n if n.is_finite() and n >= 0 else None
    except (InvalidOperation, ValueError):
        return None

def normalize(item, context):
    price = item.get('price') or item.get('currentBidPrice') or {}
    options = item.get('shippingOptions') or []
    shipping = options[0].get('shippingCost', {}) if len(options) == 1 else {}
    pv, sv = amount(price.get('value')), amount(shipping.get('value'))
    currency = price.get('currency', '')
    modes = item.get('buyingOptions') or []
    mode = 'auction' if 'AUCTION' in modes else 'fixed_price' if 'FIXED_PRICE' in modes else 'unknown'
    ident = str(item.get('itemId', ''))
    parts = ident.split('|')
    variation = parts[2] if len(parts) == 3 and parts[2] != '0' else ''
    reasons = []
    for name, good in [('item_id', ident), ('title', item.get('title')),
            ('condition', item.get('conditionId')), ('price', pv is not None),
            ('shipping', sv is not None), ('fixed_price', mode == 'fixed_price'),
            ('currency', currency == context['expected_currency']),
            ('shipping_currency', bool(currency) and shipping.get('currency') == currency)]:
        if not good:
            reasons.append(name)
    subtotal = pv + sv if not reasons else None
    return dict(item_id=ident, variation_id=variation, title=item.get('title', ''),
        price_raw=str(price.get('value', '')), price_amount=str(pv) if pv is not None else '',
        currency=currency, shipping_raw=str(shipping.get('value', '')),
        shipping_amount=str(sv) if sv is not None else '',
        condition=item.get('condition', ''), condition_id=item.get('conditionId', ''),
        buying_format=mode, seller=(item.get('seller') or {}).get('username', ''),
        item_url=item.get('itemWebUrl', ''), price_plus_shipping=str(subtotal) if subtotal is not None else '',
        numeric_ready=not reasons, review_reason=';'.join(reasons), **context)

def sample_pages():
    def item(number, value, currency='USD', auction=False, ship=True):
        row = dict(itemId=f'v1|11000000000{number}|0', title=f'Fixture camera offer {number}',
            price=dict(value=value, currency=currency), condition='Used', conditionId='3000',
            buyingOptions=['AUCTION'] if auction else ['FIXED_PRICE'],
            seller=dict(username='fixture-seller'), itemWebUrl=f'https://www.ebay.com/itm/11000000000{number}')
        if ship:
            row['shippingOptions'] = [dict(shippingCost=dict(value='8.00', currency='USD'))]
        return row
    first = item(1, '120.00')
    return [dict(itemSummaries=[first, item(2, '75.00', auction=True, ship=False)], next='fixture-page-2'),
            dict(itemSummaries=[dict(first, price=dict(value='121.00', currency='USD')), item(3, '90.00', currency='EUR')])]

def response_json(response):
    if response.status_code != 200:
        raise Stop(f'HTTP_{response.status_code}: stop; inspect authorization, access or quota')
    if 'json' not in response.headers.get('Content-Type', '').lower():
        raise Stop('MIME: expected JSON')
    try:
        data = response.json()
    except ValueError:
        raise Stop('JSON: response cannot be decoded as JSON') from None
    if not isinstance(data, dict):
        raise Stop('SCHEMA: expected a JSON object')
    if data.get('errors'):
        raise Stop('API_ERRORS: review the approved request and API diagnostics')
    return data

def session():
    s = requests.Session()
    s.trust_env = False
    return s

def token():
    key, secret = os.environ.get('EBAY_CLIENT_ID'), os.environ.get('EBAY_CLIENT_SECRET')
    if not key or not secret:
        raise Stop('CREDENTIALS: set the approved production application credentials locally')
    with session() as auth:
        with auth.post(BASE + '/identity/v1/oauth2/token', auth=(key, secret),
                data=dict(grant_type='client_credentials', scope='https://api.ebay.com/oauth/api_scope'),
                timeout=(10, 30), allow_redirects=False) as response:
            value = response_json(response).get('access_token')
    if not value:
        raise Stop('TOKEN: response has no access_token')
    return value

def route_check(client, expected):
    check_token = os.environ.get('IPINFO_TOKEN')
    if not check_token:
        raise Stop('IP_CHECK: set an IPinfo Lite token for the requested route check')
    with client.get('https://api.ipinfo.io/lite/me',
            headers={'Authorization': 'Bearer ' + check_token},
            timeout=(10, 30), allow_redirects=False) as response:
        data = response_json(response)
    country = data.get('country_code', '')
    ip = ipaddress.ip_address(data.get('ip', ''))
    masked = str(ipaddress.ip_network(f'{ip}/' + ('24' if ip.version == 4 else '48'), strict=False))
    if expected and country != expected:
        raise Stop('EXIT_COUNTRY: result does not match the requested exit country')
    return dict(observed_utc=now(), ip_network_redacted=masked, country=country,
        asn=data.get('asn', ''), as_name=data.get('as_name', ''), source='IPinfo Lite')

def collect(args):
    fixture = sample_pages() if args.demo else None
    proxy = os.environ.get('EBAY_PROXY_URL')
    if args.demo and proxy:
        raise Stop('LOCAL_INPUT: clear EBAY_PROXY_URL before an offline run')
    records, seen, duplicates, pages = [], set(), 0, 0
    routes = []
    with session() as client:
        access = ''
        if args.live:
            if proxy:
                parsed = urlsplit(proxy)
                if parsed.scheme not in {'socks5h', 'http', 'https'} or not parsed.hostname or not parsed.port:
                    raise Stop('PROXY_CONFIG: use a supported generated endpoint')
                client.proxies.update({'http': proxy, 'https': proxy})
                routes.append(dict(route='configured_proxy', **route_check(client, args.expected_exit)))
                if args.direct_baseline:
                    with session() as direct:
                        routes.append(dict(route='direct_baseline', **route_check(direct, '')))
            elif args.direct_baseline:
                raise Stop('BASELINE: a proxy route is required for the comparison')
            access = token()
        for page in range(1, args.pages + 1):
            offset = (page - 1) * args.limit
            params = dict(q=args.query, limit=args.limit, offset=offset)
            source_url = requests.Request('GET', SEARCH, params=params).prepare().url
            if fixture is not None:
                if page > len(fixture):
                    break
                payload = fixture[page - 1]
                source_url = f'fixture://browse/page/{page}'
            else:
                headers = {'Authorization': 'Bearer ' + access,
                    'X-EBAY-C-MARKETPLACE-ID': args.marketplace,
                    'X-EBAY-C-ENDUSERCTX': 'contextualLocation=' + quote(f'country={args.country},zip={args.postal}', safe='')}
                with client.get(SEARCH, params=params, headers=headers,
                        timeout=(10, 30), allow_redirects=False) as response:
                    payload = response_json(response)
                    source_url = response.url
            observed = now()  # after this page's response/fixture has been obtained
            if not isinstance(payload.get('itemSummaries', []), list):
                raise Stop('SCHEMA: itemSummaries must be a list')
            items = payload.get('itemSummaries', [])
            pages += 1
            if not items:
                break
            context = dict(observed_utc=observed, page=page, offset=offset,
                source_url=source_url, query=args.query, marketplace=args.marketplace,
                delivery_country=args.country, delivery_postal=args.postal,
                expected_currency=args.currency, tax_basis='excluded_from_subtotal',
                sample_type='local_api_fixture' if args.demo else 'browse_api_response')
            for item in items:
                if not isinstance(item, dict) or not item.get('itemId'):
                    raise Stop('SCHEMA: every record needs itemId')
                row = normalize(item, context)
                key = (row['item_id'], row['variation_id'])
                if key in seen:
                    duplicates += 1
                    continue  # first successful observation wins; its timestamp stays unchanged
                seen.add(key)
                records.append(row)
            if not payload.get('next'):
                break
    if not records:
        raise Stop('EMPTY: no listing records; no CSV written')
    return records, dict(pages=pages, duplicates_skipped=duplicates, routes=routes)

def self_test():
    from types import SimpleNamespace
    from unittest.mock import patch
    c = dict(expected_currency='USD')
    a, b = sample_pages()[0]['itemSummaries']
    checks = [normalize(a, c)['price_plus_shipping'] == '128.00',
        not normalize(b, c)['numeric_ready'], amount('NaN') is None,
        amount('-1') is None, amount('120 to 130') is None,
        normalize(sample_pages()[1]['itemSummaries'][1], c)['numeric_ready'] is False,
        normalize(dict(a, conditionId=''), c)['numeric_ready'] is False,
        normalize(dict(a, itemId='v1|110000000001|7'), c)['variation_id'] == '7']
    args = SimpleNamespace(demo=True, live=False, pages=2, limit=2, query='camera body',
        marketplace='EBAY_US', country='US', postal='10001', currency='USD')
    with patch(__name__ + '.now', side_effect=['2026-11-12T00:00:01+00:00', '2026-11-12T00:00:02+00:00']):
        rows, report = collect(args)
    checks.extend([len(rows) == 3, report['duplicates_skipped'] == 1, report['pages'] == 2,
        rows[0]['price_amount'] == '120.00', rows[2]['page'] == 2,
        rows[0]['source_url'] == 'fixture://browse/page/1',
        len({r['observed_utc'] for r in rows}) == 2,
        all(r['tax_basis'] == 'excluded_from_subtotal' for r in rows)])
    if not all(checks):
        raise Stop('SELF_TEST: a normalization check failed')
    print(f'Local normalization and pagination checks: {len(checks)} passed')

def main():
    p = argparse.ArgumentParser()
    modes = p.add_mutually_exclusive_group(required=True)
    modes.add_argument('--demo', action='store_true')
    modes.add_argument('--live', action='store_true')
    modes.add_argument('--self-test', action='store_true')
    p.add_argument('--query', default='camera body')
    p.add_argument('--pages', type=int, choices=range(1, 4), default=2)
    p.add_argument('--limit', type=int, choices=range(1, 201), default=2)
    p.add_argument('--marketplace', default='EBAY_US')
    p.add_argument('--country', default='US')
    p.add_argument('--postal', default='10001')
    p.add_argument('--currency', default='USD')
    p.add_argument('--expected-exit', default='US')
    p.add_argument('--direct-baseline', action='store_true')
    p.add_argument('--out', type=Path, default=Path('ebay-api.csv'))
    args = p.parse_args()
    if args.self_test:
        self_test()
        return
    run_file = args.out.with_suffix('.run.json')
    if args.out.exists() or run_file.exists():
        raise Stop('OUTPUT_EXISTS: choose a new output name')
    rows, report = collect(args)
    def safe(v):
        s = str(v)
        return "'" + s if s.lstrip().startswith(('=', '+', '-', '@')) else s
    with args.out.open('x', encoding='utf-8-sig', newline='') as f:
        writer = csv.DictWriter(f, fieldnames=list(rows[0]))
        writer.writeheader()
        writer.writerows({k: safe(v) for k, v in row.items()} for row in rows)
    report.update(run_utc=now(), rows=len(rows), numeric_ready=sum(r['numeric_ready'] for r in rows),
        sample_type=rows[0]['sample_type'], dedup_policy='first_successful_observation')
    with run_file.open('x', encoding='utf-8') as f:
        json.dump(report, f, indent=2)
    print(f'Pages: {report["pages"]}; unique rows: {len(rows)}; duplicates skipped: {report["duplicates_skipped"]}')
    print(f'Numeric-ready: {report["numeric_ready"]}; review: {len(rows) - report["numeric_ready"]}')
    print(f'CSV: {args.out.name}; log: {run_file.name}')

if __name__ == '__main__':
    try:
        main()
    except Stop as e:
        print(f'Stopped: {e}', file=sys.stderr)
        sys.exit(2)
    except requests.RequestException as e:
        label = 'PROXY' if isinstance(e, requests.exceptions.ProxyError) else 'TLS' if isinstance(e, requests.exceptions.SSLError) else 'TIMEOUT' if isinstance(e, requests.exceptions.Timeout) else 'DNS_OR_CONNECTION' if isinstance(e, requests.exceptions.ConnectionError) else 'NETWORK'
        print(f'Stopped: {label}; inspect the route without sharing credential-bearing exceptions', file=sys.stderr)
        sys.exit(2)
    except (ValueError, OSError, TypeError):
        print('Stopped: INPUT_OR_RESPONSE; check configuration, paths and response schema', file=sys.stderr)
        sys.exit(2)

5. Run the program and inspect its output

First clear the tutorial’s proxy variable for an offline run:

powershellpowershell
Remove-Item Env:EBAY_PROXY_URL -ErrorAction SilentlyContinue
.\.venv\Scripts\python.exe .\ebay_browse.py --self-test
.\.venv\Scripts\python.exe .\ebay_browse.py --demo --out api-fixture.csv

On macOS/Linux, use unset EBAY_PROXY_URL and substitute .venv/bin/python for the interpreter. The program’s embedded tests check eight normalization cases and eight pagination/provenance assertions; --self-test requires no account, network request or extra test file.

Our saved run returned:

texttext
Local normalization and pagination checks: 16 passed
Pages: 2; unique rows: 3; duplicates skipped: 1
Numeric-ready: 1; review: 2
CSV: api-fixture.csv; log: api-fixture.run.json
csv-results

CSV header and two output records

Here is the complete header followed by the first two local rows. The timestamps are copied from the recorded run, not filled with a common run-start value.

csvcsv
item_id,variation_id,title,price_raw,price_amount,currency,shipping_raw,shipping_amount,condition,condition_id,buying_format,seller,item_url,price_plus_shipping,numeric_ready,review_reason,observed_utc,page,offset,source_url,query,marketplace,delivery_country,delivery_postal,expected_currency,tax_basis,sample_type
v1|110000000001|0,,Fixture camera offer 1,120.00,120.00,USD,8.00,8.00,Used,3000,fixed_price,fixture-seller,https://www.ebay.com/itm/110000000001,128.00,True,,2026-11-12T06:06:20.396005+00:00,1,0,fixture://browse/page/1,camera body,EBAY_US,US,10001,USD,excluded_from_subtotal,local_api_fixture
v1|110000000002|0,,Fixture camera offer 2,75.00,75.00,USD,,,Used,3000,auction,fixture-seller,https://www.ebay.com/itm/110000000002,,False,shipping;fixed_price;shipping_currency,2026-11-12T06:06:20.396005+00:00,1,0,fixture://browse/page/1,camera body,EBAY_US,US,10001,USD,excluded_from_subtotal,local_api_fixture

The first row has USD120.00 item price plus USD8.00 shipping: USD128.00 before tax and other charges. numeric_ready=True means this arithmetic passed the defined checks; it does not establish product equivalence with another listing or a final checkout total.

The second row is an auction with missing shipping. The third uses EUR price with USD shipping, conflicting with the expected USD context. Both remain in the output with review reasons instead of becoming a false “lowest delivered price.”

quality-results

What happened to the duplicate and timestamps?

The second fixture page repeats the first offer with a changed price. The program’s explicit policy is first successful observation wins, so it retains the first page’s price and capture time. An alternative last-observation policy would require replacing the whole observation, including timestamp and source—not merely its price.

Each successful page has its own observed_utc, source URL, query, page and offset. Very fast local pages can share a clock value on some systems; the key requirement is that capture happens per page, not that timestamps are artificially made unique.

provenance-results

Execute an approved live API request

Load EBAY_CLIENT_ID and EBAY_CLIENT_SECRET from your local secret store, then run:

powershellpowershell
.\.venv\Scripts\python.exe .\ebay_browse.py --live --query "camera body" --limit 20 --pages 2 --marketplace EBAY_US --country US --postal 10001 --currency USD --out approved-api.csv

This requests US marketplace listings with an explicit US delivery destination. The Browse shipping-context guidance explains the encoded contextualLocation header. Marketplace, item location, delivery destination and proxy exit are separate inputs; none should be inferred from another.

The .run.json file reports page count, duplicates, numeric-ready count, sample type and any performed route checks. No rows produce an explicit EMPTY stop, not a success claim. A failed later page ends the run before CSV export, avoiding an apparently complete partial dataset.

6. Parse authorized saved HTML as a second path

Use this path when your permitted source is an HTML capture rather than an API response. Preserve its original capture time and URL. parsed_utc is the time you processed the file, not when the market offer was observed.

Inspect the supplied page before relying on selectors: .s-item and .s-card below are compatibility patterns, not a permanent eBay markup contract. This program deliberately routes every HTML row to context review instead of inventing condition, currency or delivery information.

Save this separate, complete program as ebay_html.py:

pythonpython
import argparse, csv, os, re
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import urljoin, urlsplit, parse_qs, urlencode
from bs4 import BeautifulSoup

def main():
    p = argparse.ArgumentParser()
    p.add_argument('html', type=Path)
    p.add_argument('--captured-utc', required=True)
    p.add_argument('--source-url', required=True)
    p.add_argument('--out', type=Path, default=Path('ebay-html.csv'))
    args = p.parse_args()
    stamp = datetime.fromisoformat(args.captured_utc.replace('Z', '+00:00'))
    if stamp.utcoffset() is None:
        p.error('captured-utc must include a timezone')
    if os.environ.get('EBAY_PROXY_URL'):
        p.error('clear EBAY_PROXY_URL; this path parses local HTML only')
    if args.out.exists():
        p.error('choose a new output filename')
    soup = BeautifulSoup(args.html.read_text(encoding='utf-8'), 'html.parser')
    def text(card, selectors):
        n = card.select_one(selectors)
        return n.get_text(' ', strip=True) if n else ''
    rows, seen = [], set()
    for card in soup.select('.s-item, .s-card'):
        for link in card.select('a[href]'):
            u = urlsplit(urljoin('https://www.ebay.com', link['href']))
            match = re.search(r'/itm/(?:[^/?]+/)?(\d{9,15})(?:[/?#]|$)', u.path)
            if u.scheme != 'https' or u.hostname not in {'www.ebay.com', 'ebay.com'} or not match:
                continue
            item_id, variation = match[1], parse_qs(u.query).get('var', [''])[0]
            if (item_id, variation) in seen:
                break
            seen.add((item_id, variation))
            price = text(card, '.s-item__price, .s-card__price')
            shipping = text(card, '.s-item__shipping, .s-item__logisticsCost, .s-card__shipping')
            review = ['html_context_review']
            if not shipping:
                review.append('shipping_missing')
            if re.search(r'\s(?:to|[-–])\s', price):
                review.append('price_range')
            rows.append(dict(item_id=item_id, variation_id=variation,
                title=text(card, '.s-item__title, .s-card__title'), price_raw=price,
                shipping_raw=shipping, captured_utc=stamp.astimezone(timezone.utc).isoformat(),
                parsed_utc=datetime.now(timezone.utc).isoformat(), source_url=args.source_url,
                review_reason=';'.join(review)))
            break
    if not rows:
        p.error('no valid items: inspect the saved page and selectors')
    def safe(v):
        return "'" + v if v.lstrip().startswith(('=', '+', '-', '@')) else v
    with args.out.open('x', encoding='utf-8-sig', newline='') as f:
        w = csv.DictWriter(f, fieldnames=list(rows[0]))
        w.writeheader()
        w.writerows({k: safe(v) for k, v in r.items()} for r in rows)
    print(f'HTML rows: {len(rows)}; source capture time retained; all rows require context review')

if __name__ == '__main__':
    main()

For an authorized capture, replace the timestamp and source with that capture’s actual metadata:

powershellpowershell
.\.venv\Scripts\python.exe .\ebay_html.py .\authorized-search.html --captured-utc "2026-11-12T06:00:00Z" --source-url "https://www.ebay.com/sch/i.html?_nkw=camera&_pgn=1" --out saved-html.csv

Our local three-card fixture produced three rows: a fixed-price-looking card with variation 7, a card with missing shipping, and a USD90–100 price range. The range retains both endpoints as text and receives price_range; it is not reduced to a USD90 offer. The actual source URL in that local run was fixture://html/page/1.

html-results

7. Configure the optional Socks5.IO connection

Use rotating residential proxies when an authorized project requires a configured residential route. A direct approved API connection may already meet the requirement. A proxy supplies connectivity, not application approval, missing fields or an automatic collection entitlement.

  1. Open the purchased residential product and its Proxy Setup view.
  2. Select the proxy sub-account, required exit market, supported protocol and session mode. Keep the gateway/access point separate from the requested exit.
  3. Copy the generated host, port, proxy username and password. These are not your dashboard-login credentials.
  4. Choose continuity for the actual workflow. The residential session-control documentation distinguishes sticky and rotating modes; use the selected product’s current generated values, not a username suffix or timeout copied from another product.
socks-subaccount
socks-proxy-setup

Create the variable from locally stored secrets, URL-encoding credential components where needed:

powershellpowershell
$env:EBAY_PROXY_URL = 'socks5h://ENCODED_USER:ENCODED_PASSWORD@GENERATED_HOST:GENERATED_PORT'

This is a template, not a working endpoint. Requests’ SOCKS documentation distinguishes socks5h proxy-side DNS from socks5 client-side DNS. HTTP(S) can also be appropriate when supported by the chosen product and client; protocol alone is not a guarantee of website access.

How do I verify a SOCKS5 proxy’s exit location before collecting eBay data?

Check from the same Requests session used for the approved Browse request. This program uses IPinfo Lite’s documented lookup, whose country/ASN response is independent of eBay’s marketplace or delivery settings. Set its lookup token locally as IPINFO_TOKEN, then run:

powershellpowershell
.\.venv\Scripts\python.exe .\ebay_browse.py --live --expected-exit US --direct-baseline --query "camera body" --limit 20 --pages 1 --country US --postal 10001 --out routed-api.csv

With EBAY_PROXY_URL present, the program checks that configured session first and stops on a country mismatch. --direct-baseline additionally checks a separate direct session. It records UTC, country, ASN and a redacted IP network—not tokens or full proxy URLs. Both sessions disable ambient proxy inheritance. The JSON route entries distinguish the two paths; matching redacted networks alone cannot prove that the full IPs match.

For a manual RoxyBrowser check, follow the same generated protocol and fields, then validate the launched profile separately. A browser check is not proof of the Python process’s route.

roxy-fields
Failure Diagnostic order Decision
407 or proxy-authentication error Product status → generated host/port → proxy credentials → supported protocol Correct configuration before another observation
DNS or connection failure Gateway name resolution → route availability → endpoint/port → proxy-side DNS support Keep the specific failing stage; do not expose credential-bearing exceptions
TLS error Certificate trust and configured endpoint Do not disable TLS verification
Exit country differs Same-session lookup → selected market → generated username/session settings Stop before collecting location-dependent records
API 401 Token environment, expiry and scope Obtain the correct application token; no automatic credential retry loop
API 403 Application eligibility and allowed use Resolve access approval rather than rotate exits
API 429 Application quota and applicable schedule Stop; follow the service’s limits
Invalid JSON, wrong MIME or missing itemId Response format and schema Stop instead of exporting misleading records

8. Turn the export into a defensible offer comparison

For ongoing price monitoring, use an explicit comparison group: model/variant, condition, buying format, currency, delivery destination and applicable charges. numeric_ready is an arithmetic/data-presence check, not a claim that all rows with the same search query represent equivalent products.

Comparison field Required treatment
item_id + variation_id Identity and duplicate control
Condition and model/variant Compare equivalent offers; resolve ambiguous variants
Fixed price versus auction Keep current bids out of fixed-price ranking
Currency and shipping currency Match the expected currency; do not silently convert
Delivery country/postal Preserve what was requested, not inferred IP geography
Price plus shipping Compute only from supported amounts; tax is excluded from this subtotal
Tax and checkout conditions Obtain the relevant terms before calling a subtotal a final delivered price
Timestamp and source Keep each page’s observation and first-appearance provenance

One of three local rows is numerically ready; one has missing shipping and another has conflicting currencies. These are data-quality results for this controlled sample, not eBay-wide missing-field or success-rate statistics. Do not interpret a listing absent from a later search as a completed sale.

Conclusion

To scrape eBay data usefully, complete the retrieval-to-validation loop: authenticate an approved application, preserve page and delivery context, separate incomplete offers, and export a traceable result. The executable example demonstrates that pipeline with repeatable local outputs and a live API branch for an approved application.

Add Socks5.IO only when the authorized workflow needs a configured residential route, and check that route in the actual client. The useful deliverable is a comparable, explainable offer record—not a large CSV whose prices cannot be defended.

FAQs

Public visibility alone does not grant permission for automated collection. eBay’s agreement requires prior express permission for automated access, and API use has separate conditions. Applicable law, the data collected and its intended use also matter. Choose an approved route, keep within its scope and exclude private buyer or account information. For commercial research, obtain appropriate advice rather than treating a tutorial as blanket legal authorization.

Latest articles

Global proxy network illustration

Plug into the global proxy network — every connection faster and more stable

Start for Free