Skip to content
Proxy locationsSetup guidesSupport
[email protected]Try 0.5 GB for £1
ProxyLabs
Talk to our team
puppeteerweb scrapingresidential proxies

Puppeteer Proxy Setup: Headless Chrome with Rotating IPs

Configure Puppeteer with rotating residential proxies. Covers proxy authentication, page-level rotation, session persistence, and stealth plugins.

James LiuLead Engineer @ ProxyLabs
6 min read

Puppeteer sets proxies at the browser level via Chromium launch args, which means every page in that browser instance shares the same proxy. If you need different IPs per page (or per request), you have to either launch separate browser instances or use the page.authenticate() workaround with a gateway proxy that supports username-based rotation.

This guide covers both approaches and the trade-offs between them.

Basic Proxy Configuration

Puppeteer accepts proxy settings through Chromium's --proxy-server flag. The catch: this flag doesn't support authentication. You need page.authenticate() separately.

javascript
const puppeteer = require('puppeteer');

async function basicProxy() {
  const browser = await puppeteer.launch({
    headless: 'new',
    args: [
      '--proxy-server=http://gate.proxylabs.app:8080',
      '--disable-blink-features=AutomationControlled',
    ],
  });

  const page = await browser.newPage();

  // Auth must be set before any navigation
  await page.authenticate({
    username: 'your-username',
    password: 'your-password',
  });

  await page.goto('https://httpbin.org/ip');
  const body = await page.$eval('pre', el => el.textContent);
  console.log('IP:', JSON.parse(body).origin);

  await browser.close();
}

basicProxy();

Per-Page IP Rotation

With a gateway proxy like ProxyLabs, each new connection gets a different IP by default. But since Puppeteer reuses connections within a browser instance, you need to force new connections for rotation.

Method 1: Username-Based Rotation (Recommended)

Add a unique session ID to the username for each page. The proxy gateway treats each unique username as a separate session with a different IP.

javascript
const puppeteer = require('puppeteer');
const crypto = require('crypto');

async function rotatingPages() {
  const browser = await puppeteer.launch({
    headless: 'new',
    args: ['--proxy-server=http://gate.proxylabs.app:8080'],
  });

  const urls = [
    'https://example.com/page/1',
    'https://example.com/page/2',
    'https://example.com/page/3',
  ];

  for (const url of urls) {
    const page = await browser.newPage();
    const sessionId = crypto.randomBytes(6).toString('hex');

    await page.authenticate({
      username: `your-username-session-${sessionId}`,
      password: 'your-password',
    });

    await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 });
    const content = await page.content();
    console.log(`${url}: ${content.length} bytes`);
    await page.close();
  }

  await browser.close();
}

rotatingPages();

Method 2: Separate Browser Instances

For complete isolation (separate cookies, storage, and proxy connections):

javascript
const puppeteer = require('puppeteer');

async function scrapeWithIsolation(url, country) {
  const username = country
    ? `your-username-country-${country}`
    : 'your-username';

  const browser = await puppeteer.launch({
    headless: 'new',
    args: [
      '--proxy-server=http://gate.proxylabs.app:8080',
      '--disable-blink-features=AutomationControlled',
      '--no-sandbox',
    ],
  });

  const page = await browser.newPage();
  await page.authenticate({ username, password: 'your-password' });

  try {
    await page.goto(url, { waitUntil: 'networkidle2', timeout: 30000 });
    return await page.content();
  } finally {
    await browser.close();
  }
}

// Run 5 in parallel, each with a different US IP
async function main() {
  const urls = Array.from({ length: 5 }, (_, i) => `https://example.com/product/${i + 1}`);
  const results = await Promise.all(urls.map(url => scrapeWithIsolation(url, 'US')));
  console.log(`Scraped ${results.filter(Boolean).length} pages`);
}

main();

Each instance uses ~80-150MB of RAM. At 10 concurrent instances, that's already 1-1.5GB. Plan your concurrency accordingly.

Geo-Targeting

Append country and city modifiers to the username:

javascript
// Country-level targeting
await page.authenticate({
  username: 'your-username-country-US',
  password: 'your-password',
});

// City-level targeting
await page.authenticate({
  username: 'your-username-country-US-city-LosAngeles',
  password: 'your-password',
});

// Country + sticky session (same IP for up to 30 min)
await page.authenticate({
  username: 'your-username-country-GB-session-checkout001',
  password: 'your-password',
});

Sticky Sessions for Multi-Step Flows

Login → navigate → add to cart → checkout requires the same IP throughout. Use a fixed session ID:

javascript
const puppeteer = require('puppeteer');

async function multiStepFlow() {
  const browser = await puppeteer.launch({
    headless: 'new',
    args: ['--proxy-server=http://gate.proxylabs.app:8080'],
  });

  const page = await browser.newPage();

  // Same session ID = same IP for up to 30 minutes
  await page.authenticate({
    username: 'your-username-session-checkout42-country-US',
    password: 'your-password',
  });

  // Step 1: Login
  await page.goto('https://example.com/login');
  await page.type('#email', '[email protected]');
  await page.type('#password', 'password123');
  await page.click('#login-btn');
  await page.waitForNavigation();

  // Step 2: Navigate to product
  await page.goto('https://example.com/product/123');

  // Step 3: Add to cart
  await page.click('#add-to-cart');
  await page.waitForSelector('.cart-updated');

  // Step 4: Checkout — still the same IP
  await page.goto('https://example.com/checkout');
  const total = await page.$eval('.total', el => el.textContent);
  console.log('Cart total:', total);

  await browser.close();
}

multiStepFlow();

For a deeper explanation of when sticky vs rotating sessions matter, see rotating vs sticky proxies.

Stealth Configuration

Puppeteer is detectable by default. The puppeteer-extra-plugin-stealth package patches most automation markers:

javascript
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');

puppeteer.use(StealthPlugin());

async function stealthScrape(url) {
  const browser = await puppeteer.launch({
    headless: 'new',
    args: [
      '--proxy-server=http://gate.proxylabs.app:8080',
      '--disable-blink-features=AutomationControlled',
      '--window-size=1920,1080',
    ],
  });

  const page = await browser.newPage();

  await page.authenticate({
    username: 'your-username-country-US',
    password: 'your-password',
  });

  // Set a realistic viewport
  await page.setViewport({ width: 1920, height: 1080 });

  // Set a real user agent
  await page.setUserAgent(
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36'
  );

  await page.goto(url, { waitUntil: 'networkidle2', timeout: 30000 });
  return await page.content();
}

The stealth plugin handles navigator.webdriver, window.chrome, navigator.plugins, and other markers. For the full list of 6 automation markers and how they're detected, see our Playwright proxy setup tutorial — the markers are identical since both use Chromium.

Handling Proxy Errors

Residential proxies occasionally fail — an IP might go offline mid-request. Build retry logic:

javascript
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
const crypto = require('crypto');

puppeteer.use(StealthPlugin());

async function scrapeWithRetry(url, maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    const sessionId = crypto.randomBytes(6).toString('hex');
    const browser = await puppeteer.launch({
      headless: 'new',
      args: ['--proxy-server=http://gate.proxylabs.app:8080'],
    });

    try {
      const page = await browser.newPage();
      await page.authenticate({
        username: `your-username-session-${sessionId}`,
        password: 'your-password',
      });

      const response = await page.goto(url, {
        waitUntil: 'domcontentloaded',
        timeout: 30000,
      });

      if (response.status() === 403 || response.status() === 429) {
        console.log(`Attempt ${attempt + 1}: Got ${response.status()}, retrying...`);
        continue;
      }

      // Check for soft blocks (CAPTCHA pages)
      const content = await page.content();
      if (content.includes('captcha') || content.includes('blocked')) {
        console.log(`Attempt ${attempt + 1}: Soft block detected, retrying...`);
        continue;
      }

      return content;
    } catch (err) {
      console.log(`Attempt ${attempt + 1}: ${err.message}`);
    } finally {
      await browser.close();
    }
  }
  throw new Error(`Failed after ${maxRetries} attempts: ${url}`);
}

Concurrent Scraping with p-limit

Control concurrency to avoid overwhelming your machine:

javascript
const pLimit = require('p-limit');

const limit = pLimit(5); // Max 5 browsers at once

async function scrapeAll(urls) {
  const results = await Promise.all(
    urls.map(url => limit(() => scrapeWithRetry(url)))
  );
  return results;
}

const urls = Array.from({ length: 50 }, (_, i) => `https://example.com/item/${i}`);
scrapeAll(urls).then(results => {
  console.log(`Successfully scraped ${results.filter(Boolean).length} / ${urls.length}`);
});

Verifying the Setup

Quick verification script to confirm your proxy is working correctly:

javascript
const puppeteer = require('puppeteer');

async function verify() {
  const browser = await puppeteer.launch({
    headless: 'new',
    args: ['--proxy-server=http://gate.proxylabs.app:8080'],
  });

  const page = await browser.newPage();
  await page.authenticate({
    username: 'your-username-country-JP',
    password: 'your-password',
  });

  // Check IP
  await page.goto('https://httpbin.org/ip');
  const ip = await page.$eval('pre', el => JSON.parse(el.textContent).origin);
  console.log(`Proxy IP: ${ip}`);

  // Check headers
  await page.goto('https://httpbin.org/headers');
  const headers = await page.$eval('pre', el => JSON.parse(el.textContent));
  console.log('User-Agent:', headers.headers['User-Agent']);

  await browser.close();
}

verify();

Confirm the IP location with ProxyLabs' IP Lookup tool or the Proxy Tester to verify it's residential and in the right geo.

For scraping-specific strategies beyond proxy setup, see our guides on scraping without getting blocked and avoiding IP bans.

Useful guide? Share it with your team.

Share
← Back to all articlesBack to top ↑
All articles ↗
playwright

Playwright Proxy Setup: The 6 Automation Markers You're Missing

Most stealth guides fix navigator.webdriver and stop. Playwright exposes 5 other automation markers that DataDome and Akamai check. Here's the complete init script with all 6 addressed.

5 min readselenium

Selenium Proxy Setup: Complete Guide for Python & Java

Configure rotating residential proxies in Selenium for Python and Java. Covers authentication, geo-targeting, session persistence, and anti-detection patterns.

6 min readaxios

Axios Proxy Setup in Node.js: Rotating IPs for API Requests

How to configure proxies in Axios with authentication, rotation, session management, and the gotchas that break HTTPS tunneling. Includes working patterns for scraping and API monitoring.

8 min read