Browser automation: Playwright and browser extensions

How a program drives a real browser through WebDriver and the DevTools Protocol, how Playwright is built and why it waits for you, how to find elements by role and accessible name, how to stop flaky tests, how a Manifest V3 extension records clicks, and how to keep a test browser away from private networks.

In short

Browser automation means a program, not a person, drives a real browser: it opens pages, clicks, types and checks what appears. Tools talk to the browser through a protocol - WebDriver or the Chrome DevTools Protocol - and Playwright wraps that protocol in an API that waits for the page by itself. A browser extension works from the other side, inside the person's own browser, which is how a recorder can watch real clicks. In {{brand}}, UI Tester records steps through an extension and replays them on the server with Playwright, in a browser that can only navigate to the site under test and can never reach private networks.

After this lesson you can Explain how WebDriver and the DevTools Protocol move a click into a browser and choose between Selenium, Cypress, Puppeteer and Playwright; write selectors and tests that survive a redesign and do not flake; explain how a Manifest V3 extension records events; and keep a server-side test browser away from private networks.

Words you need

• WebDriver The W3C standard for remote control of a browser: each command is an HTTP request to a small driver program, such as chromedriver, that controls the browser.
• DevTools Protocol (CDP) Chromium's own protocol: JSON messages over one WebSocket, commands one way and events the other. Chrome's DevTools use it too.
• Request routing Code that sees every request a page makes and can continue, change or abort it. Playwright calls it route; other tools call it a request hook or interceptor.
• Browser context A fresh, isolated profile inside one running browser - its own cookies, storage and cache. Creating one takes milliseconds.
• Locator Playwright's description of how to find an element, such as a button named Save. It is looked up again every time it is used.
• Auto-waiting Before acting, the tool checks the element is visible, stable, enabled and not covered, and retries until a timeout.
• Accessibility tree A tree the browser builds beside the DOM: every element's role, name and state. Screen readers read it.
• Accessible name The name the browser computes for an element: a button's text, a field's label, an image's alt text.
• Flaky test A test that passes and fails on the same code. Usually a timing, data or selector problem in the test.
• Manifest V3 The current format of Chrome and Edge extensions: a service worker instead of a background page, stricter permissions, no remote code.
• Content script Extension code that runs inside a web page, in an isolated world: it shares the page's DOM but not its JavaScript.
• SSRF Server-side request forgery: tricking a server into requesting an address on its own private network.

How a program drives a browser

Every automation tool sits in one of two places. Most sit outside the browser and send commands through a protocol: Selenium, Puppeteer and Playwright. Cypress sits inside the page, running the test code next to the application. The outside tools differ mainly in which protocol they speak, and that choice decides speed and how well they know what the page is doing.

• 1. The test asks The test calls something like element.click() in its own language: Java, Python, C# or JavaScript.
• 2. One HTTP request The client library sends POST /session/{id}/element/{id}/click to the driver program, a separate process for each browser.
• 3. The driver acts The driver turns the command into the browser's internal calls and the browser performs the click.
• 4. One HTTP answer The driver answers when the command is done. Nothing is pushed to the test afterwards.
• 5. The test polls To learn whether a page loaded or text appeared, the test must ask again and again. This is where classic WebDriver tests get slow and flaky.

• 1. One connection The library opens a single WebSocket (or a pipe) straight into the browser and keeps it open.
• 2. Commands with ids It sends JSON such as Input.dispatchMouseEvent with an id; the browser answers with the same id.
• 3. Events arrive by themselves The browser pushes events - a request started, a page loaded, a console message - without being asked.
• 4. The library knows the page With those events the library knows when the page is busy, so it can wait for the right moment instead of guessing.

Cypress is the odd one out Cypress runs your test code inside the browser, in the same loop as the application, with a proxy in front of the page. That gives excellent debugging and automatic retries, but the test lives in one tab, crossing origins needs a special command with limits, and it must be written in JavaScript or TypeScript.

The options in the market

• Selenium The WebDriver pioneer, since 2004: every language, every browser.; Every major browser, Safari and real-device clouds included; Every popular language; Selenium Grid spreads tests over many machines; Huge community and knowledge base; Waiting is your job: missing waits are the classic source of flakes; One HTTP round trip per command, so slower; Driver versions must match the browser, though Selenium Manager automates most of it
• Cypress Runs inside the page, next to the app; JavaScript only.; Time-travel debugging in its own runner; Automatic waiting and retried assertions; Very pleasant for front-end developers; One tab at a time; crossing origins needs cy.origin and has limits; JavaScript and TypeScript only; Not built for a server running many unrelated sites
• Puppeteer Google's Node.js library for Chrome over the DevTools Protocol; Firefox through BiDi.; Thin, fast and close to Chrome; Great for scraping, PDFs and screenshots; Node.js only; A library, not a test tool: no test runner, traces or retrying assertions of its own
• Playwright Microsoft, since 2020: Chromium, Firefox and WebKit from .NET, Java, Python or Node.js.; Locators that wait by themselves and assertions that retry; A new isolated context in milliseconds; Screenshots, video and traces built in; Finds elements by role, label and text; Can inspect, change or refuse every request; Downloads its own browser builds, which need system libraries on the server; A Node.js driver process runs underneath every language; A real browser per run is heavy compared with an HTTP call
• Hosted record-and-replay services A cloud recorder and runner you pay for.; No code at all; Nothing to install or host; Tests live in someone else's format; Your site's sessions run on their machines; Hard to fit into your own workflow and data

What {{brand}} chose, and why

{{brand}} replays recorded steps on its own servers, unattended, for many workspaces at once. Three things mattered most: a clean, isolated session for every run, waiting that is built in rather than written into every step, and finding elements the way a person does. Playwright gives all three, and it has a first-class .NET library, which fits a .NET backend.

• One context per run: a new browser context takes milliseconds and shares nothing, so cookies from one customer's run can never reach another's.
• Locators like a person's: elements are found by role and name, label, placeholder or text, which is how recorded steps describe them, with CSS and XPath as fallbacks.
• Evidence built in: a screenshot after each step, a video of the whole run, and on a failure the page address and an excerpt of the page.
• Request routing: every request the page makes can be inspected and refused, which the network sandbox needs.
• Replaceable engine: the product talks to a small browser interface, so the engine can change and tests can use a fake browser.

The trade-off accepted A real browser per run costs far more CPU and memory than an HTTP request. {{brand}} limits concurrent runs and meters browser time so one busy workspace cannot starve the others.

Another tool is the right one in other situations: Selenium when you need Safari on real devices or already run a Selenium Grid; Cypress for a front-end team testing its own app in JavaScript; Puppeteer for a small Chrome-only script, such as turning pages into PDFs. {{brand}}'s own team also uses Playwright outside the product, to walk the app and record Help pictures and clips - the clip at the top of this lesson was recorded with Playwright's video feature.

Inside Playwright: browser, context, page, locator

• Your code A test or a program in C#, TypeScript, Python or Java calls the Playwright library.
• The Playwright driver From C#, Python or Java the library starts a small Node.js driver process and talks to it over a pipe; in Node.js the same code runs in-process. Every language shares this one engine, so they behave the same.
• The browser The driver speaks the DevTools Protocol to Chromium, and its own protocol to patched builds of Firefox and WebKit that Playwright downloads.
• Events back Network, page and console events flow back the same way, which is how Playwright knows a page is still loading.

Launch once, create contexts often Starting a browser is the expensive part. Keep one browser per worker process and give every test or run its own context: you get full isolation for the price of a few milliseconds.

Auto-waiting: why Playwright tests flake less

The most common bug in browser tests is acting too early: clicking a button that is not there yet, or reading text before it changed. A fixed sleep is either too short, so the test flakes, or too long, so the suite crawls. Playwright replaces guesses with checks: before every action it waits until the element is ready, and assertions retry until they pass or time out.

• Resolve Find the elements the locator describes. Strict: if it matches more than one, the action fails rather than guess.
• Visible The element has a size on screen and is not hidden.
• Stable It kept the same position over two animation frames, so it is not sliding in.
• Receives events It is the element actually hit at that point, not covered by a spinner, toast or dialog.
• Enabled It is not disabled; for typing it must also be editable.
• Act Scroll it into view and click. If any check fails, try again until the timeout.

Web-first assertions An assertion on a locator, such as toHaveText, re-checks the page until it passes or times out. An assertion on a plain value checks once, at whatever moment the line runs. {{brand}}'s check steps work the first way: they re-check many times a second until the step's time runs out.

Seeing what happened: screenshots, video, traces

{{brand}} keeps a screenshot after every step, a video of the run, and on a failure the page address and an excerpt of the page around the element, so a person can see why a step failed without running it again. A common pattern for sites that need a sign-in is to sign in once in a separate context and reuse the saved storage state for the tests.

Robust selectors and the accessibility tree

Next to the DOM, every browser builds an accessibility tree for screen readers. Each node has a role (button, link, textbox, heading, row), an accessible name and a state such as checked or disabled. A selector built from role and name describes an element the way a person sees it, so it survives restyling, new class names and moved containers.

• aria-labelledby If the element points at other elements by id, their text is the name.
• aria-label An explicit aria-label wins next.
• Native label A field's label for=..., an image's alt, an input button's value.
• Content For buttons, links, headings, cells and similar roles, the text inside, including the alt text of images inside.
• Last resorts The title attribute, then for fields the placeholder - weak names, used only when nothing better exists.

• One CSS or XPath path div:nth-child(3) > button.css-1x2y3z; Works on any page; Exact; Breaks when layout or generated class names change; Means nothing to a person reading the test
• Visible text only the element that says Save; Readable; Breaks on a copy change or a translation; Ambiguous when the text repeats
• Test id only data-testid=save-card; The most stable: a contract with the developers; Most sites do not have them; Useless on a page you do not control
• Role and accessible name only button named Save; What a person and a screen reader see; Survives restyling; Breaks when the wording changes; Needs reasonably correct markup
• A ranked list, each checked to be unique Several of the above, best first; Survives most redesigns; A later candidate can still find the element; Shows when the page drifted; More to store; Needs a rule for when a fallback may be trusted

A sensible ranking for a recorder, best first. Another tool may order it a little differently, but the idea is the same: contracts and meaning first, structure last.

When a locator matches several elements - three rows, each with a Delete button - a good recorder adds an anchor: a nearby element that is unique, used to scope the locator.

• A locator matches three The button named Delete matches three elements, so on its own it is ambiguous.
• Walk up from the wanted button Starting at the wanted button, go up through its ancestors, one container at a time.
• Stop at something unique Stop at the first container with something unique, such as a row title that appears once on the page. Skip generated-looking names such as hashes.
• Write the scoped locator Describe the target as: the Delete button within the row named Invoice 1042.
• Check it Run the new locator on the live page and confirm it matches exactly one element.

Self-healing selectors When the first locator fails but a later one matches, a good tool reports the fallback as a healed result and lets a person decide whether to adopt it. A tool that silently rewrites tests can accept a real regression without anyone noticing.

Flaky tests: why they happen and how to stop them

A flaky test passes and fails on the same code. Each flake teaches the team to ignore red, and soon a real failure is ignored too. Almost every flake has one of a handful of causes, and each cause has a known cure.

• Collect results Take a test's most recent results, newest first - for example the last 20 passes and failures.
• Count flips A flip is a pair of neighbouring results that differ: pass then fail, or fail then pass.
• Compute the rate Flip rate = flips / pairs. A test that always fails flips 0 times: it is broken, not flaky.
• Decide with enough data Call a test flaky only with enough results (for example at least 6) and a rate of 30% or more - pick numbers that suit your suite.
• Quarantine with an owner Keep running the test but stop counting it as evidence until its owner fixes it. A person decides; the rate only suggests.

{{brand}} applies one flakiness rule across its test tools, so flaky means the same everywhere.

Retries hide flakes An automatic retry turns red into green but keeps the cause. Use retries to collect evidence - a trace of the second attempt - and track how often a retry was needed. A test that needs retries is a flaky test.

Browser extensions with Manifest V3

A browser extension is a package of HTML, JavaScript and a manifest.json that the browser loads with extra powers: it can see tabs, run code in pages and capture what is on screen. Manifest V3 is the current format in Chrome and Edge, and Firefox and Safari support it too; Chrome has switched the older Manifest V2 off.

• Web app page The app posts a message to its own window with window.postMessage, naming itself as the source.
• Content script Accepts only messages from the same window and origin, stamps the sender's origin on them and passes them on.
• Service worker Checks the sender's tab and origin against an allowlist, then acts.
• Server The worker calls the server with the credentials it holds.
• Reply The answer travels back the same way, matched by a request id.

The service worker dies Chrome stops an idle extension service worker after about 30 seconds. A setTimeout may never fire and a queue kept only in memory is lost. Write state to chrome.storage.session (in memory, cleared when the browser closes) or chrome.storage.local, and schedule later work with chrome.alarms, which wake the worker again.

How a recorder captures events

A recorder is a content script that watches a person and writes down their intent, not every raw event. Typing a name produces dozens of input events; the test needs one step: fill the Name field with Ada. The hard parts are deciding what counts as one action, describing the element so it can be found again, and never writing down a password.

• Inject on demand While recording, inject the recorder only into tabs of the approved sites, and again after every page load, which the browser's navigation events announce.
• Listen early and quietly Listen on the document in the capture phase, before the page's own handlers can stop the event, and passively, so the page is never slowed or changed.
• Keep only real events event.isTrusted is false for events a script dispatched, so the page cannot fake a step.
• Find the element meant From the exact target - composedPath()[0] reaches into shadow DOM - walk up to the nearest button, link, field or element with a role.
• Turn events into intent Many input events become one Fill when focus leaves. A click waits a moment, so a double click is not recorded as two clicks. A form submit right after the click that caused it is not recorded twice.
• Describe the element Build the ranked candidates and check each one matches exactly this element on the live page.
• Mask secrets at the source Fields that look like passwords, card numbers or one-time codes store a placeholder, never the typed text.
• Screenshot and upload Capture the visible tab, then send steps in small batches with retry. The server checks every step again.

• Navigations come from the browser's navigation events, not from the page; a load caused by a recorded click, a redirect or the start page itself is not a separate step.
• Frames are hard: an extension can see them, but a step inside a cross-origin iframe needs its own frame address on replay. Many recorders therefore record the top page only and need extra work for frames.
• Checks are not recorded: a recorder sees actions, not intentions about outcomes. Assertions - this text is visible, the address contains /orders - must be added by a person, and a test without one proves only that nothing crashed.

Why an extension and not an iframe A web page cannot look inside another site's frames, and most sites refuse to be framed at all. An extension can follow the person across pages and origins - the reason Selenium IDE and many commercial recorders are extensions, while Chrome's DevTools Recorder gets the same reach by being built into the browser.

Screenshots can still show secrets Masking protects what was typed. A screenshot shows whatever was on screen - an email address, an account number - so review a recorded test before sharing it.

Isolating test browsers on the network

A browser on your server is a powerful client inside your network. If a page under test redirects it to http://169.254.169.254/ - the cloud metadata address on most clouds - or to http://localhost:6379, it will happily go. That is SSRF, and a browser makes it easier than plain HTTP code because pages load images, frames, scripts and sockets on their own.

Below are the typical ways a test browser escapes; a real setup covers each one. A request hook (Playwright calls it routing) sees each request the page makes and can allow or abort it. An egress proxy is a program that all outgoing connections of the browser must pass through, so it can check every one of them.

• A name with a short lifetime The attacker's name answers with a public address and a very short lifetime.
• The check passes The check resolves the name, sees a public address and allows it.
• The answer changes A moment later the browser connects and the name now answers 127.0.0.1.
• The request reaches the inside The request lands on an internal service. The fix is to resolve once and connect to the very address you checked.

• At configuration time Resolve its host and refuse loopback, private and link-local addresses and cloud metadata endpoints.
• When a run starts Check again: a name can start pointing somewhere else after it was approved.
• On every request A request hook refuses navigation to other origins and refuses hosts that resolve to forbidden addresses.
• On every connection An egress proxy resolves each host itself, refuses forbidden addresses and connects to the address it checked - redirect hops and late DNS answers included.
• Below it all Add a network firewall that limits what the server may reach as the last layer, in case everything above has a bug.

A request hook is not enough on its own Routing sees the request the page made, but the browser follows redirects and resolves names by itself. Only a check at the level of each connection - a proxy every connection of the run must pass - sees every hop. {{brand}} applies several layers of this kind to every run, so no single check is trusted on its own.

Try it yourself: record, replay and harden one test

• Install In an empty folder run npm init playwright@latest and accept the defaults.
• Record Run npx playwright codegen https://demo.playwright.dev/todomvc, add two todos and tick one. Watch which locators codegen picks: roles, placeholders and text first.
• Clean up Paste the code into a test. Replace any CSS path with getByRole, getByLabel or getByPlaceholder, and add at least one assertion.
• Hunt for flakes Delay the app on purpose with the route snippet below, then compare a test that clicks and reads the text once (flaky) with one that uses a retrying assertion (passes). Run both with npx playwright test --repeat-each=20.
• Read a trace Break the expected text, run with --trace on, then open the report with npx playwright show-report and step through the trace.
• Look at the tree Open Chrome DevTools, Elements, then the Accessibility pane, and inspect the button you clicked: its role and computed name are what getByRole matched.
• Fence the browser Start python -m http.server 8000, apply the origin allowlist from this lesson with https://demo.playwright.dev as the only allowed origin, and try to open http://localhost:8000. The navigation is aborted. Then note which paths a request hook alone would miss: a redirect hop or an image the page loads by itself.
• Bonus: a tiny extension Write a manifest and a content script that logs every click with its isTrusted value; load it unpacked at chrome://extensions with Developer mode on.

Common mistakes

• Fixed sleeps: waitForTimeout(2000) is a guess. Wait for a state - a locator action or a retrying assertion - instead.
• Selectors copied from the inspector: a long CSS path with generated class names breaks on the next build. Prefer role and name, labels and test ids.
• Tests that share a profile or data: one test's cookies or records change the next one's result. Give each test its own context and its own data.
• Retrying until green: retries hide the cause. Record a trace on retry and treat a test that needs retries as flaky.
• An extension that trusts messages: any page can post a message that looks like yours. Check the sender's tab, frame and origin, and never let a page add a step.
• A server-side browser with open network access: without an origin allowlist and a connection-level check, a redirect can take it to cloud metadata or a private database.

Interview questions

• How would you choose between Selenium, Cypress, Puppeteer and Playwright for a new project? Selenium when you need Safari, real devices or an existing Grid; Cypress when a front-end team tests its own app in JavaScript and wants time-travel debugging; Puppeteer for small Chrome-only scripts such as PDFs; Playwright for cross-browser end-to-end tests with auto-waiting, isolated contexts and traces. I would also weigh team language, parallelism needs and CI cost.
• What is the difference between WebDriver and the Chrome DevTools Protocol? Classic WebDriver is a W3C standard where each command is an HTTP request to a driver program, and the test must poll to learn what changed. CDP is Chromium's own protocol over one WebSocket that carries commands and pushes events, so a tool knows when the page is busy. WebDriver BiDi brings that two-way style to a cross-browser standard.
• Why are Playwright tests usually less flaky than classic Selenium tests? Every action waits for the element to be visible, stable, enabled and not covered, and assertions on locators retry until a timeout. Each test also gets a fresh browser context, so state does not leak between tests.
• What is a browser context and why create one per test? An isolated profile inside one running browser, with its own cookies, storage and cache. It costs milliseconds, so you keep one browser per worker and still give every test a clean start.
• How do you choose a selector that survives a redesign? Prefer what a person sees or what developers promise: a test id, or role plus accessible name, then labels and text; use CSS or XPath last and never generated class names. Make sure it matches exactly one element, and scope it to a container when the same button repeats.
• What is the accessibility tree and why does getByRole use it? A tree the browser builds beside the DOM with each element's role, accessible name and state, the view a screen reader has. Matching on it ties the test to meaning rather than markup, and pushes the team towards accessible pages.
• A test fails once in ten runs. How do you investigate? Repeat it many times, record a trace on failure and compare a passing and a failing trace: timing, data, overlays and network calls are the usual causes. Fix the cause, for example with a retrying assertion or fresh data, instead of adding a sleep or a retry.
• What changed with Manifest V3? The background page became a service worker that the browser stops when idle, remote code is banned, and permissions are stricter. State must live in chrome.storage and delayed work in chrome.alarms, because memory and timers do not survive a stop.
• How does a recorder turn raw events into steps? A content script listens in the capture phase, keeps only trusted events, finds the actionable element, merges typing into one fill and separates single from double clicks. It describes the element with ranked, verified selectors and masks secrets before anything leaves the page.
• How do you stop a server-side browser reaching internal systems? Allow only the site's origins for navigation, block service workers and downloads, and check every connection through an egress proxy that resolves the name itself and connects to the address it checked, which also defeats DNS rebinding. Add a network firewall underneath as the last layer.

Related

• /help/technology/http-api-testing
• /help/technology/testing
• /help/technology/secure-links-privacy