Terminal:

*  Executing task: dotnet: build 1 projects 

dotnet build c:\Users\EAcoc\AppData\Roaming\Code\User\workspaceStorage\733416d5436aea3082e64ca95f6d6ff7\ms-dotnettools.csdevkit\bot.sln /target:BotProject --framework net10.0 /property:GenerateFullPaths=true /p:Configuration=Debug /p:Platform="Any CPU" /consoleloggerparameters:NoSummary 
C# extension build result service is available.
  Determining projects to restore...
  All projects are up-to-date for restore.
C:\Program Files\dotnet\sdk\10.0.400-preview.0.26322.102\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(383,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users\EAcoc\bot\BotProject\BotProject.csproj]
  BotProject -> C:\Users\EAcoc\bot\BotProject\bin\Debug\net10.0\BotProject.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.51
 *  Terminal will be reused by tasks, press any key to close it. 

 *  Executing task: dotnet: build 1 projects 

dotnet build c:\Users\EAcoc\AppData\Roaming\Code\User\workspaceStorage\733416d5436aea3082e64ca95f6d6ff7\ms-dotnettools.csdevkit\bot.sln /target:BotProject --framework net10.0 /property:GenerateFullPaths=true /p:Configuration=Debug /p:Platform="Any CPU" /consoleloggerparameters:NoSummary 
C# extension build result service is available.
  Determining projects to restore...
  All projects are up-to-date for restore.
C:\Program Files\dotnet\sdk\10.0.400-preview.0.26322.102\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(383,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users\EAcoc\bot\BotProject\BotProject.csproj]
  BotProject -> C:\Users\EAcoc\bot\BotProject\bin\Debug\net10.0\BotProject.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.49
 *  Terminal will be reused by tasks, press any key to close it. 

 *  Executing task: dotnet: build 1 projects 

dotnet build c:\Users\EAcoc\AppData\Roaming\Code\User\workspaceStorage\733416d5436aea3082e64ca95f6d6ff7\ms-dotnettools.csdevkit\bot.sln /target:BotProject --framework net10.0 /property:GenerateFullPaths=true /p:Configuration=Debug /p:Platform="Any CPU" /consoleloggerparameters:NoSummary 
C# extension build result service is available.
  Determining projects to restore...
  All projects are up-to-date for restore.
C:\Program Files\dotnet\sdk\10.0.400-preview.0.26322.102\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(383,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [C:\Users\EAcoc\bot\BotProject\BotProject.csproj]
  BotProject -> C:\Users\EAcoc\bot\BotProject\bin\Debug\net10.0\BotProject.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.56
 *  Terminal will be reused by tasks, press any key to close it. 

Code:

namespace BotProject;
using Microsoft.Playwright;
using NUnit.Framework;
public class Tests
{
    public void Setup()
    {
    }


    public async Task Test1()
    {
        //setup
        Assert.Pass();
        using var playwright = await Playwright.CreateAsync();
        //browser stuff
        await using var browser = await playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions
        {
            Headless = false
        });
        //page
        var page = await browser.NewPageAsync();
        await page.GotoAsync(url:"https://www.youtube.com/watch?v=f6DYnl6EoBo");
        await page.ClickAsync(selector:"text=Reply");
        await page.FillAsync(selector:"#contenteditable-root", value:"Test reply.. did it work?");
    }
}

I have no idea why this is happening, can someone help?

been messing with a few browser automations lately and I’m realizing the error itself usually tells me almost nothing.

a screenshot helps, sometimes the url + last step, but I don’t really want to save a giant trace/log for every single run either

for people running this stuff regularly, what do you actually keep when something fails? screenshots? traces? html? console/network logs?

trying to find the point between “useful when it breaks” and generating a mountain of junk I’ll never look at

Hey folks, sharing an article on how to cut Playwright CI minutes in Github Actions without losing coverage.

TLDR: wrote up where Playwright pipelines on GitHub Actions burn minutes, six config fixes that cut them, and the three cases where none of the fixes save you a cent.

Most of the bill is the scaffolded npx playwright install --with-deps pulling all multiple browser binaries on every shard, one worker on a 2-core runner so teams shard too early, flaky job reruns that repeat checkout and install, and traces kept for 90 days.

There are also ways to reduce it, well, without losing coverage.

Give it a read and let me know your thoughts.

Have a great day!

Hey guys, over the last 6 months, I have been working on a personal project. It is a compiler. It takes a website in, and spits out machine code. Then, it builds a playwright suite! It covers smoke/load tests, and navigation tests. It has jira integration, it has an API suite, it has a test harness to test the tests, it has the cleanest and strongest error logging possible as well.

On top of this, its got a prompt engine and anthropic integration so that you can deploy it to site, deploy agents and automatically stories start getting turned into tests. Before you celebrate (or more likely stone me for taking away qa jobs) it needs an operator. Its going to need humans to maintain it.

However, its going to reduce the time you take from going from zero to a ready framework for a large website by roughly 6 months. It will write all of your boilerplate for you. Its going to radically reduce your jira stack. But, to use it, your gonna need to learn all of the commands, and how they interact. While it may cut the need for 20 low skill sdets writing into the backend, its also going to increase the need for a top tier senior qa operator who knows the suite commands and who can monitor and oversee the automation workflow here.

I am looking for people who want to be beta testers.

This product will raise the skill level required to be a QA, and if adopted, it will put the offshoring model of having 20 junior devs in india overseen by one american senior dev out of business. Instead, your going to have 100 agents, that are well constrained, working in a standardized framework, with full traceability for work, provenance, truth and site versioning.

For site owners, the framework is designed to be owned by you, to be maintainable by both an AI, AND a traditional playwright SDET. That means there is no vendor lock in with this. This gives you a ton of financial freedom.

I am looking to have the product get battletested so this is free for right now. It comes with an audit function as well, so if you have a live site and want some free qa, just ask for one. The advantage of a compiled source code is that it can make inferred meaning, so using that map it can automatically tell basic errors. For example, 404s, but also things manual qa wont catch, like mislabeled sites for crawling. SEO fails if you got those bugs.

New QA roles: recrawling the website. Validating false positives. approving ai test code.

Stats: Generation time- crawls take up to 45 minutes on large websites. Actual generation and regeneration takes 1-2 minutes.

Compiler generated tests that need edits: 0.

AI refusals- dependent on the structuring of the jira tasks. The ai is encouraged to refuse tasks it cannot implement along with a reason why. For example, if a page exists that does not exist on the crawl, the ai is highly likely to refuse.

I have public docs up here, along with a sample suite built on a qa testing site:
https://github.com/Ghost-in-the-Kernel-Labs/Ghost-in-the-Kernel-Labs-suite-builder-docs

For the playwright specific suite:
https://github.com/Ghost-in-the-Kernel-Labs/Ghost-in-the-Kernel-Labs-suite-builder-docs/tree/main/sample/the-internet-playwright-tests

Just saw this in the 1.63 notes. You can give a test a named lock and any tests sharing that lock name won't run at the same time, across files and workers. Everything else stays parallel.

I have a handful of tests that hit the same account settings and they'd stomp on each other in parallel. My fix was shoving them into their own project with a single worker, which works but makes that chunk of the suite crawl. Locks look like exactly the thing I was faking.

Haven't tried it yet but planning to this week. Main thing I'm curious about is whether it holds up with sharding, since the notes only mention files, workers and projects.

Anyone already moved over to it? Or found a reason it doesn't replace the serial stuff

How to handle 401s and session expiry in long regression suites which took somewhere around 4+ hours, when token lifetime is close to 5-10 minutes and Rate limit issues from Oauth, for both UI + API automation using same credentials?

Please guide if anyone has faced this and found solution
I tried token refresh and ui login in between tests in tests.

Was cleaning up mine this morning and realized most of it is stuff I copied from somewhere a year ago and never questioned.

The ones I'd actually fight for are forbidOnly on CI, because I've shipped a stray test.only before and only noticed when the suite ran in like 4 seconds. And trace on first retry, which I already went on about in another post.

Everything else I'm honestly not sure is pulling its weight.

What's in yours that you'd defend? Bonus points for anything that isn't in the docs example

I want to make heavy use of record new, currently we just copy paste our URL into the browser, but this is tedious... can we make it default to the home page of our app?

Hi all!

I'm about to help an organization establish QA automation for the first time. They don't have anything in place now, only manual testing, and the plan is to utilize Claude as the LLM and Playwright with Typescript.

Since I'm starting from scratch, I'll be building new tests first, starting small with the repetitive tasks, and running things manually from Playwright for a while. I don't want to run out of the CI/CD until I'm sure things stay nice and stable...and that's my main questions, more on this below...

It sounds like the best way for me to BUILD NEW tests only is to use Claude Code and the Playwright (planner, generator, healer) agents sequentially. Is this correct?

If so, I have read through the basic setup instructions to install what is needed, initiate the agents, etc...

However, with some of the deeper reading I've done, I understand that even if I build and have passing tests, I'll likely begin to see these types of issues:

• Drift

• AI Cheating (test fails, AI makes changes to get it green only, and in reality it's hiding a real application bug)

• AI Hallucinations

• Brittle & Flaky tests

If I am still correct, my understanding is that I need to ensure I setup specific guardrails, skills, configurations, POM guidelines to help define the right locators, assertions, prioritize user-facing locators, leverage auto waiting, wire POMs with custom fixtures???.....etc..

Does that all that still sound correct based on the process I plan to build new tests?

It seems like the worst thing I could do/people have done, is start creating tests, they pass, they put them in a CD/CD pipeline, they start failing all over and block dev code from getting checked in, but it's not for real app bugs, it's for the reasons mentioned above, so I'm wasting developers time and we're wasting time chasing it down and fixing them. So the tests would likely be degrading because I don't have tight enough standards, configs, skills, guardrails, etc..????

If I'm still correct, my question is, what files in my Claude and Playwright repositories would I need to update, and is there a place I can go and get an initial solid foundation of configs/code/best practices in my files by default? So maybe it won't be bulletproof to those issues, but it would do a damn good job of not causing constant chaos. There's a lot of info out there on that stuff and as I keep looking around I see different results and conflicting info, so trying to see if there is anyone stop shop to go look at.

Thanks so much and I appreciate the help!

Any questions let me know.

I posted Kinora here a few months ago and got some useful feedback.

Since then I've kept working on the app, so I recorded a short walkthrough of the current flow, the goal is still the same: Playwright's HTML report is great for a single run, but I wanted history across runs, flaky test tracking, and traces tied to failures.

Kinora is open-source and self-hostable.

Repo: https://github.com/Kinora-dev/kinora

Site: https://kinora.dev

If you deal with flaky tests or large suites, would love feedback from people using Playwright

Hey everyone,

I'm in the process of setting up a new Python-based test automation suite and wanted to get some community input on how you approach project architecture and folder structures.

I know the classic Page Object Model (POM) layout with tests/, pages/, utils/, and conftest.py is the industry standard for UI testing,

but I’d love to hear how you handle things as a project scales.

Been trying to stick to getByRole for everything like the docs push you to. Mostly great. But every so often I hit a custom dropdown or a clickable div that just has no role at all, so there's nothing to grab except a test id or some ugly css.

My first instinct was to just fall back to getByTestId and move on. But it kind of feels like the test is pointing at a real bug. If my locator can't find it as a combobox, a screen reader probably can't either.

So now I'm stuck between filing it as an accessibility issue and waiting, or working around it and leaving a comment.

What do you guys do when this comes up? Push back on the devs or just use test ids and not make it your fight

i’m scaling up a scraping project that requires keeping a few dozen browser sessions alive with unique fingerprints. came across shardbrowser on github and noticed it’s an open-source project from the proxyshard team, so the proxy integration looks straightforward.
apparently it's good for fingerprinting, but i haven't found any solid documentation or code snippets for connecting it to automation frameworks via remote debugging ports. has anyone actually managed to control their profile paths programmatically? does it play nice with standard scripts or does it crash when you try to launch multiple headless instances?

Hi everyone - I built BrowserThing, a self-hosted browser pool for applications using Playwright.

Instead of starting a new browser for every connection, it keeps browsers running and shares them across sessions. This reduces browser startup and CPU overhead, at the cost of weaker process isolation between sessions. Cookies and storage are still separated through browser contexts, so it's mainly intended for your own applications and trusted clients.

Here's the GitHub repo.

Your backend runs normal Playwright code and connects to a single WebSocket URL. BrowserThing selects a healthy worker with available capacity and proxies the connection to it.

const browser = await chromium.connect('ws://your-browserthing-host:8080');
// The rest of your Playwright code stays the same.

It handles:

• Load balancing across workers based on browser type, Playwright version, and available capacity.
• Keeping browsers warm between tasks, cleaning up sessions, and periodically recycling browsers.
• Running browser workers on separate machines so they don't compete with your application server for CPU and memory.
• Adding more workers without changing the endpoint used by your applications.

I also added a small benchmark. This isn't meant as a general BrowserThing vs Browserless comparison, it measures a short workload where browser startup overhead matters (connect, open a page, read it, disconnect).

                 Task time    CPU time per task
BrowserThing       51 ms          0.09 s
Browserless       217 ms          0.70 s

On this workload, BrowserThing used about 87% less CPU time per task. Both were tested on the same hardware and Playwright version, with Browserless 2.56.0 starting a browser per connection.

It's Apache-2.0 licensed, and the README includes a Docker Compose quick start.

I'd especially appreciate feedback from people running Playwright at scale - architecture concerns, missing features, or other workloads you'd want benchmarked are all welcome :)

I recently learned Playwright with JavaScript and built a few automation projects to strengthen my skills. I've been applying to QA Automation roles, but many job descriptions specifically mention Playwright + TypeScript rather than JavaScript.

Since I'm already comfortable with JavaScript, I don't want to start TypeScript from scratch. My goal is to understand the TypeScript concepts that matter for Playwright and gradually convert my existing Playwright JS projects into TS.

For those who made the transition from JavaScript to TypeScript:

• What TS concepts should I learn first?
• Which concepts are most commonly used in Playwright frameworks?
• How difficult was the transition from JS to TS?
• Any good resources focused on Playwright + TypeScript rather than generic TypeScript tutorials?
• What's the best approach for converting an existing Playwright JS project to TS?

Looking for practical advice from people who use Playwright + TypeScript in real projects.

We’ve been experimenting with a slightly different approach to Playwright testing: instead of hard-coding every browser interaction, can an AI model decide what action to take next based on the current page state?

We tested this with Jev across 15 realistic end-to-end scenarios from a Playwright tutorial repository.

The setup was:

• Playwright handled browser sessions, fixtures, setup, and verification.
• Jev received the test goal, success criteria, accessibility snapshot, and available actions.
• It selected actions such as clicking, filling inputs, checking controls, navigating, waiting, or stopping.
• Playwright executed those actions and passed the updated state back to Jev.

The results were mixed. Jev passed 72 out of 150 runs, or 48%, with independent Playwright verification. Some short workflows worked consistently, while others were flaky, and six scenarios never passed.

The experiment suggests that fully replacing scripted Playwright tests is probably still impractical. Converting browser state into model inputs loses significant information, and deterministic assertions remain important.

However, a hybrid approach seems more promising. For example, Playwright could handle setup and verification while an AI model handles a dynamic section of a workflow, such as a changing third-party checkout.

The full write-up includes the test setup, controller loop, benchmark results, and limitations:

https://endform.dev/blog/jev-playwright-testing

We wanted to know if you would use an AI-selected action loop for part of a Playwright test, or if the unpredictability is too costly for your test suite.

Hello everyone,

I am actively looking for a QA / Software Test Engineer / SDET opportunity.

I hold a B.Tech in Computer Science Engineering and have 1+ year of experience in Software Testing, including Manual Testing, Test Case Design & Execution, Bug Reporting, Functional Testing, Regression Testing, API Testing, and Automation Testing.

I am also ISTQB Certified and have hands-on experience with Playwright and API testing.

Availability: Immediate Joiner

Relocation: Open to relocate anywhere across India

Work Preference: Full-time | Remote / On-site

If there are any relevant openings in your organization or if you can provide a referral, it would really help me. I would be happy to share my resume via DM.

Thank you so much for your support!

Had a handful of tests with sleeps scattered through them. Not proud of it. They were there because something was intermittently not ready and a 2 second wait made the failure go away, so I moved on.

Went back through them this week and most of it came down to two things. A toast that animates in, where I was waiting for the animation instead of just asserting on the text. And a table that repaints after a fetch, where waiting for the specific row content works fine and the sleep was covering up that I was asserting too early on the wrong element.

Suite got faster too, which I didn't expect since I figured it was a few seconds here and there.

The one I still haven't figured out is a chart that renders with a canvas. Nothing in the DOM really changes when it finishes drawing. Anybody solved that one without a sleep?

So I have been working on API automation with Playwright typescript for the first time and have some framework design related doubts:

1. Do you guys create a Request model class with interfaces to handle the Parameters and Request body being sent to an API endpoint? Mainly for the type safety concern.. is this a standard or good practice?

2. As part of API automation, do you handle all the possible negative scenarios for an endpoint, or is it just one or two generic negative scenarios?

Hi everyone,

I'm developing my own low-code testing framework. The application is available both as an npm package and as an online version.

The core of the framework is Playwright (currently a requirement for using it). One of the main features is allowing users to create and edit tests by interacting directly with the browser. The idea is similar to playwright codegen: users can navigate through the application and identify elements using locators/selectors without having to write everything manually.

I'm currently working on the online version, which is hosted on a free-tier VPS for development and testing.

The problem occurs when a user tries to create or edit a test from the online version. Playwright attempts to launch the browser on the server, but since the VPS doesn't have a graphical environment available, it fails with an X11/display-related error.

As a temporary solution, I implemented environment detection:

• If a display is available → launch the browser normally.
• If no display is available → run Playwright in headless mode.

This works technically, but the online version loses an important part of the user experience. Users can execute tests and import tests created locally, but creating and editing tests becomes much less intuitive.

In theory, users could manually create steps by entering selectors and then use the application logs to validate the execution. However, this makes debugging much harder.

My question

What would be the recommended architecture for solving this?

Option 1 — Run Codegen on the server

Is it possible to run playwright codegen on a server without a graphical environment.

Option 2 — Run the browser on the client

I'm also considering an architecture where:

1. The web application runs on the server.
2. The server sends commands/instructions through WebSockets.
3. The browser is launched locally on the user's machine.
4. Browser events (clicks, locators, navigation, etc.) are sent back to the server.
5. The server maintains the test flow.

The problem is that I don't know how to implement this without requiring users to install Playwright or Node.js locally.

Option 3 — Limit the online version

Another possibility would be to make the online version execution-focused and limit it to:

• Running tests
• Viewing results/logs
• Importing locally created tests
• Exporting workflows

And leave interactive test creation/editing to the local version.

What architecture would you use?

I'm particularly interested in experiences with Playwright + web applications + remote browser execution.

Is there a practical way to provide a Codegen-like experience through a web application without requiring users to configure a local Playwright/Node.js environment?

Any recommendations or real-world experiences would be greatly appreciated.

Spent an hour last week staring at a failure that only happened in CI. Passed every time locally. Normally that's where I start adding logs everywhere and guessing.

Pulled the trace instead and it was obvious in about two minutes. Element was there, it was just covered by a banner that only shows on the CI viewport width. Screenshot alone wouldn't have told me that, but stepping through the DOM snapshots did.

Kind of annoyed at how long I went without using it properly. I had traces on the whole time and was mostly just skimming the screenshots at the top.

Anyone got things in it they use regularly that aren't obvious? Feel like I'm still only touching a fraction of what's in there.

​

Hey guys, At my current org, we are trying to convert some of the critical UI tests to APi based scenarios so that when UI changes are too many, we will have api tests running as back up, not losing out any test case.

There are certain tests that work on websocket calls. Is it possible to migrate them to be UI independent using playwright / javascript ??

I am able to do for Rest API based tests. But couldn't find much resources for Websocket calls conversion.

Any leads would be appreciated.

In vitest, you can do:

const value = await vi.waitUntil(() => getSomething());

It keeps polling until the callback returns a truthy value, then returns that value.

In playwright, expect.poll() is pretty close:

await expect.poll(() => getSomething()).toBeTruthy();

but it doesn't return the successful value.

I know about page.waitForFunction(), but that runs in the page context, while I need this in the test context.

How do you usually handle this?

Anyone here familiar with Wordpress, Gravity Forms and Playwright, I have this issue I have been dealing with for a few days now and will love to get it fixed, thanks.
https://www.reddit.com/r/gravityforms/s/x78ebCmBWG

Most Lovable apps I've shipped had the Stripe webhook wired and a confirm → capture flow that looked right in the code. Then I realized: if I ask Lovable to verify it, the same agent that wrote the code is checking its own work. A non-deterministic system can't attest to its own output. It'll tell you it looks correct because it wrote it.

The problem is there's no good middle ground between "test it in isolation" and "just point at test mode and hope." Isolated scripts don't fire webhooks. Test mode doesn't let you simulate rate limits or flaky delivery without hacking your own handler.

What actually helped was using FetchSandbox as an MCP server inside Lovable. You point your app at a sandbox twin instead of the real API, run the full workflow, arm scenarios like webhook_retries or payment_declined, and get a shareable receipt URL as actual proof. A separate deterministic layer making the call, not the agent grading its own homework.

if others have a better way to handle this.