Prompt
How do I troubleshoot browser automation that gets stuck after a popup or cookie banner?
Latest observation
When browser automation gets stuck after a popup or cookie banner, the usual problem is that the script is still targeting the page underneath, while the popup is intercepting clicks or keyboard focus. Here’s a practical troubleshooting checklist.
1) Confirm what kind of blocker it is
Common blockers:
- Cookie consent banner
- Modal dialog / overlay
- Alert/confirm prompt
- New tab or popup window
- Iframe-based consent widget
- Hidden full-page overlay with
z-index
2) Inspect the page state before interacting
Add logging or screenshots right before the step that fails:
- Current URL
- Number of open tabs/windows
- Screenshot
- HTML of top-level overlays if possible
If using Playwright/Selenium, a screenshot often makes the issue obvious immediately.
3) Wait for the page to be ready, but don’t rely only on page load
A page can be “loaded” while a popup is still blocking:
- Wait for network idle only if appropriate
- Wait for the specific element you need
- Check whether an overlay is visible
4) Handle popups explicitly
Cookie banners / modals
Try:
- Click “Accept”, “Reject”, “Continue”, or “Close”
- Use selectors for accessible labels/text
- If inside an iframe, switch into the iframe first
JavaScript alerts/confirm/prompt
These require special handling:
- Accept/dismiss the dialog
- Attach a dialog handler before the action that triggers it
5) Check for overlays blocking clicks
If clicks fail, common causes are:
- Overlay covers the target
- Element is off-screen
- Element is present but not visible
- Another element intercepts pointer events
In that case:
- Scroll element into view
- Wait for overlay to disappear
- Use a more specific selector
- Verify with a hover or
isVisible/isDisplayedcheck
6) Handle iframes and shadow DOM
Cookie banners are often embedded:
- In an iframe
- Inside a shadow root
If your script can’t find the button, inspect the DOM structure. A selector that works in the main document may not work inside the frame.
7) Watch for stale or detached elements
After closing a popup, the DOM may re-render. If your script saved an element reference earlier, it may be stale:
- Re-query the element after the popup closes
- Don’t reuse old handles after navigation or DOM updates
8) Add a generic popup-handling step
A good pattern is:
- Detect known banners/modals
- Close/accept them
- Retry the original action
This is especially useful for flaky sites.
9) Use timeouts carefully
If the script “hangs”:
- Lower the timeout on the failing step
- Catch the timeout and take a screenshot
- Log which selector or action was attempted
This helps distinguish “slow page” from “blocked by popup.”
10) Debug interactively
Run the browser in non-headless mode and slow down actions:
- Watch where the popup appears
- See whether the click happens too early
- Verify whether the banner reappears after navigation
11) If possible, block the popup upstream
Depending on your goal:
- Pre-set cookies/localStorage to indicate consent
- Use a test profile with consent already stored
- Disable notifications/popups in the browser profile
- Mock or stub the consent API in test environments
12) Example debugging flow
- Take a screenshot when the script gets stuck
- Check whether a banner/modal is visible
- Try closing it manually in the script
- If not found, inspect if it is in an iframe or shadow DOM
- Retry the blocked action
- If it still fails, log the DOM and selectors involved
13) Minimal checklist
- Screenshot before failure
- Detect and dismiss popups
- Handle iframes
- Re-locate elements after DOM changes
- Wait for overlays to disappear
- Use explicit, robust selectors
- Run non-headless with slow motion for debugging
If you want, I can also give you:
- a Playwright-specific troubleshooting recipe, or
- a Selenium-specific one with example code.
Brands