Prompt
I'm building a browser automation system for sites with logins, 2FA, and popups; what should I consider?
Latest observation
Nov 16, 2026
OpenAI APIWeb search: off
If you’re building browser automation for sites with logins, 2FA, and popups, the main things to consider are:
1) Reliability and session handling
- Persist cookies/storage safely so users don’t have to log in every run.
- Detect session expiry and have a clean re-auth flow.
- Expect redirect chains after login and handle them deterministically.
- Use waits for page stability instead of fixed sleeps:
- DOM ready
- network idle
- specific element visible/enabled
2) Login flows
Login pages vary a lot, so build for:
- Username/password forms
- OAuth/SSO redirects
- “Magic link” email flows
- “Remember this device” prompts
- CAPTCHA or bot checks
- Multi-step login forms
Good practice:
- Use selectors that are robust but not too brittle.
- Maintain a per-site adapter/config for custom flows.
- Capture screenshots and DOM snapshots when login fails.
3) 2FA handling
2FA is often the hardest part.
Plan for:
- TOTP apps (e.g. authenticator codes)
- SMS/email OTP
- Push approvals
- Backup codes
- WebAuthn/security keys (hardest to automate safely)
Recommendations:
- Prefer user-in-the-loop completion for 2FA unless you explicitly own the account and have permission to automate it.
- Store secrets securely if you must support TOTP.
- Design a “pause and resume” flow so the browser can wait for human action.
- Timeouts should be long enough for real users, but bounded.
4) Popups, modals, and overlays
Sites often interrupt flows with:
- Cookie consent banners
- Newsletter popups
- “Welcome” tours
- Age gates
- App install prompts
- Chat widgets
- Permission dialogs
Approach:
- Handle them with a dismissal layer that runs before/after each major step.
- Prefer clicking explicit “accept/close/skip” buttons.
- Be careful not to close modals that are actually part of the workflow.
- Support multiple overlays at once, since they can stack.
5) Anti-bot and abuse detection
Many logged-in sites monitor automation.
You should consider:
- Rate limiting your actions
- Human-like pacing where appropriate
- Avoiding excessive parallel logins
- Stable fingerprints across sessions
- Detecting challenges and falling back gracefully
Important:
- Don’t build around bypassing security controls. Keep automation aligned with site rules and user consent.
6) Security and secrets
Since you’ll be handling credentials and 2FA:
- Encrypt secrets at rest
- Avoid logging passwords, OTPs, or full session tokens
- Separate per-user credential storage
- Use least privilege for any backend that stores auth material
- Rotate and revoke stored sessions if compromised
7) Site variability and maintenance
Web UIs change frequently:
- Buttons get renamed
- DOM structures change
- A/B tests alter flows
- Different locales change labels
- Mobile vs desktop views differ
Mitigations:
- Build a site-specific configuration layer
- Use semantic locators when possible:
- labels
- role/name
- accessibility tree
- Create observability: screenshots, traces, step logs, HAR/network logs
- Add alerting when failure rates increase
8) Human handoff
For tricky steps, support:
- Live user takeover
- Remote browser viewing
- “Click to continue” prompts
- SMS/email OTP entry by the user
- Manual recovery when account risk checks appear
A good system often mixes automation with human confirmation.
9) Timeouts, retries, and idempotency
- Make every step safe to retry
- Distinguish transient failures from hard failures
- Use bounded retries with backoff
- Ensure actions like “submit” or “buy” aren’t executed twice accidentally
- Verify state after each critical action
10) Testing strategy
Test against:
- A curated set of real sites you’re authorized to automate
- Mock login/2FA sandboxes
- Different browsers and viewport sizes
- Slow networks and intermittent failures
- Locales and time zones
Also:
- Record and replay happy-path flows
- Regression-test selectors and popup handling
- Maintain fixtures for common auth patterns
A practical architecture
A useful design is:
- Core runner: navigation, waits, retries, screenshots
- Auth module: login/session/2FA handling
- Popup manager: dismiss common interruptions
- Site adapters: site-specific logic
- Human handoff layer: pause/resume and approvals
- Telemetry: logs, traces, screenshots, metrics
Final recommendation
Treat authentication as a workflow, not a single step:
- Check existing session
- Attempt login
- Handle 2FA
- Dismiss popups
- Verify authenticated state
- Persist session securely
- Recover gracefully on expiration
If you want, I can also help you design:
- a reference architecture
- a Playwright/Puppeteer implementation pattern
- or a state machine for login + 2FA + popup handling
Brands