Prompt
I'm building a tool to automate a web portal login and export flow; what stack should I use?
Latest observation
Nov 17, 2026
OpenAI APIWeb search: off
A good stack depends mainly on how the portal behaves:
- Simple HTML form login, then download/export files → use HTTP automation first
- Heavy JavaScript app, anti-bot protections, SSO, or multi-step UI flow → use browser automation
- Need reliability at scale → combine both: browser for login, HTTP for export if possible
Recommended stack
1) Browser automation
Use Playwright as the default choice.
Why:
- More reliable than Selenium for modern sites
- Great waiting model for dynamic pages
- Good multi-browser support
- Works well with headless and headed runs
- Strong download handling and session persistence
Use it for:
- Logging in
- MFA / SSO flows
- Clicking through UI
- Capturing cookies/session storage
- Triggering exports/downloads
2) HTTP client for exports and API calls
Use:
- Python:
httpxorrequests - Node.js/TypeScript: built-in
fetch, oraxios
If the export endpoint can be hit directly after login, this is often:
- faster
- less flaky
- easier to retry
Typical pattern:
- Use Playwright to authenticate
- Extract cookies / auth headers
- Use HTTP client to call export endpoint directly
- Save CSV/XLSX/PDF output
3) Parsing and file handling
Depending on export type:
- CSV: Python
pandasor built-incsv - Excel:
openpyxl/pandas - PDFs:
pypdforpdfplumber - HTML scraping:
BeautifulSoup4/lxml
4) Orchestration / scheduling
If this is a recurring job:
- Lightweight: cron + container
- Moderate: Prefect or Airflow
- Cloud-native: AWS Lambda / ECS / Cloud Run / Azure Container Apps
5) Secrets and config
Store credentials in:
- Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault
- Never hardcode them in code or env files committed to git
6) Observability
Add:
- structured logging
- screenshots on failure
- HTML dump on failure
- retry logic with backoff
- metrics/alerts for failed logins or failed exports
Best practical choices by language
If you want the easiest path overall
Python + Playwright + httpx + pandas
- Great for automation + data handling
- Fast to prototype
- Excellent for export processing
If your team prefers frontend/backend JS
TypeScript + Playwright + fetch/axios
- Good if the portal is very JS-heavy
- Nice type safety
- Easy integration into web apps/services
When not to use Selenium
Use Selenium only if you have a specific reason. Playwright is usually better for new projects unless:
- your org already standardizes on Selenium Grid
- you need a legacy browser setup
- you have an existing Selenium framework you must extend
Suggested architecture
A solid production setup looks like:
- Playwright worker: logs in, handles MFA, gets session
- HTTP export module: pulls files directly if possible
- Job runner: cron/queue/scheduler
- Storage: S3/GCS/Azure Blob or local network storage
- Monitoring: logs + alerts + screenshots
Quick recommendation
If you want a straightforward answer:
Use Python + Playwright + httpx, with pandas/openpyxl for file processing.
If you tell me:
- whether the portal uses SSO/MFA,
- whether the export is a CSV/XLSX/PDF,
- and whether you need this as a script or a service,
I can recommend a more specific stack and project structure.