Automation & applied AI
Replacing a paid data vendor with a browser
An internal prospecting tool that drives a real Chromium instance to source businesses, enriches them from their own websites, and writes a personalised opening line for each with an LLM.
Project snapshot
- Platform
- Internal tool
- Industry
- Sales automation
- Team
- Solo build
- Interface
- Streamlit dashboard
- Runtime
- Local-first
- Modules
- 11
- Status
- In development
- Built
- 2026
- Primary language
- Python
- Last updated
- 2 weeks ago
- 22 July 2026
Technology
- Python
- Streamlit
- Playwright
- Headless Chromium
- Groq API
- Gmail SMTP
- App passwords
- PBKDF2
Overview
A prospecting pipeline that costs nothing per search
Built to answer one question: can the work a per-record data vendor charges for be done locally, and what does it actually cost you to do that?
Lead Finder takes a business type and a location and returns a qualified, contactable list — name, address, phone, website, rating and category — then visits each business's own site looking for a public email address and social profiles, and can send every one of them a personalised cold email.
It began as a wrapper around a paid third-party scraping service. That service worked well and cost money on every search, which put a meter on exactly the activity the tool exists to encourage. Replacing it meant taking on the work the vendor was doing, and being honest about the two things that got worse in the process.
It is an internal tool, and it is built like one: a Streamlit dashboard, local execution, no infrastructure to operate. That is a deliberate scope decision, not an unfinished state — the fastest way to make this thing worse would be to turn it into a hosted product nobody asked for.
The challenge
What made this hard
A metered cost on the core action
The original implementation called a paid scraping service. Every search had a price, which meant the tool's usefulness and its cost grew together — precisely the wrong incentive for something meant to be used freely during prospecting.
Source pages that are not an API
Google Maps markup is not public, not stable, and not documented. Anything reading it is reading an implementation detail that can change without notice and break extraction silently rather than loudly.
Generic contact data
A regex sweep of a company's own website finds info@ and contact@ far more often than it finds a named decision-maker. That is a real, permanent quality reduction against a curated B2B database, and no amount of engineering removes it.
Credentials in a tool that sends email
The system holds a Gmail app password and an LLM API key, and can send mail on the operator's behalf. A dashboard reachable over a URL with that capability behind it needs a genuine access gate, not a token check.
Personalisation that does not read as generated
A mail-merged first name is not personalisation and every recipient knows it. The opening line has to reference something specific enough about the business to be worth reading — at a per-message cost near zero.
Research & strategy
The decisions behind it
The tradeoffs here are reproduced from the project's own README, which states them plainly. They are not softened.
Buy the data, or scrape it?
ChoseDrive a real headless Chromium via Playwright, locally.
A real browser executes the page's JavaScript, so it sees what a user sees rather than what a raw HTTP fetch returns. It scrolls the results list to load listings, then opens each business's own page to pull the phone number, website and full address. No API key, no per-search cost, no usage ceiling.
TradeoffGoogle's terms prohibit scraping Maps, and the markup is undocumented and changes periodically — which can break extraction silently until the selectors are updated. That maintenance is the cat-and-mouse game the vendor was being paid to play. Choosing this means choosing to play it yourself.
How are email addresses found?
ChoseFetch each business's homepage plus likely contact and about pages, and extract with regex.
A business that wants to be contacted publishes an address. Checking the two or three pages where one is conventionally placed catches most of them at negligible cost.
TradeoffContact quality is materially lower than the curated database it replaced. That produced named individuals with direct addresses and job titles; this mostly produces role accounts. It is the single biggest regression in the rebuild, and it is a property of the approach rather than a bug in it.
Should Apollo.io be integrated for enrichment?
ChoseNo.
Apollo's terms prohibit unauthorised scraping, and doing it properly requires their official paid API — which reintroduces exactly the per-record cost the rebuild existed to remove. Deliberately left out rather than quietly worked around.
How should the dashboard be secured?
ChoseA salted PBKDF2 hash in the environment, constant-time comparison, and a file-based lockout.
A setup script prompts for the credential with the terminal's hidden-input mode and writes only the hash — the plaintext is never stored and never displayed anywhere in the app. Comparison uses `hmac.compare_digest`, so response timing leaks nothing about how much of a guess was correct. Five failures lock out all attempts for fifteen minutes, tracked in a local file rather than a cookie, so a new tab or an incognito window does not reset it.
TradeoffIt is one shared credential, not per-person accounts, and it protects against someone finding the URL — not against a compromised machine or a stolen .env. The README states that limit explicitly rather than implying more.
Which model writes the personalised line?
ChoseGroq, on its free tier.
The task is one short sentence conditioned on a few facts about a business — well within a small fast model's competence, and latency matters more than sophistication when generating for a whole list. The free tier makes the marginal cost of personalising a campaign effectively zero, which is what allows it to be on by default.
Capabilities
What it does
Local Maps sourcing
A headless Chromium instance scrolls the results list, opens each listing, and extracts name, address, phone, website, rating and category — with a masked fingerprint and human-paced scrolling to reduce the chance of being blocked.
- No API key
- No per-search cost
- No result ceiling
Contact enrichment
Each business's own site is fetched — homepage plus likely contact and about pages — and swept for email addresses and LinkedIn, Facebook, X and Instagram profiles.
- Public sources only
- Social profiles alongside email
LLM personalisation
An opening line written per recipient from what was found about that specific business, so the first sentence is not one every other recipient also received.
- One line per lead
- Effectively zero marginal cost
Hardened access gate
Salted PBKDF2 hashing, constant-time comparison, and a fifteen-minute lockout after five failures that survives a new tab because it is tracked on disk rather than in a session.
- Plaintext never stored
- Timing-safe comparison
- Lockout cannot be reset client-side
Campaign sending
Sends through Gmail using an app password scoped to this tool alone — revocable independently, and never the account's real password.
- Revocable credential
- Templated body with a personalised opening
Filtering and export
Filter by minimum rating, skip permanently closed businesses, target only businesses without a website, or keep only leads where an email was found. Export the result as CSV.
- CSV export
- Composable filters
Engineering
How it is built
- Interface
Streamlit — a dashboard with no frontend build, chosen because the audience is one operator.
- Every page gated behind the login
- Filters and export in the same view as results
- Acquisition
Playwright driving headless Chromium against Google Maps.
- Scrolls the results pane to trigger lazy loading
- Visits each listing for phone, website and full address
- Masked fingerprint and paced delays to reduce blocking
- Enrichment
Direct HTTP fetches of each business's own site, with regex extraction.
- Homepage plus conventional contact and about paths
- Email addresses and four social networks
- Generation
Groq's API for the per-recipient opening line.
- Conditioned on the facts gathered for that business
- Free tier, so personalisation stays default-on
- Security
PBKDF2 with a salt, constant-time comparison, and disk-backed brute-force lockout.
- Credential set by a CLI script using hidden terminal input
- Only the hash reaches .env; no plaintext anywhere
- hmac.compare_digest defeats timing analysis
- Five failures lock all attempts for fifteen minutes
- Configuration
Everything through .env — no secret is ever typed into the running app.
- Groq key, Gmail address and app password, credential hash
- A committed .env.example documents each
Process
How it was built
First build
Vendor integration
Shipped against a paid third-party scraping service. It worked, and it put a per-search price on the tool's core action.
- Working pipeline
- A metered cost
Rebuild
Bringing it in-house
The vendor call was replaced with a locally driven browser, taking on the extraction and the maintenance it implies.
- local_scraper.py
- contact_enricher.py
Extension
Personalisation
LLM-written opening lines and templated sending, turning a list of contacts into a campaign that can actually be sent.
- personalizer.py
- send_campaign.py
Current
Hardening
Access control added once the tool could send mail on the operator's behalf, with the limits of that protection documented rather than implied.
- auth.py
- set_credentials.py
Technology
The stack, by layer
Grouped the way the system is actually organised, not as an alphabetical logo wall.
Application
- Python
- Streamlit
Acquisition
- Playwright
- Headless Chromium
AI
- Groq API
Delivery
- Gmail SMTP
- App passwords
Security
- PBKDF2
- hmac.compare_digest
- Rate limiting
Data
- CSV export
- Regex extraction