Lead Generation Engine

Lead Finder

A dashboard for finding and emailing cold-email leads. Type in a business type/keyword and a location, and it scrapes matching businesses from Google Maps — name, address, phone, website, rating, category — then visits each business's own website to look for a public email address and social links, and lets you export the results as CSV or send each of them a personalized cold email automatically.

How the scraping works (100% local, no API keys, no per-search cost)

• local_scraper.py drives a real (headless) Chromium browser via Playwright to search Google Maps directly: it scrolls the results list to load listings, then visits each business's own Maps page to pull its phone number, website, and full address.

• contact_enricher.py fetches each business's homepage (and a couple of likely contact/about pages) and regex-extracts any email addresses and social links (LinkedIn, Facebook, X/Twitter, Instagram) it finds.

Two things worth knowing about this approach, since it replaced a paid third-party scraper (Apify) that used to do this:

• Google's Terms of Service prohibit scraping Maps, and this includes anti-detection measures (masked browser fingerprint, human-like scroll delays) specifically to reduce the chance of being blocked. That doesn't make it risk-free — Google can still block or rate-limit the IP running it, and the underlying page markup isn't public and changes periodically, which can silently break scraping until the selectors in local_scraper.py are updated. This is the tradeoff for being free and unlimited instead of paying a service that maintains that cat-and-mouse game for you.

• Contact quality is lower than before. The old Apify actor pulled from what looked like a real B2B contact database — named people with direct emails and job titles. A regex scrape of a business's own site mostly finds generic addresses (info@, contact@, hello@) rather than named individuals, unless the site happens to list a person's email directly.

Apollo.io integration was intentionally left out — Apollo's own terms prohibit unauthorized scraping, and doing it properly requires their official paid API.

Setup

1. Create a virtual environment, install dependencies, and download the Playwright browser binary (one-time, ~180 MB):

   powershell

    python -m venv .venv
    .venv\Scripts\Activate.ps1
    pip install -r requirements.txt
    playwright install chromium

2. Copy .env.example to .env and fill in the values below. Everything reads from .env only — none of these ever get typed into the app itself.

   Groq API key (for writing the personalized line in each email):

   • Sign up free at https://console.groq.com (no credit card needed for the free tier).

   • Go to API Keys and create a new key.

   • Set GROQ_API_KEY=... in .env.

   Gmail App Password (for sending from tradyperch@gmail.com):

   • Go to https://myaccount.google.com/security and turn on 2-Step Verification if it isn't already on (required for app passwords).

   • Go to https://myaccount.google.com/apppasswords, name it something like "Lead Finder", and generate it.

   • Copy the 16-character password (remove spaces) into .env as GMAIL_APP_PASSWORD=.... Set GMAIL_ADDRESS=tradyperch@gmail.com.

   • This is a separate password just for this app — it does not change or expose your real Gmail password, and you can revoke it anytime from the same page.

3. Set your login ID and password (required before the dashboard will show anything — see Login / access control below):

   powershell

    python set_credentials.py

4. Run the app:

   powershell

    streamlit run app.py

   It opens in your browser, usually at http://localhost:8501.

Login / access control

Every page of the dashboard is gated behind a single ID/password — nobody gets past the login screen without it, whether they're on localhost or on a public deployment URL.

• Set it locally, once, with python set_credentials.py. It asks for an ID and password (typed with the terminal's normal hidden-password prompt), then writes a salted PBKDF2 hash of the password into .env — never the plaintext. Nobody, including whoever's reading this file or any AI assistant helping you edit the code, ever sees the actual password; only you, when you type it. Re-run the script anytime to change it.

• The app itself never displays it. There's no settings page, no masked field with a reveal toggle, nothing to inspect in dev tools — the same rule this project already follows for the Groq/Gmail credentials.

• Brute-force lockout: 5 wrong attempts locks out all login attempts for 15 minutes. This is tracked in a local file (.auth_lockout.json, not a browser cookie or session), so it can't be bypassed by opening a new tab or incognito window.

• Comparisons are constant-time (hmac.compare_digest), so a network attacker measuring response times can't incrementally guess the password character-by-character.

Honest limits, so expectations are calibrated correctly: this stops someone from opening the link and browsing in — it is not a claim that the app is unhackable. It doesn't protect against someone who already has your .env file, a compromised machine, or a phished password; and it's a single shared credential, not per-person accounts. If you deploy this publicly, the single most important thing is making sure the URL is only served over HTTPS (see Deploying below) — logging in over plain HTTP would send the password across the network in the clear.

Finding leads

1. Enter one or more search terms (e.g. dentist, one per line for multiple).

2. Enter a location (city, region, or country).

3. Adjust filters if needed (min rating, skip closed businesses, only businesses without a website, only leads with an email found).

4. Click Find Leads. A real browser window runs in the background (you won't see it — headless), searching and then visiting each result's page individually with human-like delays, so larger searches can take several minutes rather than seconds.

5. Review the results table, then click Download CSV if you just want the data.

Sending a campaign

Below the results, the Compose & Send section lets you email every lead that has an address:

1. Edit the subject and body. Placeholders {contact_name}, {contact_title}, {company}, and {personalized_line} get filled in per lead — {personalized_line} is a one-sentence opener Groq writes for each lead based on their website content (falls back to a generic line if the site can't be read).

2. Click Preview personalization to see how the first 3 emails would read before sending anything.

3. Set the delay between sends (default 5 minutes, to avoid looking like spam).

4. Check the confirmation box and click Start campaign. Sending runs as a background process (send_campaign.py), independent of the browser tab — it keeps going even if you close the dashboard, as long as your computer stays on. Progress is logged to campaigns/log_<timestamp>.csv and shown live under Campaign status, with a Stop campaign button.

5. If you stop and restart a campaign against the same leads file, already-sent addresses are skipped automatically (based on the log).

A couple of practical notes on the email itself: the default template signs off with a one-line opt-out ("reply and let me know" to stop hearing from you) — worth keeping for deliverability and basic courtesy even though this is B2B outreach to publicly listed business contacts, not a purchased list.

Deploying this as a public website

The login gate works the same wherever this runs — local or deployed. For putting it online for free with no credit card and no risk of a surprise bill, Streamlit Community Cloud is the right fit: it's built specifically for Streamlit apps, doesn't ask for billing details, gives you a free HTTPS URL automatically, and has no paid tier that could silently kick in. Push this project to a (private, if you want) GitHub repo, connect it at share.streamlit.io, and set GROQ_API_KEY / GMAIL_ADDRESS / GMAIL_APP_PASSWORD / AUTH_USERNAME / AUTH_SALT / AUTH_PASSWORD_HASH in its Secrets panel instead of a .env file (same idea, just entered in their dashboard instead of a local file — get the three AUTH_* values by running set_credentials.py locally first and copying its output there, not by typing a password directly into Streamlit's web UI).

One real architectural caveat before you deploy: this app drives a real headless Chromium browser (Playwright) to scrape, and runs the email campaign as a background process on the machine it's running on. Free shared hosts like Community Cloud are built for typical Streamlit apps, not for running a full browser or for background processes that need to survive tab closures and machine sleep — so scraping and campaign-sending are more reliable run the way you're running them now (on your own machine), while still putting the dashboard itself (viewing/downloading previously-scraped leads) behind the same login on a public URL if that's what you need. Avoid AWS/GCP/Azure-style hosts for this specifically because their free tiers require a card on file and can bill you if usage creeps past the free allowance — the opposite of what you asked for.

Cost notes

• Scraping: free and unlimited — it's your own machine and browser, not a paid API. The cost is time (each search takes minutes, not seconds) and occasional maintenance if Google changes their page markup.

• Groq: has a free tier; llama-3.3-70b-versatile (the default model) is inexpensive even beyond that — a fraction of a cent per email.

• Gmail: free, but Google does rate-limit and may flag high-volume sending from a personal account — the 5-minute default gap between sends exists specifically to stay well under that radar. Don't drop it too low for large lists.

If scraping stops finding results

Google's Maps markup isn't public and changes periodically. If local_scraper.py starts returning nothing, the CSS selectors it looks for (div.Nv2PK, a.hfpxzc, a[data-item-id='authority'], button[data-item-id^='phone:'], div[role="feed"]) are the first thing to check against a live page — open Google Maps in a regular browser, right-click a result, "Inspect", and compare against what the current selectors expect.

Extending later

• Add official Apollo.io API enrichment as a second pass over the CSV.

• Swap Groq for a different model.

• If scraping gets blocked often, consider rotating a residential proxy into local_scraper.py's browser context — not implemented here to keep things free.