- ETL
- Scrape → normalize → Directus
- 7
- Analysis reports generated
- Quality
- Completeness & coverage profiling
Tech stack
- Python
- RTILA Studio
- Playwright
- pandas
- Directus
- Matplotlib
Problem
Building a usable dataset of local businesses from Google Maps means more than just scraping names — you need structured, deduplicated records with contact details, and you need to know how complete and trustworthy that data actually is before anyone relies on it.
Approach
I built a Python + RTILA scraper that harvests business listings (schools, companies) into structured records, running the extraction through a Playwright runtime and loading the results into a headless CMS (Directus). A dedicated data-analysis stage then profiles the harvested dataset — completeness, top categories, contact types, email validity, social coverage and contacts-per-organisation — and renders the results as charts.
Results
A clean, analysed dataset plus a set of quality reports that make the data's coverage and gaps explicit — completeness by field, category distribution, email-validity breakdown and social-profile coverage — so downstream users know exactly what they're working with.
Notable engineering
- Harvests Google Maps business listings (schools, companies) into structured records with a normalized schema.
- ETL into a headless CMS (Directus) with a Playwright runtime for the extraction stage.
- Data-analysis component profiles record completeness, top categories, contact types, email validity and social coverage.
- Produces publication-ready charts summarising the harvested dataset's quality.
Screenshots & diagrams
Real UI captures and figures — source is private and client brands are not shown.
Deep dive
This project pairs a scraping stage with an explicit data-quality stage, which is what separates a raw dump from a usable dataset. The scraper normalises each listing into a consistent schema and loads it into Directus; the analysis stage then answers the questions a consumer of the data actually cares about — how complete is each field, which categories dominate, how many records have a valid email, and how many have social profiles? Rendering those as charts turns "we scraped some businesses" into a defensible statement about coverage and quality.