I’m not a designer. I’m the data guy.

My main focus is an ingestion platform that can take heterogeneous regulatory documents and turn them into structured, traceable knowledge for retrieval and reasoning.

• Designed the end-to-end document ingestion pipeline that converts arbitrary source material into structured knowledge graphs with provenance for GraphRAG and auditable regulatory answers.
• Worked across the product stack — Django APIs, asynchronous Celery workflows, PostgreSQL/Supabase, Neo4j, frontend integration, authentication, and data contracts — to move ingestion from pipeline to user-facing capability.
• Built validation and QA stages around AI-assisted extraction so documents, entities, relationships, and citations remain inspectable instead of becoming an opaque embedding dump.
• Supported production infrastructure and delivery across Docker, Kubernetes, Redis, object storage, monitoring, and the interfaces used by internal teams and customers.

I worked directly with clients to translate broken coverage, missing fields, blocked traffic, and delivery problems into production fixes — then verified that the expected data actually arrived.

• Managed Scrapy Cloud projects end to end: deployments, scheduled jobs, concurrency, logs, stats, exceptions, item validation, crawl health, and feed exports to customer S3 buckets.
• Debugged and repaired client spiders under live delivery pressure, using logs and extracted data to separate access failures from parsing, pagination, schema, and pipeline defects.
• Partnered directly with customers on requirements, incidents, data quality, coverage, and delivery expectations while keeping crawler changes measurable and production-safe.

I built and optimized Scrapy and Zyte API pipelines at a scale where bans, session state, rendering cost, retry storms, and silent data loss were operating concerns, not edge cases.

• Bypassed IP and session bans, fingerprinting, JavaScript challenges, rate limits, and browser-only flows with proxy rotation, cookie and session management, browser rendering, adaptive retries, and request-level diagnostics.
• Operated crawling portfolios above 30 billion annual requests, including individual client workloads reaching 2 billion requests per month on distributed infrastructure.
• Balanced unblock rate, throughput, completeness, latency, and infrastructure cost while preventing successful HTTP responses with empty or malformed items from passing as good data.
• Scaled containerized crawlers with Docker and Kubernetes and maintained resilient output pipelines from extraction through validation and S3 delivery.

Promoted to lead development while remaining hands-on across the architecture and codebase.

• Owned project organization, system architecture, delivery scheduling, and hiring for the development function.
• Ran weekly alignment with the board and CEO, translating operational priorities into technical sequencing and accountable delivery.
• Kept architectural decisions close to implementation instead of turning coordination into status-report middleware.

Designed the internal platform that coordinated RPAs, integrations, and vehicle-data collection instead of relying on isolated scripts and paid orchestration tools.

• Architected a full-stack orchestrator with Express, Next.js, and PostgreSQL on RDS for REST/SOAP integrations and RPAs running on AWS Lambda and S3.
• Removed roughly 4,500 hours of manual work per month, improved operational efficiency by 40%, and reduced automation costs by 50% by replacing paid third-party orchestrators.
• Built vehicle-data extraction pipelines handling more than 500,000 daily requests to keep automotive records continuously current.

• Automated repetitive workflows with Selenium and Puppeteer, saving more than 500 work hours annually.
• Built BI dashboards in Power BI and GoodData plus NestJS and React reporting systems.
• Reduced processing time by 70% and gave operations faster, data-backed feedback on what was happening.