6sense: From Hackathons to AI Pipelines
A 4.5-year journey scaling frontend architectures, merging acquisitions, building big data pipelines, and shipping Generative AI infrastructure.
PySpark
Big Data Engineering: Optimizing PySpark Pipelines at 6sense
Owned critical big data infrastructure handling millions of signals daily. Optimized a core PySpark DAG saving 45 mins/day and retired legacy systems to save $130K annually.