Big Data Engineering: Optimizing PySpark Pipelines at 6sense
Owned critical big data infrastructure handling millions of signals daily. Optimized a core PySpark DAG saving 45 mins/day and retired legacy systems to save $130K annually.
Owned critical big data infrastructure handling millions of signals daily. Optimized a core PySpark DAG saving 45 mins/day and retired legacy systems to save $130K annually.