Hive on Shubham | Full Stack & AI Engineerhttps://w3shubh.com/tags/hive/Recent content in Hive on Shubham | Full Stack & AI EngineerHugoenSun, 01 Oct 2023 00:00:00 +0530Big Data Engineering: Optimizing PySpark Pipelines at 6sensehttps://w3shubh.com/posts/big-data-pipelines/Sun, 01 Oct 2023 00:00:00 +0530https://w3shubh.com/posts/big-data-pipelines/Owned critical big data infrastructure handling millions of signals daily. Optimized a core PySpark DAG saving 45 mins/day and retired legacy systems to save $130K annually.