No Commute

🌟 Accomplished Tech Visionary: Embark on an exciting journey into the realm of software development with 3Pillar! We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Senior Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries. If your passion for data analytics solutions that make a real-world impact, consider this your pass to the captivating world of Data Science and Engineering! 🔮🌐 \n Key Responsibilities Understanding the business requirements Writing code and implementing the proposed solutions Creating data pipelines, versioning and change management Manage the complexity inherent in versioned data pipelines Develop ETL/ELT processes to help extract and manipulate data from multiple sources. Design, build and maintain batch or real-time data pipelines in production. Automate data workflows such as data ingestion, aggregation, and ETL processing. Logging and instrumentation of pipelines and services. Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures. Train and mentor junior engineers. Minimum Qualifications Demonstrated expertise with a minimum of 5+ years of experience as data engineer or similar role Advanced SQL skills and experience with relational databases and database design. Experience working with cloud Data Warehouse solutions (e.g., Snowflake, Redshift, BigQuery, Azure Synapse, etc.). Strong Python skills with hands-on experience on Pandas, NumPy and other data related libraries Experience with Big Data technologies like Spark, Map Reduce, Hadoop, Hive etc Proficiencient in data pipeline and workflow management tools e.g. Airflow Experi