Mid-Senior Data Engineer

ppWe are looking for a mid-experience Data Engineer to help modernize our data ingestion landscape. You will play a key role in migrating legacy ETL/ingestion processes to state-of-the-art cloud-native workloads on Microsoft Azure and Snowflake, enabling reliable, scalable, and observable data pipelines. You’ll work closely with data architects, analysts, platform teams, and business stakeholders to ensure high-quality data delivery across operational and analytical use cases. /p h3Responsibilities /h3 ul liDesign, build, and maintain robust ingestion pipelines (batch and near-real-time) from diverse sources such as databases, APIs, files, and event streams. /li liMigrate legacy ETL/ELT processes to modern Azure and Snowflake patterns (re-platform and re-factor). /li liImplement incremental loads, CDC patterns, schema evolution handling, and backfills/reprocessing strategies. /li liStandardize and automate ingestion workflows using reusable frameworks, templates, and best practices. /li liDevelop cloud-native ingestion solutions leveraging Azure services such as Azure Data Factory/Synapse Pipelines, Azure Databricks and/or Spark, Azure Storage/ADLS Gen2, and optional event-driven services like Event Hubs. /li liBuild ingestion and loading patterns into Snowflake using Snowflake stages, file formats, COPY INTO, Streams/Tasks, and data modelling foundations for raw to curated data layers with dbt. /li liBuild components to capture streaming data sources. /li liDevelop real-time transformation pipelines and ensure timely delivery to consumer services. /li liDevelop shared service components, Python libraries, and integration templates to accelerate delivery across Data Engineering and Application teams. /li liFollow guidance on integration best practices and ensure consistency across digital services. /li liImplement data validation and quality checks for completeness, freshness, duplicates, and schema drift. /li liEnsure pipelines are reliable and recoverable, incorporating idempotency, retries, re-runs, and alerting. /li liApply security best practices such as least privilege, secrets management, encryption, and secure connectivity. /li liObserve data governance by following naming standards, applying data retention, classification, and documentation. /li liCollaborate in agile delivery through code reviews, CI/CD, iterative release planning, and cross-team coordination. /li /ul h3Qualifications /h3 ul li4-6 years of hands-on data engineering experience, with a strong focus on data ingestion. /li liExperience building production pipelines using Azure Data Factory, Databricks, and Synapse. /li liSolid SQL skills and experience working with modern cloud data warehouses, ideally Snowflake. /li liProficiency in Python for data processing, automation, and pipeline utilities. /li liGood understanding of data lake/lakehouse concepts and ingestion patterns. /li liInfrastructure-as-Code exposure (Terraform) and CI/CD (Azure DevOps). /li liFamiliarity with orchestration frameworks such as Dagster. /li liAble to prototype quickly while adhering to Group standards and controls. /li liClear communication with business stakeholders and technical teams. /li liExperience with energy commodity trading is a real advantage. /li liAbility to manage and prioritize multiple tasks in a fast-paced, deadline-driven environment. /li liStrong problem-solving and troubleshooting skills. /li liSelf‑motivated, creative, highly organized, and proactive. /li liFluent in English (spoken and written). /li /ul pBy submitting your resume, you agree to the retention and use of your personal data by TSG for recruitment purposes, including sharing with our clients in the context of your application. /p /p #J-18808-Ljbffr