You will be part of a data engineering team responsible for designing, developing, and maintaining data pipelines and analytics platforms. The role involves working with GCP services, modernising legacy ETL systems, and ensuring high-performance, reliable data solutions.
- Gather business requirements and translate them into data engineering solutions
- Design, develop, and maintain ETL/ELT pipelines and workflows
- Support migration of legacy ETL systems (e.g., DataStage/JCL) to GCP-native services
- Build and optimise pipelines using BigQuery, Dataflow, Pub/Sub, and Airflow
- Integrate data from multiple sources (Oracle, Teradata, DB2, flat files, etc.)
- Perform performance tuning and debugging of ETL jobs
- Monitor and maintain pipelines using Cloud Logging and Monitoring
- Collaborate with cross-functional teams to understand dependencies and requirements
- Support production issues and ensure data reliability and quality
- Contribute to estimation, testing, and deployment activities
Technical Skills
Must-Have:
- 3+ years of experience in Data Engineering / ETL development
- Hands-on experience with GCP or any cloud platform (GCP preferred)
- Good understanding of:
- BigQuery
- Dataflow / ETL tools
- Airflow / scheduling tools
- Strong SQL and programming skills (Python / Java)
- Experience in data pipeline development and data warehousing
- Experience integrating multiple data sources
Good-to-Have:
- Experience with IBM DataStage or legacy ETL tools
- Knowledge of Terraform / Infrastructure as Code
- Familiarity with CI/CD (GitHub)
- Exposure to Power BI or reporting tools
- GCP certification (e.g., Professional Data Engineer)
- Agile/Scrum experience
Behavioural Skills
- Strong analytical and problem-solving skills
- Good communication and teamwork abilities
- Ability to work independently and take ownership
- Attention to detail and quality
Educational Qualifications
- BE/BTech in Computer Science, IT, or related discipline
- Experience in data engineering, ETL development, or data platforms