Data Engineer SR (Databricks | ADF | AI) | SR
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer SR (Databricks | ADF | AI) | SR based in Brazil.
This senior data engineering role sits at the heart of modern data-driven transformation initiatives, where you will design and build scalable data pipelines and architectures that enable advanced analytics and AI use cases.
You will work with large-scale cloud data ecosystems, integrating structured and unstructured data from multiple enterprise sources into robust data platforms.
The role has a strong focus on Databricks and Azure-based services, supporting the development of high-performance ETL/ELT pipelines and data products.
You will contribute directly to critical business solutions such as credit analysis, financial guarantees, and document intelligence using RAG and AI-driven approaches.
Working in a highly technical and collaborative environment, you will partner with architects and stakeholders to translate complex requirements into scalable solutions.
You will also play a key role in modernizing legacy reporting systems and enabling real-time, data-powered decision-making across the organization.
Accountabilities:
- Design, build, and maintain scalable data models and pipelines using Databricks, Azure Data Factory, and Azure Data Lake Storage (ADLS).
- Develop and optimize ETL/ELT processes to integrate structured and unstructured data from APIs, files, and enterprise systems.
- Work with SAP Datasphere and relational databases (SQL Server, Oracle) to support enterprise data integration and modeling.
- Structure and implement RAG solutions for contractual, financial, and document-based datasets supporting AI-driven analysis.
- Build data pipelines to process external API data (including CXL systems), transforming it into analytics-ready datasets.
- Develop and maintain datasets for financial domains such as judicial deposits, guarantees, and credit-related structures.
- Support the creation of data products such as Position Manager and reporting layers replacing legacy BI tools.
- Collaborate with architects and stakeholders to refine requirements and ensure scalable, high-quality data solutions.
- Create and expose APIs for data consumption and integration across systems.
- Apply DevOps practices to ensure reliable deployment, monitoring, and version control of data workflows.
- Strong hands-on experience with Databricks, Azure Data Factory, ADLS, and cloud-based data architectures.
- Solid knowledge of SQL (SQL Server, Oracle) and data modeling concepts.
- Proven experience in Python for data engineering and pipeline development.
- Experience with ETL/ELT tools and building scalable data integration solutions.
- Ability to analyze business requirements and support architects in solution design.
- Experience in API development and integration.
- Familiarity with DevOps practices applied to data engineering environments.
- Minimum 4 years of professional experience in IT, supported by a relevant degree or postgraduate qualification (minimum 360h).
- Strong communication skills and ability to collaborate with technical and non-technical stakeholders.
- Experience with SAP Datasphere.
- Knowledge of tools such as TIBCO, Knime, PowerCenter, or SAP PowerDesigner.
- Exposure to data cataloging, governance, or advanced data platform ecosystems.
- Competitive compensation package
- Health and dental insurance
- Flexible work arrangements (remote/hybrid depending on project)
- Meal and food allowance
- Access to cutting-edge AI and data technologies
- Continuous learning and upskilling programs
- Career development in global transformation projects
- Participation in innovative AI and data initiatives
- Life insurance and wellness benefits
- Corporate partnerships and employee support programs.
Requirements:
Differentials: