About Us
Visa is a world leader in payments technology, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories, dedicated to uplifting everyone, everywhere by being the best way to pay and be paid.
At Visa, you'll have the opportunity to create impact at scale — tackling meaningful challenges, growing your skills and seeing your contributions impact lives around the world.
Join Visa and do work that matters – to you, to your community, and to the world. Progress starts with you.
Job Description
What the Senior Data Engineer does:
The Senior Data Engineer can take a vision, e.g. “Improve the pipeline in Iceberg-based stack to enable point-in-time analysis”, and own it end-to-end: design, build, validate, ship, and operate. Give clear overarching objectives, you will make architectural calls under ambiguity and know exactly when a timeline is slipping and escalation is required. What makes this role different: you will be given the destination (business objective, best practice ideals, success criteria) and trusted to chart the route:
- Translate vision into systems: Take a strategic brief (e.g. “migrate this pipeline off legacy Hive to Iceberg with row-level parity across 12+ markets”) and decompose it into an executable architecture, milestones, and a credible delivery plan.
- Own data quality: Build reconciliation, parity, and validation frameworks. Catch the bug before the dashboard users do.
- Operate what you build: Monitor, alert, debug production incidents, write runbooks, do postmortems.
- Raise the bar: Code review, mentor, document, and push the team toward simpler, deeper abstractions.
- Leverage GenAI in your daily workflow across all of the above objectives: code generation and refactoring, log triage, doc summarization, agentic automations, and have an opinion on where it helps and where it doesn’t.
- Build production pipelines in PySpark / SparkSQL on Spark-on-K8s or YARN, orchestrated via Airflow (or equivalent), reading and writing across Hive, Iceberg, HDFS, and Trino / Starburst.
- Solve the hard problems independently — SCD2 logic, window-function correctness, late-arriving dimensions, partition design, backfills, OOMs, skew, query-engine quirks. Escalate when timeline, scope, or safety is at risk with clear asks.
Visa requires at least 3 days in office, expectations of these days will be confirmed by your Hiring Manager.
Qualifications
What you will need (Must-have technical skills):
Strong hands-on experience with PySpark, SparkSQL, Python, and advanced SQL (window functions, CTEs, analyze query plans to optimize pipeline behavior).
Production experience with big-data platforms: Spark, Hive, Trino / Starburst, HDFS, Iceberg. Cloud platform experience will be a valuable addition: AWS, Databricks, BigQuery, Snowflake.
Built and operated production pipelines on Airflow or a comparable orchestrator.
Hands-on experience with data quality frameworks, troubleshooting and bug fixing — validation checks, reconciliation against source of truth, drift detection, monitoring, alerting.
- Working knowledge of GenAI tools and concrete, production-relevant use cases in data engineering (e.g. LLM-assisted SQL generation, schema mapping, log triage, agentic workflows).
- Strong Git, testing (unit + integration), documentation, and production-support discipline. You write tests because you’ve been burned without them.
What will also help (Nice-to-have):
- Experience in banking, payments, financial services, or e-commerce at scale (billions of rows per day, multi-region).
- Exposure to Kubernetes, Spark-on-K8s, container-based job execution.
Familiarity with BI / semantic layers (Tableau, Looker, dbt, Cube) and how upstream data quality affects them.
- Experience leading a legacy-to-modern migration with parity gates and zero-downtime cutover.
Ideal candidate:
- Ownership over results. You work independently, with insatiable curiosity and a commitment to excellence. You don’t wait to be told.
- Knows when to escalate. You don’t silently burn the timeline. The moment scope, risk, or schedule is in jeopardy, you surface it with a concrete recommendation.
- Bias to ship. Tasteful over perfect. Iterate in production behind safe gates.
Visa is an EEO Employer
Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.