The opportunity

We're a data collaboration company. Our enterprise clients in financial services and retail use our platform to gain insight from shared data without exposing individual identities. Data and analytics engineering sit at the core of what we deliver.

The role

We're looking for a Python/Spark/AI Developer to replatform legacy T-SQL logic onto Spark and Delta Lake pipelines and to build the APIs and AI features around them. You'll write type-safe, spec-first Python.

What you'll do

Pipelines: Build Spark and PySpark pipelines on Delta Lake, re-expressing T-SQL logic in Spark SQL.
Code quality: Write type-hinted, tested Python with pytest suites and CI lint and type-check gates.
APIs: Build REST APIs with FastAPI for pipeline jobs, including auth, idempotency and job-status semantics.
Parity: Validate migrated pipelines against the legacy system with parity evidence.
Specs: Work against design docs and ADRs, and document changes as you go.

What you bring

Python 3.12 as a primary language, with type-hinted code that passes strict static analysis (pyright or mypy), Pydantic models, and modern packaging (uv or Poetry)
Apache Spark and PySpark experience building production pipelines on open-source Spark, including the DataFrame API, Spark SQL, partitioning and performance tuning
Delta Lake or a comparable lakehouse format (Iceberg, Hudi), including MERGE INTO, schema evolution, time travel and idempotent writes
Strong, dialect-portable SQL, with the ability to read T-SQL and reproduce its semantics faithfully in set-based logic
Automated testing with pytest, including fixtures, markers and tiered unit, integration and end-to-end suites
REST API development with FastAPI or equivalent, including request validation and signed-request auth
Docker-based development against Compose stacks
Git and CI discipline: PR-driven work with ruff, type-check and test gates
4+ years of professional Python experience, with 2+ years building Spark or comparable distributed pipelines in production
Ability to work autonomously from written requirements and specifications
Bachelor's degree in Computer Science, Engineering or equivalent experience

Nice to have

LLM integration: provider-neutral AI features, structured output validation, and local or hosted inference
Data governance and privacy engineering: PII tokenisation, k-anonymity, GDPR-style data handling
Lakehouse platform components such as Hive Metastore, Trino or Apache Ranger
Azure data estate familiarity: ADLS Gen2, Synapse or ADF concepts, Key Vault
Observability tooling: structured logging, OpenTelemetry, OpenLineage
Migration or parity-testing experience
Spark certification, such as Databricks Certified Developer

Logistics

Location: Remote. Your working hours should overlap with Poland's time zone (CET) within ±2 hours.
Employment: Full-time
Compensation: Not disclosed
Benefits: Wellness programme, home-office reimbursement, continuous learning, and performance incentives
How to apply: Through TechTree. One application can also match you to other relevant roles.