Data Engineer – GCP & BigQuery Specialist

About the role

As a Data Engineer you will design and implement modern, scalable data solutions on Google Cloud Platform. You will join NETCONOMY’s Data & AI team to help clients unlock the full value of their data through robust pipelines, warehouses, and real‑time processing.

Key responsibilities

• Build efficient ETL/ELT processes to ingest and transform data from databases, APIs and streaming sources into BigQuery and Cloud Storage.
• Implement batch and streaming workflows using Dataflow (Apache Beam) and Pub/Sub.
• Develop modular SQL transformation pipelines with Dataform, including version control, testing and scheduling.
• Create cost‑optimized BigQuery queries using partitioning, clustering and denormalisation techniques.
• Orchestrate complex workflows with Cloud Composer (Apache Airflow) and event‑driven processing via Cloud Functions.
• Set up centralized data governance and metadata management with Dataplex, ensuring automated cataloguing and lineage tracking.
• Monitor and optimise pipeline performance, scalability and cost using Cloud Monitoring and Cloud Logging.
• Collaborate with data scientists and analysts to translate requirements into actionable data solutions.

Required profile

• Minimum 3 years of hands‑on experience as a Data Engineer, preferably on GCP.
• Strong analytical mindset with the ability to design end‑to‑end data architectures.
• Proven track record of delivering production‑grade data pipelines.

Required skills

• Google Cloud Platform (GCP)
• BigQuery (SQL, partitioning, clustering, optimisation)
• Dataflow / Apache Beam
• Python (including PySpark and pandas)
• SQL
• Dataform
• Cloud Composer / Apache Airflow
• Cloud Functions
• Pub/Sub
• Cloud Storage
• Dataplex
• Cloud Monitoring and Cloud Logging