Data Platform Engineer Kozak Agency $$$
Madfish
Software Engineering
Europe
Industry: Energy / Commodity Trading
Location: Offshore / Remote
Start: ASAP / to be agreed
Workload: Full-time preferred
Language: English
The project focuses on building, maintaining, and optimizing scalable data pipelines, analytics tools, and automation solutions for high-impact commercial analytics across gas, LNG, power, and weather markets.
The client is expanding its analytics platform by integrating external data sources such as ENTSOE, EPEX, KPLER, MetDesk, WoodMac, and other market data providers. The goal is to structure and prepare data for advanced analytics, forecasting models, dashboards, and real-time or batch use cases in a dynamic trading environment.
Key Responsibilities
- Build and enhance scalable data pipelines and automation workflows on Azure
- Integrate external data sources via APIs, web scraping, and automation frameworks
- Develop end-to-end automated data solutions for gas, LNG, power, and weather markets
- Maintain and optimize existing databases, dashboards, toolkits, and API-based solutions
- Design and manage Snowflake data models with a focus on performance and scalability
- Support analytics teams by structuring data for models and advanced analytics use cases
- Enable reliable backend infrastructure for dashboards and GUI tools
- Consult and support real-time and batch analytics use cases in a trading environment
- Improve data quality, performance, and automation across the analytics platform
Must-Have Skills
- Strong hands-on experience with Azure Data Factory
- Strong experience with Databricks
- Experience with Azure Data Lake
- Strong knowledge of Snowflake, including data modeling and performance tuning
- Proficiency in Python
- Proficiency in SQL
- Experience with API integration
- Experience with automation workflows
- Experience building scalable data pipelines for large-scale data ingestion and processing
Nice-to-Have Skills
- Experience with web scraping
- Knowledge of real-time / event-driven data processing
- Experience in energy, commodity trading, gas, LNG, power, or weather analytics
- Experience working with external market data providers such as ENTSOE, EPEX, KPLER, MetDesk, or WoodMac
- Experience supporting analytics dashboards or GUI tools