Senior Data Engineer

To design, build, and maintain scalable data infrastructure and pipelines that enable reliable, efficient, and secure data flow for analytics, business intelligence, and AI/ML initiatives. Reporting directly to the Data Analytics & AI Manager, the role ensures that data from ERP (Oracle), production systems, supply chain, and other sources is ingested, transformed, and made available for data-driven decision-making across FCG's frozen food manufacturing and export operations.RESPONSIBILITIESDesign, develop, and maintain robust ETL/ELT pipelines to ingest data from multiple sources (Oracle ERP, production systems, WMS, IoT sensors, external APIs).Build batch and real-time data pipelines using orchestration tools (Airflow, Prefect, or similar) to support analytics and AI/ML workloads.Ensure data quality, consistency, and reliability throughout the pipeline lifecycle.Design and manage data warehouse and data lake solutions (Snowflake, BigQuery, Redshift, or Azure Synapse) optimized for analytics and AI consumption.Implement dimensional modeling (Kimball/Inmon) and data vault methodologies to support BI dashboards and AI feature stores.Optimize storage, partitioning, and query performance for large-scale datasets used by data analysts and data scientistsBuild and maintain feature stores and training datasets for machine learning models.Collaborate with data scientists to understand data requirements for predictive modeling, forecasting, and optimization algorithms.Implement data transformation logic for model training, validation, and inference pipelines.Integrate data from internal systems (Oracle ERP, WMS, HRIS) and external sources (supplier portals, logistics APIs, market data feeds).Build and maintain API-based data extraction and ingestion services for analytics consumption.Collaborate with application teams to ensure seamless data flow between operational systems and the analytics environment.Implement data quality checks, validation rules, and monitoring within pipelines to ensure accuracy for BI and AI use cases.Ensure data lineage, cataloging, and metadata management for full traceability.Collaborate with the Data Governance team to enforce data standards and security policies.Tune SQL queries, pipeline performance, and data processing jobs to meet SLA requirements for dashboards and AI model refreshes.Optimize resource utilization and cost efficiency for cloud data platforms.Troubleshoot data pipeline failures and ensure timely resolution with minimal impact on analytics users.Work closely with Data Analysts, BI Developers, and Data Scientists to understand data requirements for dashboards, reports, and models.Provide clean, well-documented, and timely datasets for analytics and AI/ML initiatives.Mentor junior data engineers and contribute to best practices within the Data & Analytics team.Maintain technical documentation for data pipelines, schemas, data dictionaries, and feature stores.Ensure compliance with data security, privacy, and retention policies.REQUIREMENTSBachelor's degree in Computer Science, Data Engineering, Information Systems, or related field.Master's degree is a plus.5–8 years of experience in data engineering, ETL/ELT development, or data warehousing.Experience in manufacturing, FMCG, or food processing industries is preferred.Proven experience supporting analytics, BI, and AI/ML initiatives is highly desirable.Experience with ERP data extraction is a strong advantage.Programming: Advanced Python (Pandas, PySpark) and SQL.ETL/Orchestration: Airflow, dbt, Prefect, or similar tools.Cloud Platforms: AWS (S3, Redshift, Glue), Azure (Data Factory, Synapse), or GCP (BigQuery, Dataflow).Data Warehousing: Snowflake, Big Query, Redshift, or Azure Synapse.Big Data Technologies: Spark, Kafka (preferred).Version Control: Git (GitHub/GitLab). o Containerization: Docker (preferred).Feature Stores: Experience with Feast, Tecton, or similar (preferred).Strong analytical and problem-solving abilities.Excellent communication and collaboration skills, especially with analytics and data science teams.Attention to detail and commitment to data quality.Ability to work independently and handle multiple priorities.