Lead Data Engineer - AWS

Valleysoft | Center of Excellence is a regional IT services provider based in Egypt, serving clients globally since 2006. The company collaborates with global partners like Amazon Web Services (AWS) to offer comprehensive solutions that address diverse business and technical challenges. Valleysoft's commitment to excellence ensures that clients receive high-quality, innovative, and cost-effective services, leveraging the power of cloud technologies.Valleysoft is seeking a highly-skilled Lead Data Engineer specializing in AWS to join our team. In this role, you will lead the design and implementation of cutting-edge data solutions, unlocking the potential of big data and machine learning for our clients. You will drive data strategy, manage data architecture, and oversee a talented team of data engineers. If you are passionate about cloud-based data engineering and want to make an impact in a fast-paced environment, we want to hear from you!RequirementsLead design and implementation of batch and real-time data pipelines on AWSArchitect and implement ETL/ELT solutions using AWS Glue, Spark on EMR, and other AWS-native servicesLead workflow orchestration using Apache Airflow (MWAA)Architect serverless data processing solutions with AWS LambdaDesign enterprise-grade data warehousing solutions using Amazon RedshiftDevelop and enforce data storage strategies using Amazon S3 and DynamoDBImplement and govern data cataloging, governance, and access control using AWS DataZoneEstablish monitoring, logging, and observability using Amazon CloudWatchDrive performance optimization, scalability, and cost efficiencyTroubleshoot complex data processing and infrastructure issuesCollaborate with cross-functional teams including Analytics, BI, Data Science, and Business teamsProvide technical leadership, conduct code reviews, and mentor junior engineersEnsure adherence to security, compliance, and data governance standardsParticipate in architecture discussions and contribute to long-term data strategyRequired Skills & Experience10-12 years of experience in Data Engineering or related roles5+ years of hands-on experience designing enterprise-grade AWS solutionsExpertise with: AWS Glue, Amazon EMR (Spark), AWS Lambda, Apache Airflow (MWAA), Amazon EC2, Amazon CloudWatch, Amazon Redshift, Amazon DynamoDB, AWS DataZone, Amazon S3Strong programming skills in PythonAdvanced SQL skills with performance tuning expertiseStrong understanding of data warehousing, data modeling (Star/Snowflake), ETL/ELT best practicesExperience designing scalable, distributed data processing systemsProven experience in leading teams, mentoring engineers, and making architectural decisionsExperience working in Agile/Scrum delivery environmentsNice to Have.AWS Professional Certification (Solutions Architect / Data Analytics Specialty)Experience with real-time streaming (Kinesis, Kafka)Exposure to metadata management, data lineage, and governance toolsExpertise in cost optimization strategies in AWSExperience working with large-scale enterprise data platformsBenefitsPrivate Health InsuranceTraining & Development