Application Support - Data Engineer (+3 Years Experience)

Valleysoft is a leading IT services provider offering innovative solutions to clients worldwide. As a Data Engineer at Valleysoft, you will be responsible for designing, building, and managing scalable data pipelines and architectures that enable data-driven decision making. You will collaborate with data scientists, analysts, and business stakeholders to ensure the availability, reliability, and quality of data.Key responsibilities include:Develop, construct, test, and maintain data architectures such as databases and large-scale processing systemsEnsure data quality, integrity, and security throughout the data lifecycleDesign and implement ETL/ELT processes to gather and prepare data for analysisCollaborate with cross-functional teams to understand data needs and deliver solutionsMonitor and optimize the performance of data systems and pipelinesRequirementsProvide second and third-line application support for the enterprise data platform, covering data warehouses, data marts, ETL pipelines, and data integration layers across production and non-production environmentsMonitor, triage, and resolve incidents related to ETL job failures, data pipeline breaks, data quality anomalies, and performance degradation — ensuring SLA adherence and minimal business disruptionInvestigate root causes of recurring data issues, pipeline failures, and feed delays; implement permanent fixes and preventive measures through proper change control channelsPerform day-to-day operational tasks including job reruns, data corrections, parameter adjustments, and table refreshes in coordination with data owners and business stakeholdersCollaborate with data engineering, DBA, middleware, and infrastructure teams to resolve cross-system issues affecting data availability and pipeline integrityMaintain and update ETL code, mappings, workflows, and configuration files in response to source system changes, schema evolution, or business rule updatesSupport onboarding of new data feeds and sources by assisting in integration testing, environment validation, and go-live stabilizationMaintain accurate and up-to-date support documentation: runbooks, known error records, escalation procedures, and post-incident reportsParticipate in change advisory board (CAB) reviews for data platform changes, providing impact assessments and rollback plansProactively identify opportunities to improve platform reliability, automate repetitive operational tasks, and reduce manual intervention in pipeline managementTechnical RequirementsHands-on experience supporting ETL platforms such as Informatica PowerCenter, IBM DataStage, Talend, or equivalent enterprise integration toolsStrong SQL skills (Oracle, SQL Server, or DB2) — capable of writing diagnostic queries, tracing data lineage manually, and performing targeted data fixes under change controlSolid understanding of data warehouse architecture: ODS, staging layers, data marts, and EDW — sufficient to trace and isolate issues across pipeline stagesExperience with enterprise job scheduling tools such as Control-M, TWS, or Autosys — managing job dependencies, calendars, and failure alertsFamiliarity with Unix/Linux shell scripting for log analysis, file transfer monitoring, and lightweight automation of support tasksUnderstanding of database performance concepts: execution plans, index usage, partition pruning, and statistics — to diagnose slow-running jobs and queriesExposure to data quality monitoring frameworks and the ability to validate pipeline outputs against expected row counts, aggregates, and business rulesExperience with ITSM platforms (ServiceNow, Remedy, or equivalent) for incident logging, change requests, and problem management workflowsFamiliarity with cloud data platform components (Azure Data Factory, AWS Glue, Snowflake) is a plusBanking Sector Requirements (Preferred)Prior experience supporting data platforms in a bank or financial institution, with exposure to time-sensitive regulatory and financial reporting pipelinesFamiliarity with banking source systems feeding the data warehouse — core banking, GL, loan origination, cards, and payment systemsUnderstanding of data reconciliation and balancing requirements: ensuring ETL outputs align with source system totals and downstream report figuresExperience supporting pipelines that feed regulatory reports (CBE submissions, IFRS 9 staging, AML transaction monitoring) where data accuracy and timeliness are non-negotiableAwareness of data classification and access control requirements for sensitive banking data, including PII handling and audit trail obligationsExperience operating within ITIL-aligned support models with strict change control, incident prioritization, and escalation procedures applicable to regulated environmentsBenefitsPrivate Health InsuranceTraining & Development