Senior Data Engineer responsible for designing and validating data solutions, consolidating disparate data sources, documenting functional specifications, reviewing data models, supporting production deployments, investigating data quality issues, and collaborating with stakeholders and technology leads to ensure data completeness and lineage.
Responsibilities
- Big Data - Spark, Hive, Java, CDP, 6 Years + with 3 Years of Dev experience on Big Data
- Analyze data requirements and identify disparate data sources required for consolidation and distribution.
- Document functional specifications and coordinate delivery of the same with technology team.
- Review logical and conceptual data models in alignment with business requirements.
- Work with the stakeholders to understand and gather requirements and produce business specifications
- Validate solution implementations and ensure they meet business and functional requirements.
- Provide production deployment support and investigate data quality issues.
- Work with various technology leads to ensure the gaps in the data completeness or accuracy are bridged.
Qualifications
- Subject matter expertise in financial industry - wholesale loans /lending business OR Capital Markets or Finance or Risk Reporting
- Strong hands on experience with database and SQL is required.
- Excellent documentation, analytical skills to produce process flow diagrams, business modelling, and functional design.
- Proficiency in MS Office (Word, Excel, Visio, PowerPoint) with extensive experience using Excel for data analysis.
- Experience with Data tracing/ Lineage efforts
- Knowledge of logical and physical data model
Similar Jobs
Agency • Information Technology
Senior Java Big Data Developer with 8+ years building applications on Cloudera/Hadoop ecosystems using Spark and MapReduce. Responsible for designing performant Java (1.8+) solutions, data analysis of large datasets, unit testing (JUnit), and collaborating with/managing global technology teams. Preferred skills include Unix/Python scripting and familiarity with Hive, Impala, YARN, and Kafka.
Top Skills:
SparkClouderaHadoopHiveImpalaJava 1.8JunitKafkaMapreduceNoSQLPythonUnix ShellYarn
Agency • Information Technology
Design, develop, test, and deploy high-performance Apache Spark/Scala data processing applications and ETL pipelines on Cloudera (CDH). Optimize Spark and platform performance, build data pipelines using HDFS, Hive, Impala, HBase, and Kafka, ensure data integrity and security, collaborate with data scientists and analysts, and implement version control and CI/CD for deployments.
Top Skills:
SparkCi/CdCloudera (Cdh)FlumeGitGitlabHbaseHdfsHiveImpalaJenkinsKafkaNifiOoziePostgresScalaSpark SqlSqoop
Agency • Information Technology
Design, develop, and maintain data engineering solutions across the SDLC using Python (primary) and Scala (secondary). Build distributed data pipelines on Hadoop/Spark ecosystems, extend data processing platforms (Spark SQL, Hive, Starburst), manage CI/CD, and deploy on Kubernetes/OpenShift. Collaborate with stakeholders in Agile teams and ensure maintainable, reusable code.
Top Skills:
AirflowAnsibleBitbucketHadoopHarnessHiveImpalaJenkinsJIRAJupyterhubKerberosKubernetesNumpyOpenshift4PandasPythonScalaSparkSpark SqlStarburstTektonUdeploy
What you need to know about the Montreal Tech Scene
With roots dating back to 1642, Montreal is often recognized for its French-inspired architecture and cobblestone streets lined with traditional shops and cafés. But what truly sets the city apart is how it blends its rich tradition with a modern edge, reflected in its evolving skyline and fast-growing tech industry. According to economic promotion agency Montréal International, the city ranks among the top in North America to invest in artificial intelligence, making it le spot idéal for job seekers who want the best of both worlds.
Key Facts About Montreal Tech
- Number of Tech Workers: 255,000+ (2024, Tourisme Montréal)
- Major Tech Employers: SAP, Google, Microsoft, Cisco
- Key Industries: Artificial intelligence, machine learning, cybersecurity, cloud computing, web development
- Funding Landscape: $1.47 billion in venture capital funding in 2024 (BetaKit)
- Notable Investors: CIBC Innovation Banking, BDC Capital, Investissement Québec, Fonds de solidarité FTQ
- Research Centers and Universities: McGill University, Université de Montréal, Concordia University, Mila Quebec, ÉTS Montréal
