Design, develop, and maintain data engineering solutions across the SDLC using Python (primary) and Scala (secondary). Build distributed data pipelines on Hadoop/Spark ecosystems, extend data processing platforms (Spark SQL, Hive, Starburst), manage CI/CD, and deploy on Kubernetes/OpenShift. Collaborate with stakeholders in Agile teams and ensure maintainable, reusable code.
Qualifications:
- Strong communication skills
- Experience of Agile development and scrums
- Banking and securities domain knowledge would be an added advantage
Skills Required:
- Strong experience working across the entire SDLC lifecycle
- Programming experience in one or more application or systems languages, Python - Primary, Scala - Secondary (basic knowledge)
- Strong experience working with Python concepts and libraries such as Jupyterhub, Airflow, Pandas, NumPy etc.
- Good experience with classes based OOP and design patterns
- Distributed Systems Design Experience - including understanding of distributed systems concepts and principles
- Knowledge and understanding of Kerberos and authentication
- Hadoop Ecosystem of Tools (Spark, Hive, Impala, etc).
- Experience extending and implementing core functionality and libraries in data processing platforms (Spark / Spark SQL, Hive, Starburst, etc)
- Strong experience working with the CI/CD pipeline and tools like Jenkins , Harness/Tekton, Udeploy/Ansible,Bitbucket, Jira
- Strong experience in cloud platforms like Kubernetes, OpenShift4
- Ability to deal with multiple stakeholders and follow through on open issues.
- A commitment to writing understandable, maintainable, and reusable software.
- Willingness to learn new languages and methodologies.
- Experience working with business partners and engineers to gather, understand, and bridge definitions and requirements.
- An innate desire to deliver and a strong sense of accountability for your work.
Education:
- Bachelor’s degree/University degree or equivalent experience
Similar Jobs
Agency • Information Technology
Design, develop, test, and deploy high-performance Apache Spark/Scala data processing applications and ETL pipelines on Cloudera (CDH). Optimize Spark and platform performance, build data pipelines using HDFS, Hive, Impala, HBase, and Kafka, ensure data integrity and security, collaborate with data scientists and analysts, and implement version control and CI/CD for deployments.
Top Skills:
SparkCi/CdCloudera (Cdh)FlumeGitGitlabHbaseHdfsHiveImpalaJenkinsKafkaNifiOoziePostgresScalaSpark SqlSqoop
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
Lead and manage internal investigations, develop and implement compliance policies, advise on regulatory requirements, analyze operational risks, communicate findings to stakeholders, coach and lead teams, and support compliance program implementation and training to strengthen internal controls and ethical standards.
Artificial Intelligence • Professional Services • Business Intelligence • Consulting • Cybersecurity • Generative AI
The role involves developing, testing, and validating Generative AI agents and maintaining automated testing standards. Responsibilities include mentoring junior associates, analyzing complex issues, and applying governance controls in AI-driven solutions.
Top Skills:
AIAutomated TestingCi/CdData EngineeringLlmsMlPower Automate
What you need to know about the Montreal Tech Scene
With roots dating back to 1642, Montreal is often recognized for its French-inspired architecture and cobblestone streets lined with traditional shops and cafés. But what truly sets the city apart is how it blends its rich tradition with a modern edge, reflected in its evolving skyline and fast-growing tech industry. According to economic promotion agency Montréal International, the city ranks among the top in North America to invest in artificial intelligence, making it le spot idéal for job seekers who want the best of both worlds.
Key Facts About Montreal Tech
- Number of Tech Workers: 255,000+ (2024, Tourisme Montréal)
- Major Tech Employers: SAP, Google, Microsoft, Cisco
- Key Industries: Artificial intelligence, machine learning, cybersecurity, cloud computing, web development
- Funding Landscape: $1.47 billion in venture capital funding in 2024 (BetaKit)
- Notable Investors: CIBC Innovation Banking, BDC Capital, Investissement Québec, Fonds de solidarité FTQ
- Research Centers and Universities: McGill University, Université de Montréal, Concordia University, Mila Quebec, ÉTS Montréal
