Web QA Automation Analyst, Contract

BIMM is booming, so we’re looking for an Intermediate Web QA Automation Analyst (on contract until December 24, 2026) to join our team! 

Local candidates (GTHA) will be asked to join us at BIMM HQ (Distillery District) 2 days per week. We are open to remote candidates across Canada - you will be required to work on EST hours.  

Who we are 

BIMM is a Toronto-based agency that lives at the intersection of data, technology, and creativity to provide clients with high-performing campaigns and digital products that shape connected customer experiences. We’ve helped our clients implement fully integrated CRM customer journeys, built robust digital experiences, and created national loyalty programs with decades of experience in sectors of automotive, financial, telco and retail. 

We are creative realists, which means our ideas and solutions are grounded in the realities of our clients’ business. We believe big ideas are nothing without big results (and flawless execution); high-performing agencies deliver high-performing campaigns, products, and experiences; and the best solutions happen when we collaborate with clients as partners. 

Our Technology department is a 50+ person distributed team of Developers, QA, and Architects delivering amazing digital products on a leading-edge technology stack. 

This is an existing vacancy, not a made-up role — this one’s already got a desk waiting. 

Who we’re looking for 

Reporting to the QA Team Lead, the QA Automation Analyst will develop, execute, and maintain test plans, scripts, and processes for projects. As the sole QA in your agile pod, you’ll lead the QA stream, champion quality, and ensure systems are robust. If you’re an engineer at heart with a passion for automation, you’ll thrive in this role. 

Technology Requirements 
To ensure a secure and compatible working environment, the following technology requirements must be met:

• Use a Mac or Windows device (Linux or other OS not supported) 
  • Windows 11 or macOS Sonoma/Sequoia or newer is required 
  • Windows 10 may be used up to October 2025  
• macOS Big Sur, Monterey, and Ventura are not permitted 
• Device must be less than 4 years old 
• Agree to have us install and maintain BIMM’s security tools on your device. These tools must remain active, with all permissions and configurations unchanged for the duration of the contract 

Responsibilities 

• Website and Mobile Testing: Ability to validate content for accuracy and formatting, functionality of internal and external links, proper display and loading of images, and the integration of third-party tools in different environments. 
• Adhering to Tight Deadlines: Managing time effectively to meet tight project deadlines without compromising quality. 
• Reporting and Tracking Defects: Documenting and reporting defects in a clear and concise manner using bug tracking systems (like Jira or Xray).   
• Analyzing Test Results: Identifying patterns and trends in test results to improve the testing process and prevent future defects. 
• Working with Developers: Collaborating with developers to understand software functionality and identify potential areas for testing.   
• Manual Testing: Execute functional, integration, accessibility and end-to-end manual tests to validate features and uncover defects. 
• Automation Testing: Design, develop, and maintain automated test scripts in Cypress using Javascript to increase efficiency and coverage. 
• Communicating with Stakeholders: Reporting on test progress, results, and potential risks to project stakeholders and clients. 
• Staying Up to Date: Keeping abreast of the latest testing tools, technologies, and best practices. 
• Improving Test Coverage: Continuously expanding the scope of automated tests to cover new features and functionalities. 
• Optimizing Test Performance: Refining automated tests to improve their efficiency and speed. 
• AI-assisted test generation — use LLM-based tools (e.g. Copilot, Cursor, or purpose-built QA tools) to accelerate the authoring of Cypress test scripts, reducing manual scripting time without sacrificing quality 
• Prompt-driven test case design — leverage AI tools to generate edge case scenarios, boundary conditions, and negative test paths from requirements or user stories 
• LLM output validation — where the product under test includes AI-generated content or agentic workflows, design test strategies that evaluate output correctness, consistency, and regression across prompt/model changes 
• Exploratory testing with AI assist — use AI tools to identify untested surface areas, suggest test paths, and flag anomalies in test result patterns 

Requirements 

• 3+ years of proven manual testing experience of web applications 
• Experience with designing and writing automated tests using Javascript and testing frameworks (like Cypress, or Selenium). 
• Working experience using defect tracking systems such as JIRA/Xray 
• Passion for product quality and a positive user experience 
• Familiarity with QA methodologies, best practices, and software development life cycle 
• Ability to learn quickly and manage priorities in a fast-paced development environment 
• Highly driven and proactive with demonstrated creative and critical thinking capabilities, while staying ahead on QA technologies and methodologies 
• Good oral and written communication skills, specifically writing detailed tickets and communicating QA findings to both technical and non-technical audiences 
• Excellent analytical, debugging, and problem-solving skills 
• Be available to collaborate with the team in our Toronto office. 
• Experience with Accessibility Testing 
• Hands-on experience using AI-assisted tools to accelerate test authoring or QA workflows 
• Ability to test products that include LLM or agentic components — understands what "correct" output means in a non-deterministic system and can design repeatable validation strategies around it 

Nice to have 

• Experience with BrowserStack, Axe, PostMan, GraphQL or Firebase. 
• Developing Test Frameworks: Building and maintaining the infrastructure for automated testing, including libraries, tools, and guidelines for writing and running tests.   
• Selecting and Implementing Test Automation Tools: Evaluating and choosing the right tools for the project, considering factors like programming languages, application types, and testing environments. 
• Experience in an agency, consultancy, or startup environment 
• Experience with LLM evaluation tooling (LangSmith, PromptFoo, or custom eval pipelines) 
• Familiarity with golden dataset testing or LLM-as-judge approaches for validating AI feature output 
• Experience writing evals or regression suites for prompt changes 

What’s in it for you? 

• Culture: Our squad is fun, friendly and entirely egoless. Our social committee plans BIMM bashes, Tasty Thursdays and quarterly outings like candle making workshops, board game nights, Jays games and cooking classes. Our summer and holiday parties are quite memorable as well! Having Fun Everyday is one of our core values.  
• We believe in working & meeting with purpose. We observe a hybrid working model in our beautiful dog-friendly office located in the Distillery District, stuffed with snacks. 
• Belonging: Our dedicated DEI committee provides thought-provoking insights which are reflected in our work; They also organize engaging awareness activities and events to showcase BIMMER’s professional talents and personalities. What talents might you bring? :) 
• Growth: BIMM is part of the Kyu Collective which gives our employees access to the network’s resources, training, offices, and more. We also prioritize your personal and professional development with opportunities like on-site Lunch n’ Learns, conferences and online courses. Love 2 Learn. 

BIMM is committed to providing an environment that is inclusive and accessible. We are an equal opportunity employer and consider all applicants for employment without discrimination. Please let us know if accommodation for the recruitment/interview process is required and we will work with you to make sure your needs are met. 

AI In Recruitment 

While we are exploring ways to best utilize AI in our hiring processes, BIMM currently does not use AI-enabled tools in our hiring process. All candidates are screened by humans, and approved by humans.