Menlo Security Inc. Logo

Menlo Security Inc.

AI Security Engineer

Posted 3 Days Ago
Remote
Hiring Remotely in Greece
Mid level
Remote
Hiring Remotely in Greece
Mid level
Build and operate automated data collection, web-scraping, processing, labeling, curation, training, and evaluation pipelines for AI security models. Develop benchmark harnesses, integrate false-positive and false-negative feedback into retraining workflows, and improve model quality across prompt injection, Shadow AI, malicious macro, PII, and content classification systems. Collaborate with research and engineering teams to convert detection telemetry into measurable model improvements.
The summary above was generated by AI

Menlo Security is the leader in Browser Security for human and agentic workforces. Our mission is to enable humans and agents to connect, communicate, and collaborate securely, without compromise. The Menlo Browser Security Platform protects organizations from cyberattacks by stopping threats across the web, documents, and email before they reach the user. With Menlo Agent Runtime Security (MARS), that protection now extends to the AI agents working alongside every employee. Menlo Security is trusted by major global businesses, including Fortune 500 companies and government agencies, to protect their most valuable asset, their data, and is backed by top-tier investors.

We are looking for a Data & AI / ML Evaluation Engineer to own the data collection, mining, curation and continuous evaluation foundation for our various AI security products. High-quality models require high-quality datasets and ongoing enrichment and retraining. In this role, you will build automated data ingestion pipelines from surrogate web traffic and open sources, manage dataset curation, labeling workflows, model training and operate our benchmark and evaluation harnesses. This is a data and AI / ML software engineering role focused on building the data infrastructure that trains our proprietary Prompt Injection SLMs, Shadow-AI detection models, Malicious Macro classifiers, PII, Content Classification engines and more.

Outcomes & KPIs Key Outcome(s) Owned:

  • Automated, continuous generation and curation of training, evaluation, and false-positive (FP) datasets across web, file, and macro threat surfaces.

  • Standardized, reproducible evaluation benchmarks that guide model improvements before deployment.

  • Automated data pipeline uptime and ingestion throughput from web/file sources.

  • Size, diversity, and labeling accuracy of threat dataset repositories (PI, Shadow-AI, PII, Macros).

  • Mean time to ingest live false-positive/false-negative feedback into training evaluation loops.

  • Accuracy and benchmark coverage across historical vs. newly trained SLMs.

What You'll Do

  • Design and operate automated data collection, mining, and web-scraping pipelines to gather threat samples from surrogate traffic and web sources.

  • Build data processing, annotation, and dataset curation workflows, training and re-training various models, e.g. Prompt Injection (PI) SLMs, Shadow-AI detection models, Malicious Macro classifiers, and PII detection.

  • Build mechanisms to capture false-positive (FP) samples from live user/DDR flows and represent / integrate them into model retraining pipelines.

  • Develop automated benchmarking and evaluation harnesses to continuously test model detection quality, runtime performance, and accuracy against baseline models (e.g., Vertex AI).

  • Build automated pipelines for model training for various models.

  • Work closely with research and software engineering teams to turn raw detection telemetry into ongoing actionable model improvements.

Required:

  • 2–4 years of experience in software engineering, data engineering, or automated QA/evaluation roles.

  • Strong programming proficiency in Python. Adv: C++, SQL.

  • Experience in model training and various ML, AI algorithms.

  • Experience building data pipelines, web scrapers, or automated data processing systems.

  • Familiarity with data structures, storage formats, and database configurations.

  • Methodical approach to dataset labeling, validation, bug logging, and metric tracking.

Preferred / Nice to Have:

  • Hands-on experience with ML evaluation frameworks, dataset curation, or annotation tools.

  • Familiarity with text extraction, NLP preprocessing, or document parsing techniques.

  • Understanding of cybersecurity concepts (Prompt Injection, Malicious Macros, DLP/PII).

  • Basic experience with cloud storage, object stores, and basic containerization (Docker).

  • Experience with AWS, GCP, Terraform, Kubernetes, and Linux.

Follow us on LinkedIn!

Why Menlo?

At Menlo, we don't settle for the status quo — in our technology or our culture. How we think and act is just as important as what we build. Our culture is defined by five core mindsets: Proactive Leadership, Straight Talk, United Impact, Elevated Talent, and Customer-Compelled. We take ownership and drive outcomes without waiting to be told. We communicate directly and seek hard truths. We break down silos and win together. We hold a high bar for ourselves and the people around us. And we treat every customer interaction as mission-critical. If you're someone who sees it, owns it, solves it, and does it — you'll thrive here.

All qualified applicants will receive consideration for employment without regard to race, sex, color, religion, sexual orientation, gender identity, national origin, protected veteran status, or on the basis of disability.

TO ALL AGENCIES: Please, no phone calls or emails to any employee of Menlo Security outside of the Talent organization. Menlo Security’s policy is to only accept resumes from agencies via Ashby (ATS). Agencies must have a valid services agreement executed and must have been assigned by the Talent team to a specific requisition. Any resume submitted outside of this process will be deemed the sole property of Menlo Security. In the event a candidate submitted outside of this policy is hired, no fee or payment will be paid.

Similar Jobs

26 Days Ago
In-Office or Remote
Senior level
Senior level
Information Technology • Software
Own and operate the AI security runtime control plane across Azure, AWS, and GCP. Build inline AI enforcement, tune prompt-injection, data protection, tool-call, and agent access policies, and integrate security platforms with SIEM/SOC workflows. Lead L3 investigations, automated containment, cloud and Kubernetes security, DevSecOps controls, vendor evaluations, and closed-loop red-team remediation. The role requires automation, infrastructure-as-code security, multi-cloud identity management, incident response, and collaboration with technical leadership.
Top Skills: AksAWSAws BedrockAws IamAzureAzure Ai FoundryAzure DevopsCopilot StudioDatabricksEksEntra IdGCPGcp IamGcp Vertex AiGithub ActionsKqlKubernetesMcpPowershellPythonSIEMSoarTerraform
Yesterday
Remote or Hybrid
Senior level
Senior level
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Lead technical discovery, presentations, demonstrations, proofs of value, RFP responses, and security architecture discussions for CrowdStrike’s Falcon platform. Advise technical and executive stakeholders on endpoint, cloud, identity, exposure management, SIEM, SOAR, and AI security. Partner with sales, product, marketing, specialists, and customer success to qualify opportunities, develop technical wins, and strengthen customer relationships across Greece.
Top Skills: Active DirectoryAIAWSAzureBashCehCisspCloud SecurityCrowdstrike FalconCtemEdrEndpoint SecurityExposure ManagementForensicsGCPIdentity ProtectionIncident ResponseItdrLinuxmacOSMicrosoft Entra IdMicrosoft SentinelOktaOscpOsi ModelPingPowershellPythonQradarSansSIEMSoarSplunkThreat HuntingVdiWindowsXdrZero Trust
2 Days Ago
Remote
Senior level
Senior level
Big Data • Cloud • Fintech • Professional Services • Software
Analyze product performance, define metrics, evaluate A/B tests, and identify opportunities to improve conversion and user experience. Build scalable data models, dashboards, and self-service tools using SQL, dbt, Looker, and related platforms. Lead analytical projects, communicate actionable insights, and collaborate cross-functionally to influence product decisions and roadmap priorities.
Top Skills: DbtLookerPower BISQLTableau

What you need to know about the Montreal Tech Scene

With roots dating back to 1642, Montreal is often recognized for its French-inspired architecture and cobblestone streets lined with traditional shops and cafés. But what truly sets the city apart is how it blends its rich tradition with a modern edge, reflected in its evolving skyline and fast-growing tech industry. According to economic promotion agency Montréal International, the city ranks among the top in North America to invest in artificial intelligence, making it le spot idéal for job seekers who want the best of both worlds.

Key Facts About Montreal Tech

  • Number of Tech Workers: 255,000+ (2024, Tourisme Montréal)
  • Major Tech Employers: SAP, Google, Microsoft, Cisco
  • Key Industries: Artificial intelligence, machine learning, cybersecurity, cloud computing, web development
  • Funding Landscape: $1.47 billion in venture capital funding in 2024 (BetaKit)
  • Notable Investors: CIBC Innovation Banking, BDC Capital, Investissement Québec, Fonds de solidarité FTQ
  • Research Centers and Universities: McGill University, Université de Montréal, Concordia University, Mila Quebec, ÉTS Montréal

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account