← Back to jobs

Research Engineer / Scientist, Data & Evals

harvey

Híbrido San Francisco
Data

Job Score

90 pts
Hybrid model (+80) Data (+10)

Why Harvey

At Harvey, we’re transforming how legal and professional services operate. By combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise, we’re reshaping how critical knowledge work gets done for decades to come.

This is a rare chance to help build a generational company at a true inflection point. We have strong product-market fit and world-class investor support. We’re scaling fast and defining a new category in real time. The work is ambitious, the bar is high, and the opportunity for growth — personal, professional, and financial — is unmatched.

Our team moves fast, takes ownership, and is deeply committed to the mission — operating with intensity, staying close to our customers, and pushing each other for excellence. We live by three values: Decisiveness, Simplicity, and Job's Not Finished. We act quickly on clear judgment over perfect information, we believe simplicity is what scales, and we're never satisfied with where we are. If you want to do the best work of your career alongside people who share that drive, we'd love to build with you.

At Harvey, the future of professional services is being written today — and we’re just getting started.

Role Overview

In this role, you'll build the data and measurement systems that power our post-training, benchmarking, and agent capability efforts.

We recently post-trained our first model built for legal work, and we’re now building the team to scale up our post-training efforts. You’ll own synthetic data pipelines, evaluation design, the tasks and grading behind training environments, and the systems that turn feedback from domain experts into training signals and rewards. These are tasks where correctness often can’t be verified automatically, so defining and measuring quality is a core research challenge. You’ll join a small, high-bar team of hands-on engineers and researchers, where each person owns a large part of the work.

We’re looking for research engineers who can build reliable data and evaluation systems, question what a metric actually tells us, and work closely with researchers and legal experts to turn model failures into measurable improvements.

Learn more about our work at harvey.ai/research.

What You'll Do

  • Build synthetic data pipelines that generate, filter, and curate training and evaluation samples for complex legal reasoning and agentic workflows.

  • Design evaluations and verifiers that reflect real legal work, calibrate them against expert judgment, and investigate where they fail.

  • Build and maintain capability benchmarks like our Legal Agent Benchmark (LAB), along with product-focused evals that track how models perform on real customer workflows.

  • Work with legal domain experts to define quality metrics, rubrics, and other grading mechanisms for training and evaluation.

  • Own training and evaluation data quality, including provenance, coverage, and safeguards against contamination.

  • Close the loop from product usage and customer feedback to new evaluations and training data, so models improve where customers need them most.

  • Partner with researchers to investigate model failures, distinguish real gains from misleading metrics, and use findings to guide training and release decisions.

What You Have

  • Experience building data pipelines, ML evaluation systems, or experimentation tooling, with strong Python skills and ownership of production-quality code.

  • A rigorous approach to measurement, including statistical reasoning, experimental design, and understanding when a metric can mislead.

  • The ability to translate ambiguous goals or expert judgment into structured datasets, rubrics, and measurable criteria.

  • Experience with synthetic data, LLM evaluations, automated raters, or preference data is a plus. Strong data and measurement engineering experience close to ML model development is valuable even without prior LLM work.

We are hiring for this role across multiple levels.

Compensation

$230,000 - $450,000 USD

Depending on your location, an Applicant Privacy Notice may apply to you. You can find all of our Applicant Privacy Notices here.

#LI-DZ1

Harvey is an equal opportunity employer and does not discriminate on the basis of race, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition, or any other basis protected by law.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made by emailing accommodations@harvey.ai

What did you think of this job?

Comments 0

Want to leave a comment?
Sign in or create your account in seconds to join the discussion.
Loading comments...

About Data

The Data field has undergone a radical transformation with the rise of Generative AI. Data professionals are fundamental for evidence-based decision-making across all industries.

Key specializations include Data Engineering, Data Science, Business Intelligence, Machine Learning Engineering, and Analytics. Tools like SQL, Python, Spark, dbt, and cloud platforms (AWS, GCP, Azure) are essential.

The data market continues with high demand and salaries among the most competitive in the technology sector, with many remote work opportunities.

Discover Other Areas

Understand the scope of work, key skills, and tools used in different career areas.

About Public Relations

The Public Relations (PR) area focuses on managing the reputation, image, and communication of an organization with its various stakeholders (such as clients, investors, employees, media, and the community). PR professionals develop corporate communication strategies, manage media relations (press relations), organize institutional events, and work in image crisis prevention and management.

About Systems Analyst

The Systems Analyst is the professional responsible for analyzing, designing, and implementing technology solutions that meet business needs. They act as a bridge between business areas and the development team, ensuring that systems deliver real value to the organization.

Key skills include requirements gathering and analysis, process modeling (BPMN), data modeling, technical and functional documentation, system integration (APIs, microservices), and knowledge of ERPs and CRMs. Tools like Jira, Confluence, Visio, and project management platforms are essential.

Systems Analysts in technology companies are highly valued, especially those who master agile requirements analysis (user stories, backlog), system integration, and solution architecture. The field offers opportunities from junior analyst to solution architect, with a focus on efficiency, quality, and technological innovation.

About Administrative

The Administrative area is responsible for ensuring the efficient functioning of all organizational operations. Administrative professionals manage processes, human resources, procurement, and facility management.

Key skills include process management, Office 365, administrative ERPs, compliance, and people management. Knowledge of automation and AI tools is becoming increasingly relevant.

The digitization of administrative processes has created new opportunities for professionals who master technology and management.

About Web Master

The Web Master is the professional responsible for maintaining, securing, and ensuring the technical performance of websites and web applications. They manage servers, hosting infrastructure, uptime monitoring, and ensure everything runs fast and reliably.

Key skills include server management (Apache, Nginx), hosting (AWS, Google Cloud, Azure), CDN (Cloudflare), SSL, DNS, web security (WAF, firewall), performance (Core Web Vitals, cache, compression), and versioning (Git, CI/CD). Knowledge of Docker, WordPress, cPanel, and monitoring (Sentry, New Relic) is a differentiator.

Web Masters in technology companies are highly valued, especially those who master DevOps, SRE, and can guarantee uptime and performance at scale. The field offers opportunities from junior webmaster to SRE and infrastructure engineer, with a focus on reliability, security, and speed.

About Engineering

Software Engineering goes beyond traditional development, focusing on scalability, performance, and system architecture. Software engineers are responsible for designing infrastructures that support millions of simultaneous users.

Skills include microservices architecture, DevOps, cloud computing, application security, and performance optimization. Knowledge of containerization (Docker, Kubernetes) and CI/CD is increasingly required.

Senior software engineers are rare and highly compensated professionals, with opportunities at major global tech companies.

Career Guides

Technology Career Guide

Planning, skills, interviews, and professional growth in IT, Data Science, DevOps, and Product.

Read full guide →

Design Career Guide

UX/UI, Graphic Design, Product Design. Portfolio, tools, interviews, and growth in the Design field.

Read full guide →

Marketing Career Guide

SEO, Paid Media, Growth, Content Marketing. Certifications, tools, and strategies to grow in Digital Marketing.

Read full guide →

Finance Career Guide

Financial market, investments, corporate finance, certifications, and strategies to grow in the financial field.

Read full guide →

Communication Career Guide

Journalism, PR, Corporate Communication, Content Marketing, and Multimedia Production.

Read full guide →

Administration Career Guide

Business Management, HR, Logistics, Consulting, Project Management, and Entrepreneurship.

Read full guide →

Data Career Guide

Data Science, Data Engineering, BI, Machine Learning, and AI. From training to the job market.

Read full guide →

Product Career Guide

Product Management, Product Ownership, Agile, Scrum, and OKRs. From strategy to execution.

Read full guide →

Tech & Remote Glossary

Stop getting lost in interviews and job descriptions

The job market, especially within tech and global companies, has developed its own dialect. Not understanding these acronyms can make you lose valuable opportunities or poorly negotiate your contract. To end this problem, we created the Definitive Glossary for the Remote Professional.

🏢 Work Models & Routine

Async (Asynchronous Work)
A communication model where responses don't need to be immediate. Instead of back-to-back meetings, the team relies on well-structured documents, threads, and messages. It's the gold standard for global companies spanning multiple time zones.
Sync (Synchronous Work)
The opposite of Async. It requires the team to be online and available at the same time for meetings, live chats, and real-time collaboration.
Daily / Stand-up
A quick daily meeting (usually 15 minutes) common in Agile (Scrum) methodologies. The team answers three questions: What did I do yesterday? What will I do today? Are there any blockers?
All-Hands / Town Hall
A company-wide meeting involving all employees. Usually led by the founders (C-Levels) to present results, new goals, and answer team questions.
1:1 (One-on-One)
A recurring individual meeting between a professional and their direct manager. It is used for career alignment, feedback, and problem-solving, not just for project status updates.

💰 Contracts, Benefits & Compensation

PTO (Paid Time Off)
Instead of strict, categorized leave policies, modern US companies usually offer a flexible pool of days (e.g., 20 days of PTO, or even "Unlimited PTO") that you can use for vacations, sick days, or personal matters, while receiving your regular compensation.
Equity / Stock Options
Company ownership. The startup offers you the right to buy shares at a heavily discounted strike price in the future. If the company grows, goes public, or is acquired, these shares can be highly lucrative.
Vesting (Vesting Schedule)
The rule that controls your Equity. It usually lasts 4 years. You don't get all the shares on day one; you "earn" them gradually as you stay with the company. A "1-year Cliff" means you must stay for at least one year to receive your first batch of shares.
RSUs (Restricted Stock Units)
Unlike Stock Options (where you have the right to buy the stock), RSUs are actual shares the company grants you as a bonus or part of your compensation package, following a strict Vesting schedule.
Independent Contractor (1099 / B2B)
The most common international hiring model for global talent working for US companies. You act as a service provider (business-to-business), receiving the gross salary (often six-figure compensation) without standard local payroll tax deductions at the source.

🤖 Recruitment & Hiring Process

ATS (Applicant Tracking System)
The "robot" that reads your resume. Software like Greenhouse, Ashby, and Workday are used to filter candidates by keywords before a human even looks at the document. (Pro tip: this is why your resume must be clean, semantic, and have the right keywords).
JD (Job Description)
The document that lists the responsibilities, technical requirements, and benefits of the open position.
Cultural Fit
The interview stage that evaluates if your core values, communication style, and worldview align with the company's culture. This is the ultimate test of your Soft Skills.
Onboarding
The integration process. It's the period of your first few weeks at the company, where you get your access credentials, learn about the culture, study the internal documentation, and understand how the product works.

🚀 How to use this to your advantage?

The secret isn't just knowing what these acronyms mean, but using them actively. If during an interview for a premium tech role you ask, "How does your PTO policy and Vesting schedule work?", the recruiter will immediately perceive you as a high-level professional, familiar with the global market standards.

The remote job market requires preparation. And having the right vocabulary is the first big step to securing six-figure proposals and standing out among thousands of applicants.

Expert Tip

Agile and Project Management in Today's Market

1. Move Beyond "Doing Agile": Deliver Real Value

One of the biggest mistakes candidates make in interviews for Agile roles is focusing too much on ceremonies (Dailies, Plannings, Retrospectives) and forgetting their actual purpose. Recruiters from global tech companies want to know how you eliminated bottlenecks and boosted team efficiency.

"True Agile success isn't measured by how many practices you implement, but by how much value you deliver to the customer and how quickly you do it."

Instead of writing "facilitated Scrum ceremonies" on your resume, use a results-driven approach: "Reduced team Cycle Time by 20% by identifying and removing bottlenecks in the value stream."

2. Certifications That Actually Bypass the ATS

While practical experience is the ultimate deciding factor, Applicant Tracking Systems (ATS) used by top US companies (like Greenhouse, Ashby, and Workday) filter resumes strictly through keywords. Holding the right certifications ensures you pass this first robotic barrier and get your profile in front of a human recruiter.

  • Professional Scrum Master (PSM I, II) by Scrum.org: Highly respected because it doesn't require renewal and features a rigorous, Scrum Guide-based exam.
  • Project Management Professional (PMP) by PMI: The gold standard for traditional, predictive, and hybrid Project Management.
  • PMI Agile Certified Practitioner (PMI-ACP): Excellent for demonstrating broad knowledge across multiple Agile frameworks (Scrum, Kanban, Lean).
  • SAFe (Scaled Agile Framework): Absolutely essential if you are targeting Enterprise-level corporations that have scaled Agile across thousands of employees.

3. Mastering Flow Metrics and Tools (Hard Skills)

The modern Agile professional must be data-driven. Companies offering six-figure compensation expect you to extract actionable insights from tools like Jira, Confluence, or Azure DevOps.

During technical interviews, be prepared to discuss how you utilize flow metrics, such as:

  • Lead Time and Cycle Time: To measure efficiency and the actual time it takes to deliver requests.
  • Throughput: The number of work items completed in a given timeframe.
  • Cumulative Flow Diagram (CFD): To visually identify bottlenecks in the workflow.
  • OKRs (Objectives and Key Results): To align the development team's daily work with the company's high-level strategic goals.

4. Soft Skills and Conflict Resolution in Async Teams

In global, remote-first, and asynchronous environments, communication is the biggest challenge. A top-tier Project Manager or Scrum Master acts as a shield for the tech team and a bridge for stakeholders.

Demonstrate emotional intelligence, empathy, and negotiation skills. You will often face situational questions like: "How would you handle a Product Owner who constantly changes the Sprint scope mid-cycle?" Your answer should highlight peaceful, data-driven negotiation focused on business impact and team predictability.

Conclusion: Your Competitive Edge

To land premium Project Management and Agile roles, you must position yourself as a strategic business partner, not just a process enforcer. Understand the product, master the metrics, and know how to guide people towards continuous improvement.

References & Useful Links