← Back to jobs

Member Of Technical Staff (Software Engineer, Enterprise Agent Evals & Efficiency)

perplexity

San Francisco
Development Systems Analyst Account Manager Public Relations

Job Score

100 pts
On-site model (+70) Development (+10) Systems Analyst (+10) Account Manager (+10)

About the Role

The Enterprise Agent Evals & Efficiency team makes Perplexity's agents better at completing real work across enterprise connectors, tools, and data. Our goal is to help agents choose the right tools, use the right context, and complete complex workflows reliably, with less time and compute. This team owns the improvement loop: define what successful work looks like, build evaluations that measure it, understand where agents fail, and ship changes that improve outcomes.

This is a hands-on engineering role with end-to-end ownership, from building realistic enterprise evaluation environments to shipping improvements across the agent harness, tools, and skills. You'll create complex scenarios spanning connected systems, data, and permissions, use them to identify where agents struggle, and improve the tools agents call, the skills they follow, the context they receive, and the models they use. Working with AI Research, Connectors, and Platform teams, you'll turn experimental results into measurable gains in task success, latency, and cost per successful task. The goal is not just to measure agents, but to make more enterprise work reliable and practical to automate.

Key Responsibilities

  • Build realistic enterprise evaluation environments spanning connected applications, representative datasets, user roles, and permissions, with complex, multi-step workflows and reproducible system state.

  • Own evaluation infrastructure, task suites, and automated graders that measure how well agents complete enterprise work, calibrated against human judgment and grounded in end-to-end outcomes.

  • Analyze agent trajectories and production failures, identify root causes, and ship improvements across the agent harness, tools, and skills that measurably improve task success and reliability.

  • Develop reusable skills and improve tool descriptions, schemas, and result handling so agents can select the right tools, execute workflows correctly, and recover from errors with fewer unnecessary calls.

  • Build and evaluate model-selection and routing strategies that match each task to the right model and effort level, balancing quality, latency, reliability, and cost.

  • Improve agent efficiency through context management, skill loading, caching, and execution strategies that reduce wasted tokens and cost per successful task without compromising quality or safety.

  • Build regression gates and launch-readiness signals for model, connector, tool, skill, and harness changes, and validate that evaluation gains translate into better production outcomes.

Qualifications

  • 5+ years of software engineering or machine learning engineering experience, with a track record of owning complex production systems from design through deployment and operation.

  • Strong system design skills, including building reliable, scalable infrastructure for experimentation, evaluation, or data-intensive applications.

  • Strong proficiency in a backend language such as Python, Go, or Rust, and the ability to work effectively across a multi-language production stack.

  • Hands-on experience building, evaluating, or improving LLM-based applications or agents, including tool use and multi-step workflows.

  • Deep expertise in at least one of the following:

    • Evaluation infrastructure, benchmark design, and automated grading.

    • Agent harnesses, tool interfaces, and reusable skills for complex workflows.

    • Realistic test environments or simulations spanning multiple systems, datasets, permissions, and stateful workflows.

    • Model selection, routing, and inference efficiency, including context management and caching.

  • Strong experimental and statistical judgment, including choosing representative tasks, defining meaningful success criteria, diagnosing failures, and distinguishing real improvements from noise.

  • Sound technical judgment in balancing task quality, latency, reliability, and cost, with a track record of translating evaluation findings into shipped improvements.

  • A track record of leading ambiguous, cross-team initiatives from architecture through production adoption, setting technical direction, and raising engineering quality through mentorship.

If you’re excited about this role, we encourage you to apply even if you don’t meet every qualification. We value demonstrated impact, deep ownership, and the initiative to turn ambiguity into results.

What did you think of this job?

Comments 0

Want to leave a comment?
Sign in or create your account in seconds to join the discussion.
Loading comments...

About Software Development

Software Development is one of the most dynamic and constantly evolving fields in the job market. Professionals in this area are responsible for creating, maintaining, and optimizing web, mobile, and desktop applications that impact millions of users daily.

Key languages and frameworks include JavaScript (React, Node.js, Vue.js), Python (Django, Flask), Java (Spring), PHP (Laravel), and TypeScript. Demand for full-stack developers continues to grow, especially in tech companies and startups.

Salaries range from entry-level to senior positions, with growing opportunities for remote work and international freelancing.

About Systems Analyst

The Systems Analyst is the professional responsible for analyzing, designing, and implementing technology solutions that meet business needs. They act as a bridge between business areas and the development team, ensuring that systems deliver real value to the organization.

Key skills include requirements gathering and analysis, process modeling (BPMN), data modeling, technical and functional documentation, system integration (APIs, microservices), and knowledge of ERPs and CRMs. Tools like Jira, Confluence, Visio, and project management platforms are essential.

Systems Analysts in technology companies are highly valued, especially those who master agile requirements analysis (user stories, backlog), system integration, and solution architecture. The field offers opportunities from junior analyst to solution architect, with a focus on efficiency, quality, and technological innovation.

About Account Manager

The Account Manager is the professional responsible for managing and expanding the relationship with clients after the sale. They act as a strategic partner, ensuring satisfaction, retention, and account growth, connecting client needs with company solutions.

Key skills include relationship management, negotiation, upsell and cross-sell, contract renewal, account planning, business reviews, metrics analysis (NPS, churn, LTV), and CRM knowledge (Salesforce, HubSpot). Communication, empathy, and business vision are fundamental differentiators.

Account Managers in technology and SaaS companies are highly valued, especially those who can increase recurring revenue (MRR/ARR) through account expansion and churn prevention. The field offers opportunities from account executive to director of accounts, with a focus on strategic relationship, revenue growth, and customer success.

About Public Relations

The Public Relations (PR) area focuses on managing the reputation, image, and communication of an organization with its various stakeholders (such as clients, investors, employees, media, and the community). PR professionals develop corporate communication strategies, manage media relations (press relations), organize institutional events, and work in image crisis prevention and management.

Discover Other Areas

Understand the scope of work, key skills, and tools used in different career areas.

About Fullstack

Fullstack developers are versatile professionals capable of working on both frontend and backend of web and mobile applications. They master multiple technologies and can build complete products end-to-end, from the user interface to server infrastructure.

Key skills include proficiency in at least one complete stack (React/Vue/Angular + Node.js/PHP/Python/Java), databases (SQL and NoSQL), REST/GraphQL APIs, Git versioning, CI/CD, and basic infrastructure knowledge (Docker, cloud). Clean architecture, DDD, and testing are important differentiators.

Fullstack developers are highly valued in startups and companies that need versatile and autonomous professionals. The field offers opportunities from junior developer to software architect, with a focus on complete delivery, holistic product vision, and ability to work across multiple application layers.

About Traffic Manager

The Traffic Manager is the professional responsible for planning, executing, and optimizing paid media campaigns across various digital platforms. With the competitiveness of the digital market, paid traffic professionals are essential for generating qualified leads and maximizing return on advertising investment.

Key skills include campaign management on Google Ads, Meta Ads, LinkedIn Ads, and TikTok Ads, media planning, metrics analysis (ROAS, CPA, CPC, CTR), A/B testing, remarketing, and landing page creation. Tools like Google Analytics, Google Tag Manager, Hotjar, and automation platforms are essential.

Traffic managers in technology companies are highly valued, especially those who master performance marketing, conversion funnel optimization, and scaling strategies. The field offers opportunities from media analyst to head of performance, with a focus on growth, budget efficiency, and return on investment.

About People Analyst

The People Analyst is the professional responsible for transforming people data into strategic insights for HR decision-making. They combine data analysis knowledge with people management vision to help organizations understand workforce metrics, turnover, engagement, and diversity.

Key skills include people analytics, workforce analytics, turnover and retention analysis, HR metrics (time-to-hire, cost-per-hire, e-NPS), data visualization (Power BI, Tableau, Visier), workforce planning, and compensation analysis. Knowledge of statistics, SQL, and people analytics tools is a differentiator.

People Analysts in technology companies are highly valued, especially those who can translate complex people data into actionable insights for retention, diversity, and growth strategies. The field offers opportunities from HR analyst to head of people analytics, with a focus on data-driven people management.

About Product Manager

The Product Manager (PM) is the professional responsible for defining the strategy, vision, and roadmap of a digital product. They work at the intersection of technology, business, and user experience (UX), leading the discovery and delivery of solutions that solve real problems in a viable way for the company.

Key skills include product discovery, data and metrics analysis (AARRR, NPS, LTV), user research, go-to-market strategy, roadmapping, strategic prioritization, and leadership by influence. Tools like Amplitude, Mixpanel, Hotjar, Jira, and Notion are fundamental.

Product Managers play a central role in the growth of startups, scale-ups, and large technology companies, with career progression opportunities to Product Leader, Head of Product, and Chief Product Officer (CPO).

About UX Design

The User Experience (UX) Design area focuses on optimizing the overall user experience when interacting with a product or service. UX professionals conduct user research (UX Research), map journeys, create wireframes, perform usability tests, and define navigation flows to ensure the product is intuitive, useful, and meets users' real needs.

Career Guides

Technology Career Guide

Planning, skills, interviews, and professional growth in IT, Data Science, DevOps, and Product.

Read full guide →

Design Career Guide

UX/UI, Graphic Design, Product Design. Portfolio, tools, interviews, and growth in the Design field.

Read full guide →

Marketing Career Guide

SEO, Paid Media, Growth, Content Marketing. Certifications, tools, and strategies to grow in Digital Marketing.

Read full guide →

Finance Career Guide

Financial market, investments, corporate finance, certifications, and strategies to grow in the financial field.

Read full guide →

Communication Career Guide

Journalism, PR, Corporate Communication, Content Marketing, and Multimedia Production.

Read full guide →

Administration Career Guide

Business Management, HR, Logistics, Consulting, Project Management, and Entrepreneurship.

Read full guide →

Data Career Guide

Data Science, Data Engineering, BI, Machine Learning, and AI. From training to the job market.

Read full guide →

Product Career Guide

Product Management, Product Ownership, Agile, Scrum, and OKRs. From strategy to execution.

Read full guide →

Tech & Remote Glossary

Stop getting lost in interviews and job descriptions

The job market, especially within tech and global companies, has developed its own dialect. Not understanding these acronyms can make you lose valuable opportunities or poorly negotiate your contract. To end this problem, we created the Definitive Glossary for the Remote Professional.

🏢 Work Models & Routine

Async (Asynchronous Work)
A communication model where responses don't need to be immediate. Instead of back-to-back meetings, the team relies on well-structured documents, threads, and messages. It's the gold standard for global companies spanning multiple time zones.
Sync (Synchronous Work)
The opposite of Async. It requires the team to be online and available at the same time for meetings, live chats, and real-time collaboration.
Daily / Stand-up
A quick daily meeting (usually 15 minutes) common in Agile (Scrum) methodologies. The team answers three questions: What did I do yesterday? What will I do today? Are there any blockers?
All-Hands / Town Hall
A company-wide meeting involving all employees. Usually led by the founders (C-Levels) to present results, new goals, and answer team questions.
1:1 (One-on-One)
A recurring individual meeting between a professional and their direct manager. It is used for career alignment, feedback, and problem-solving, not just for project status updates.

💰 Contracts, Benefits & Compensation

PTO (Paid Time Off)
Instead of strict, categorized leave policies, modern US companies usually offer a flexible pool of days (e.g., 20 days of PTO, or even "Unlimited PTO") that you can use for vacations, sick days, or personal matters, while receiving your regular compensation.
Equity / Stock Options
Company ownership. The startup offers you the right to buy shares at a heavily discounted strike price in the future. If the company grows, goes public, or is acquired, these shares can be highly lucrative.
Vesting (Vesting Schedule)
The rule that controls your Equity. It usually lasts 4 years. You don't get all the shares on day one; you "earn" them gradually as you stay with the company. A "1-year Cliff" means you must stay for at least one year to receive your first batch of shares.
RSUs (Restricted Stock Units)
Unlike Stock Options (where you have the right to buy the stock), RSUs are actual shares the company grants you as a bonus or part of your compensation package, following a strict Vesting schedule.
Independent Contractor (1099 / B2B)
The most common international hiring model for global talent working for US companies. You act as a service provider (business-to-business), receiving the gross salary (often six-figure compensation) without standard local payroll tax deductions at the source.

🤖 Recruitment & Hiring Process

ATS (Applicant Tracking System)
The "robot" that reads your resume. Software like Greenhouse, Ashby, and Workday are used to filter candidates by keywords before a human even looks at the document. (Pro tip: this is why your resume must be clean, semantic, and have the right keywords).
JD (Job Description)
The document that lists the responsibilities, technical requirements, and benefits of the open position.
Cultural Fit
The interview stage that evaluates if your core values, communication style, and worldview align with the company's culture. This is the ultimate test of your Soft Skills.
Onboarding
The integration process. It's the period of your first few weeks at the company, where you get your access credentials, learn about the culture, study the internal documentation, and understand how the product works.

🚀 How to use this to your advantage?

The secret isn't just knowing what these acronyms mean, but using them actively. If during an interview for a premium tech role you ask, "How does your PTO policy and Vesting schedule work?", the recruiter will immediately perceive you as a high-level professional, familiar with the global market standards.

The remote job market requires preparation. And having the right vocabulary is the first big step to securing six-figure proposals and standing out among thousands of applicants.

Expert Tip

Agile and Project Management in Today's Market

1. Move Beyond "Doing Agile": Deliver Real Value

One of the biggest mistakes candidates make in interviews for Agile roles is focusing too much on ceremonies (Dailies, Plannings, Retrospectives) and forgetting their actual purpose. Recruiters from global tech companies want to know how you eliminated bottlenecks and boosted team efficiency.

"True Agile success isn't measured by how many practices you implement, but by how much value you deliver to the customer and how quickly you do it."

Instead of writing "facilitated Scrum ceremonies" on your resume, use a results-driven approach: "Reduced team Cycle Time by 20% by identifying and removing bottlenecks in the value stream."

2. Certifications That Actually Bypass the ATS

While practical experience is the ultimate deciding factor, Applicant Tracking Systems (ATS) used by top US companies (like Greenhouse, Ashby, and Workday) filter resumes strictly through keywords. Holding the right certifications ensures you pass this first robotic barrier and get your profile in front of a human recruiter.

  • Professional Scrum Master (PSM I, II) by Scrum.org: Highly respected because it doesn't require renewal and features a rigorous, Scrum Guide-based exam.
  • Project Management Professional (PMP) by PMI: The gold standard for traditional, predictive, and hybrid Project Management.
  • PMI Agile Certified Practitioner (PMI-ACP): Excellent for demonstrating broad knowledge across multiple Agile frameworks (Scrum, Kanban, Lean).
  • SAFe (Scaled Agile Framework): Absolutely essential if you are targeting Enterprise-level corporations that have scaled Agile across thousands of employees.

3. Mastering Flow Metrics and Tools (Hard Skills)

The modern Agile professional must be data-driven. Companies offering six-figure compensation expect you to extract actionable insights from tools like Jira, Confluence, or Azure DevOps.

During technical interviews, be prepared to discuss how you utilize flow metrics, such as:

  • Lead Time and Cycle Time: To measure efficiency and the actual time it takes to deliver requests.
  • Throughput: The number of work items completed in a given timeframe.
  • Cumulative Flow Diagram (CFD): To visually identify bottlenecks in the workflow.
  • OKRs (Objectives and Key Results): To align the development team's daily work with the company's high-level strategic goals.

4. Soft Skills and Conflict Resolution in Async Teams

In global, remote-first, and asynchronous environments, communication is the biggest challenge. A top-tier Project Manager or Scrum Master acts as a shield for the tech team and a bridge for stakeholders.

Demonstrate emotional intelligence, empathy, and negotiation skills. You will often face situational questions like: "How would you handle a Product Owner who constantly changes the Sprint scope mid-cycle?" Your answer should highlight peaceful, data-driven negotiation focused on business impact and team predictability.

Conclusion: Your Competitive Edge

To land premium Project Management and Agile roles, you must position yourself as a strategic business partner, not just a process enforcer. Understand the product, master the metrics, and know how to guide people towards continuous improvement.

References & Useful Links