This is a 5 day in office position in our San Francisco office. This role will start part-time with possibility of converting to a full-time permanent position.

About Us

Preference Model is building automated ML research engineering.

Existing frontier models are brittle when applied to real-world ML tasks. The present bottleneck is the lack of high-quality RL training environments. Our first step is to build RL environments that reflect real-world complexity, with diverse tasks and robust reward functions.

Our founding team has previous experience on Anthropic’s data team building data infrastructure, and datasets behind Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential.

About the Role

Preference Model is growing fast, and we want one person who owns our space and the in-person experience. You run the office day to day, give new hires a good first day, and organize the events that bring people in.

This is a hands-on job. The first thing you'll do is get us moved into our new office space. After that, you keep the office running and put on the events that make people want to be there.

This is a 5 day in office position in our San Francisco office. This role will start part-time with possibility of converting to a full-time permanent position.

What You Will Do

Run the office day to day:

• Coordinate office moves:

  • Movers, furniture, floor layout, and seating

  • Set up the space so it's ready on day one: desks, wifi, kitchen, supplies

  • Stand up the new conference rooms, including AV and room booking

• Keep the office stocked and running: snacks, coffee, supplies

• Greet guests and receive mail and packages

• Set up desks for new hires, keep the space tidy and clutter-free, clean out the fridge at the end of each day, and water the plants

• Coordinate with building management to obtain keys for new hires, and collect keys and office access when someone leaves

• Keep the first-aid kit (including an EpiPen), earthquake kit, and fire extinguishers stocked and current

• Ensure required compliance posters and the business certificate are posted correctly

• Ensure EpiPen and CPR training has taken place

Culture and events:

• Welcome every new hire in person on their first day: meet them outside, hand them their key and a swag package, and get them settled

• Run recurring team events: weekly board game nights and a monthly social outing (escape rooms and similar)

• Plan and run company offsites

New office launches:

• Travel to help set up new offices in other cities (for example Seattle or Toronto) for their initial launch

What We are Looking For

You are organized and outgoing. You take pride in running an office space and taking care of the people in it, and you stay on top of the small things other people don't notice.

Must have:

• Experience running an office or workplace at a startup or fast-growing company (office manager, workplace coordinator, or similar)

• A logistics brain: you can run a move, juggle deliveries and setup, and keep track of many moving pieces without dropping any

• Good with people: you excel at making people and new hires feel welcome

• Based in San Francisco and able to work onsite five days a week, with occasional travel to help open new offices

Any of these would make you stand out:

• You've managed an office move, buildout, and/or conference-room setup

• Event planning experience, from weekly socials to company offsites

• Familiarity with office and operations tools (Forkable, Brex, access/badging systems, or similar)

What We Offer:

• Competitive cash compensation

• Ownership and autonomy in a fast moving startup environment

• Lunch provided everyday onsite

• Weekly snack orders

About Us

Preference Model is building automated ML research engineering.

Existing frontier models are brittle when applied to real-world ML tasks. The present bottleneck is the lack of high-quality RL training environments. Our first step is to build RL environments that reflect real-world complexity, with diverse tasks and robust reward functions.

Our founding team has previous experience on Anthropic’s data team building data infrastructure, and datasets behind Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential.

About the Role

Models of the future will be able to train themselves on tasks that they are not good at. We are interested in investigating how far we can push the boundaries of self-directed learning. We are looking for machine learning Research Engineers or Research Scientists to push the frontier of post-training on large language models in a role that blends research and engineering, requiring you to implement novel approaches and shape research directions.

What You Will Do:

• Train and evaluate models on our proprietary RL environments to validate data quality, surface gaps in task coverage, and close the feedback loop between environment design and model capability.

• Architect and optimize our RL training infrastructure, from training abstractions to distributed experiment management, using frameworks like Verl, OpenRLHF, or similar. Help scale our systems to handle increasingly complex research workflows.

• Design, implement, and test training environments, evaluations, and methodologies for RL agents.

• Profile and optimize training runs end-to-end, from data loading through reward computation, to maximize experiment throughput and shorten the research iteration cycle.

What We are Looking For

• Experience running end-to-end LLM post-training pipelines of models sizes at least 7B in size

• Proficiency in Python and PyTorch or JAX

• Experience with at least one modern RL training framework

• Experience building and operating ML infrastructure at scale

You may be a good fit if you also:

• Have experience evaluating model outputs and building reward or evaluation signals

• Stay current on post-training research and can translate papers into running code

• Have strong opinions (loosely held) about how to structure RL training code for reproducibility and fast iteration

• Can balance research exploration with engineering rigor

• Have strong systems design and communication skills

Candidates don't need a PhD or extensive publications. Some of the best researchers have no formal ML training and gained experience building industry products. We believe adaptability combined with exceptional communication and collaboration skills are the most important ingredients for successful startup research.

What We Offer:

• Competitive cash and equity compensation (>90th percentile)

• Ownership and autonomy in a fast moving startup environment

• Opportunity to work with top machine learning engineers

• Health, vision, dental, benefits

• 401K match

• Lunch provided everyday onsite

• Weekly snack orders

• Visa sponsorship & relocation support available

We value diverse perspectives and experiences. If you're excited about this role but don't check every box, we still encourage you to apply.

About Us

Preference Model is automating ML engineering and a critical component is models' abilities to develop software.

The way we build software is changing fast. Five years ago we wrote every line of code by hand. Today, we don't. What does our work look like five years from now? We are shaping this future.

Recent models work well on narrow tasks but are still brittle on real software work: large codebases with real conventions and technical debt, judgment-heavy design decisions, and multi-step problems. The bottleneck on fixing that is the supply of hard, high-fidelity scenarios that find where the best models still break. That is what we build.

Our founding team has previous experience on Anthropic’s data team building data infrastructure, and datasets behind Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential.

About the Role

You will work on frontier AI from day one. You will be finding the limits of the most capable models on earth before the public ever sees them and building the hard problems that push those limits further.

The work: hunt for the specific places the best coding models in the world still fail, then build the self-contained, rigorously graded scenarios that expose it. You own each one end to end. There is no permission to ask for and no queue to wait in. If you find a place a frontier model breaks, you build the thing that teaches it to do better.

Each problem is a fresh challenge that you own end to end: a realistic system, a genuinely difficult problem, and verification robust enough that a frontier model can't game it. It’s build-the-future work, and the people who do it develop something rare: a deep intuition for how frontier models behave that only a handful of engineers in the world have.

You will work closely with a small team of engineers and directly with our founders, with full ownership and autonomy over what you build. This is independent, high-ownership work with regular feedback.

What You Will Do

• Hunt for where frontier models break across software, and build the hard, high-fidelity scenarios that expose those failures and push the ceiling of what the best models can do.

• Own the hardest problems on the roadmap end to end: multi-step workflows, realistic stakeholder interactions, large codebases with real conventions and technical debt, and challenging system design.

• Build verification robust enough that a frontier model can't hack it, and tell genuine capability gaps apart from artifacts of your own setup.

• Direct coding agents heavily in your day-to-day work, evaluate their output critically, and recognize when they are failing in subtle ways.

• Build the tooling your own work depends on.

• Mentor newer engineers on the team as it grows.
  

What We are Looking For

• Deep software engineering experience across multiple domains, with genuine expertise in at least one specialty: infrastructure, distributed systems, performance, security, compilers, databases, or similar.

• Proficiency in Python.

• Extensive hands-on experience with coding agents (Claude Code, Cursor, Codex, or similar), including an intuition for where they cut corners and how to direct them well.

• Strong intuition for how models behave, even without prior ML or AI experience. You can anticipate where a model will take shortcuts and design around that.

• Comfort working independently on complex, ambiguous problems with minimal direction.

• Track record of owning work end-to-end in previous roles.

You may be a good fit if one of the following applies

• You have been a senior or staff software engineer at a company known for engineering rigor (e.g., a frontier lab, infrastructure startup, or systems-heavy team) and want to apply that experience to model training.

• You have deep specialty expertise in an area that current models struggle with (distributed systems, low-level performance, security, compilers) and can build the problems that expose those weaknesses.

• You get excited about building a new hard problem from scratch on a regular basis.

• You have been an early engineer at a previous startup, shipped independently, and want to do it again in AI.

• You have spent significant time building with coding agents, written about their failure modes, or contributed to agent evaluation work.

What We Offer:

• Competitive cash and equity compensation (>90th percentile)

• Ownership and autonomy in a fast moving startup environment

• Opportunity to work alongside senior and staff engineers from frontier labs and infrastructure companies, plus top ML engineers

• Health, vision, dental, benefits

• 401K match

• Lunch provided everyday onsite

• Weekly snack orders

• Visa sponsorship & relocation support available

We value diverse perspectives and experiences. If you're excited about this role but don't check every box, we still encourage you to apply.