Lead Assessment Developer
Outsmart is a mission-driven, early-stage startup reimagining higher education for the age of AI by creating an entirely new collegiate experience that expands access and opportunity. Backed by $38M from top venture firms including Khosla Ventures, Forerunner, DST Global Partners, and Lightspeed, the company is led by former Duolingo executives and a founding team that also includes Emmy Award winners. With a strong culture centered around high-impact, detail-oriented team members, Outsmart is already seeing significant traction, generating 1–5 million weekly organic views on social media.
What You’ll Be Doing
You'll manage the full assessment development lifecycle at Outsmart — from item authoring through field testing, standard setting, scoring, and reporting. You'll work closely with psychometricians, faculty, product managers, and engineers to build rigorous, scalable assessments. As an AI-forward organization, you'll also help shape how AI tools are responsibly woven into our development and QA workflows.
Who You Are
Master's degree (Ph.D. preferred) in educational measurement, assessment , or a related field
3+ years managing operational, high-stakes assessment lifecycles end-to-end
Proven experience leading standard-setting studies and managing diverse SME panels
Experience managing external vendors for field tests and pilot studies
Working knowledge of validity, reliability, and fairness standards (e.g., AERA/APA/NCME)
Familiarity with cognitive complexity frameworks (DOK, Bloom's)
Strong communicator who can translate psychometric concepts for non-technical audiences
Bonus: knowledge of CTT, IRT, CDM, DIF,, AI-assisted item generation, or accreditation reviews
What You’ll Own
End-to-end assessment development operations for assigned assessments
SME network — recruitment, training, calibration, and quality
Standard-setting studies — method selection, panel facilitation, and final reporting
Vendor and external partner relationships, SLAs, and security protocols
Field testing and pilot operations
Scoring operations for constructed-response and performance-based assessments
Project timelines, risk identification, and stakeholder communication
Core assessment documentation — specs, blueprints, training materials, technical reports
What Helps You Succeed
You thrive in ambiguity and are energized by building from scratch
You balance psychometric rigor with startup pragmatism — you know when to ship and iterate
You communicate with unusual clarity across technical and non-technical audiences
You take ownership without being asked and surface risks early
You're excited about AI's role in assessment and know where validation is non-negotiable
You care deeply about fairness and the learners behind every score
Salary Range: $120K-$180K
Perks
Competitive salary, equity, and benefits
Healthcare for you and your dependents (medical, dental, and vision)
Parental leave for any full-time employee (birthing, non-birthing, and adoptive parents)
15 PTO days per year, a winter break, and 12+ paid company holidays per year
Remote work days
Lunch provided in-office daily
Working with super talented people who want to make a difference in the world
Outsmart is committed to building an equitable and inclusive workplace and is proud to be an equal opportunity employer. We consider all qualified applicants for employment regardless of race, color, religion, gender, gender identity or expression, sexual orientation, national origin, disability, age, veteran status, or any other characteristic protected by applicable law.
Senior Applied AI Engineer
Outsmart is a mission-driven, early-stage startup reimagining higher education for the age of AI by creating an entirely new collegiate experience that expands access and opportunity. Backed by $38M from top venture firms including Khosla Ventures, Forerunner, DST Global Partners, and Lightspeed, the company is led by former Duolingo executives and a founding team that also includes Emmy Award winners. With a strong culture centered around high-impact, detail-oriented team members, Outsmart is already seeing significant traction, generating 1–5 million weekly organic views on social media.
What You’ll Be Doing
In this role, you’ll be at the forefront of the AI that powers Outsmart’s applications. As an early member of the engineering team you’ll play a critical role in setting the tone for Outsmart’s technical culture including setting standards for algorithmic quality, scalability, and AI-first development. You will work on both generative and discriminative use cases, focusing on content generation at scale, gamification, benchmarking and analytics, multi-modal behavior classification from mobile signals, and continuously modernizing Outsmart’s SDLC.
Who You Are
5+ years experience in hands-on AI/ML role
Deep and proven experience with agentic engineering
Experience shipping AI-first applications at scale
Experience using AI effectively to contribute cross-platform
Demonstrable sense of Product, QA, and DevX
Ability to work autonomously in a fast‑paced environment
What You’ll Own
Co-ownership of AI-first SDLC
MCP/ML tooling
Contribute to prompting standards across the engineering team
Define and enforce benchmarks for AI applications
Work with Product and Engineering leadership to define AI roadmap
Own end-to-end deployment and maintenance of AI systems
What Helps You Succeed
Craftsmanship and Extreme Ownership
Open-mindedness
Frequent, direct, honest, kind communication
Demonstrable adoption of AI in all aspects of work
Track record of shipping in an environment of high-performance
User obsession
Salary Range: $190K-$250K
Perks
Competitive salary, equity, and benefits
Healthcare for you and your dependents (medical, dental, and vision)
Parental leave for any full-time employee (birthing, non-birthing, and adoptive parents)
15 PTO days per year, a winter break, and 12+ paid company holidays per year
Remote work days
Lunch provided in-office daily
Working with super talented people who want to make a difference in the world
Outsmart is committed to building an equitable and inclusive workplace and is proud to be an equal opportunity employer. We consider all qualified applicants for employment regardless of race, color, religion, gender, gender identity or expression, sexual orientation, national origin, disability, age, veteran status, or any other characteristic protected by applicable law.
Faculty Lead & Learning Engineer - Sciences
The full posting opens here — pay, setting and the full description, without leaving the list.