Role Title: AI Agent Evaluator
Role Type: Contractor
**Location: U.S based **
Opportunity Snapshot
Pay: $25-45/hour
Time per task: ~1.5 hours per run (evidence and scoring included)
Project kickoff: Monday, October 12
Onboarding: Today (Fri), Saturday, or Sunday, so you’re ready to start Monday October 10th
You’ll need: An active personal popular AI account (Free, Go, or Plus; not Pro), an active personal and a smartphone that can screen record
Your tasks: Run everyday errands with two AI agents, approve key steps, screen record, and score the results
About the role
AI agents are starting to handle real-life errands: paying bills, booking trips, doing the groceries. We want to know how well they actually do it, from the point of view of a real user. That’s you.
You’ll test **two AI agents **side by side, using your own accounts on your phone, and tell us what worked, what didn’t, and how it felt.
What you’ll do
- Run real-life scenarios: bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms
- Approve or stop the agent at key steps
- Screen record every attempt
- Score each run on completion, quality, control issues, satisfaction, and time
- Share clear written feedback on the agent experience
Who we’re looking for
- US-based, with a Bachelor’s degree
- Someone who already uses AI in daily life
- 2-3 years of experience, preferably in engineering, business management, or office/operations
- Detail-oriented, great at following instructions and documenting each step
- Clear written communication
- Generalists welcome, domain experts preferred
- No coding or AI background needed
Why you’ll like it
- Get paid to use the AI tools you already use
- Work entirely from your phone, on your schedule
- Have a direct say in how next-generation AI agents get built and improved
How the selection process works
- Apply
- Complete the screening questions
- Interview: 15-30 minutes max
- Profile review: if everything looks good, we present you to the client
- Onboarding: our Talent Ops team will contact you to get you started