Overview
We are seeking an Cloud Storage Management Expert to perform human-in-the-loop testing and evaluation of agentic storage management experiences. This role supports the calibration and validation of AI-powered storage agents by providing expert, manual evaluation of user journeys that complement existing LLM-based evaluation pipelines. As a Storage Management Evaluation Specialist, you will manually evaluate AI agent responses across end-to-end storage management workflows using natural language interfaces. Your storage administration expertise will be essential for assessing the completeness, accuracy, and practical value of agent-generated recommendations from an industry practitioner's perspective.
This is a W-2 employment position with Cincinnatus LLC, with the opportunity to be placed at a leading technology company as part of their extended workforce.
Responsibilities
- Discovery Evaluation: Test User Journeys across migration options to calibrate agent response completeness and accuracy of presented options.
- Planning Evaluation: Assess agent guidance on trade-offs between Online and Offline transfer methods, including Cloud Interconnect ROI analysis and time estimates across varying file sizes and object counts.
- Validation Evaluation: Execute simulation and dry-run migration User Journeys; identify issues that could cause migration failures.
- Execution Evaluation: Initiate actual transfer operations via agent interfaces and assess operational reliability.
- Reporting Evaluation: Evaluate agent-generated progress tracking, outcome reporting, and variance analysis against migration plans.
- Traffic Simulation: Simulate realistic storage traffic in test projects to create representative evaluation conditions.
- Agent Invocation & Feedback: Invoke AI Agents with natural language queries and provide subjective, expert-level feedback on outcomes.
- Scoring Calibration: Calibrate offline scoring mechanisms using evaluation findings to improve automated assessment accuracy.
- 3P Agent Testing: Test third-party agent blueprints built for customers on the Cloud Storage MCP platform.
Deliverables
- Completed User Journey evaluation reports for each agent (per 10-week cycle)
- Documented subjective insights and actionable feedback on agent responses from a storage admin persona
- Calibration data and recommendations for the offline scoring mechanism
- Issue logs identifying agent failure modes, inaccuracies, or gaps
- Summary reports comparing agent-planned outcomes vs. actual migration results
Required Qualifications
- Storage administration experience with a strong understanding of cloud storage concepts, data migration strategies, and enterprise storage management practices
- Familiarity with cloud storage platforms (AWS S3, GCS, Azure)
- Hands-on experience with data migration — including online/offline transfer methods, Cloud Interconnect, and transfer sizing considerations
- Ability to evaluate AI/LLM-generated recommendations from a practitioner's perspective, identifying gaps in completeness and accuracy
- Strong written communication skills for documenting evaluation findings and providing structured feedback
- Comfortable working with natural language interfaces and conversational AI tools
Preferred Qualifications
- Background in storage performance analysis, security auditing, or storage intelligence/analytics
- Prior experience in QA, evaluation, or red-teaming of AI/ML systems
- Familiarity with MCP (Model Context Protocol) or agentic AI frameworks
About Cincinnatus LLC
Cincinnatus LLC is an enterprise staffing company that partners with leading technology companies to source and employ highly skilled professionals for contingent and contract-based opportunities. Cincinnatus serves as the employer of record for these engagements, providing W-2 employment, payroll, benefits, and compliance, while placing employees directly within client teams to work on high-impact initiatives. Roles hired through Cincinnatus are not project-based or freelance engagements. They are structured, role-based positions that typically involve part-time or full-time commitments, close collaboration with a client's internal teams, and integration into standard enterprise workflows.
Cincinnatus is a legal entity separate from Mercor. While opportunities may be discovered through Mercor's platform, employment, onboarding, payroll, and benefits for these roles are administered by Cincinnatus LLC.
Equal Employment Opportunity
Cincinnatus is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, reproductive health decisions, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, political views or activity, or any other legally protected characteristic.
Cincinnatus is committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans throughout the job application process.