We are seeking an Cloud Storage Management Expert to perform human-in-the-loop testing and evaluation of agentic storage management experiences. This role supports the calibration and validation of AI-powered storage agents by providing expert, manual evaluation of user journeys that complement existing LLM-based evaluation pipelines. As a Storage Management Evaluation Specialist, you will manually evaluate AI agent responses across end-to-end storage management workflows using natural language interfaces. Your storage administration expertise will be essential for assessing the completeness, accuracy, and practical value of agent-generated recommendations from an industry practitioner’s perspective.

This is a W-2 employment position with Cincinnatus LLC, with the opportunity to be placed at a leading technology company as part of their extended workforce.

Responsibilities

• Discovery Evaluation: Test User Journeys across migration options to calibrate agent response completeness and accuracy of presented options.

• Planning Evaluation: Assess agent guidance on trade-offs between Online and Offline transfer methods, including Cloud Interconnect ROI analysis and time estimates across varying file sizes and object counts.

• Validation Evaluation: Execute simulation and dry-run migration User Journeys; identify issues that could cause migration failures.

• Execution Evaluation: Initiate actual transfer operations via agent interfaces and assess operational reliability.

• Reporting Evaluation: Evaluate agent-generated progress tracking, outcome reporting, and variance analysis against migration plans.

• Traffic Simulation: Simulate realistic storage traffic in test projects to create representative evaluation conditions.

• Agent Invocation & Feedback: Invoke AI Agents with natural language queries and provide subjective, expert-level feedback on outcomes.

• Scoring Calibration: Calibrate offline scoring mechanisms using evaluation findings to improve automated assessment accuracy.

• 3P Agent Testing: Test third-party agent blueprints built for customers on the Cloud Storage MCP platform.

Deliverables

• Completed User Journey evaluation reports for each agent (per 10-week cycle)

• Documented subjective insights and actionable feedback on agent responses from a storage admin persona

• Calibration data and recommendations for the offline scoring mechanism

• Issue logs identifying agent failure modes, inaccuracies, or gaps

• Summary reports comparing agent-planned outcomes vs. actual migration results

Required Qualifications

• Storage administration experience with a strong understanding of cloud storage concepts, data migration strategies, and enterprise storage management practices

• Familiarity with cloud storage platforms (AWS S3, GCS, Azure)

• Hands-on experience with data migration — including online/offline transfer methods, Cloud Interconnect, and transfer sizing considerations

• Ability to evaluate AI/LLM-generated recommendations from a practitioner’s perspective, identifying gaps in completeness and accuracy

• Strong written communication skills for documenting evaluation findings and providing structured feedback

• Comfortable working with natural language interfaces and conversational AI tools

Preferred Qualifications

• Background in storage performance analysis, security auditing, or storage intelligence/analytics

• Prior experience in QA, evaluation, or red-teaming of AI/ML systems

• Familiarity with MCP (Model Context Protocol) or agentic AI frameworks