AI Product Analyst – Voice AI Agents

About the role

MEDvidi is seeking an AI Product Analyst to evaluate, monitor, and improve Voice AI Agents that interact directly with patients. You will work at the intersection of product, engineering, operations, and AI to understand real‑world behavior, identify value, and drive enhancements.

Key responsibilities

• Analyze production behavior of Voice AI Agents and conversational AI workflows.
• Investigate hallucinations, inconsistencies, silent failures, and quality degradation in AI‑driven conversations.
• Work directly with call transcripts, logs, generated outputs, and operational datasets to uncover patterns and actionable insights.
• Design evaluation frameworks for measuring AI quality, reliability, and business impact.
• Build monitoring systems and dashboards for AI performance visibility.
• Measure the impact of AI systems on operational efficiency, business outcomes, and patient experience.
• Detect quality shifts, anomalies, and emerging failure modes in production environments.
• Partner closely with Product, Engineering, QA, and Operations teams on AI initiatives.
• Support rapid decision‑making in fast‑changing and ambiguous environments.

Required profile

• 3+ years of experience in product analytics, data analytics, operational analytics, or similar roles within complex production systems.
• Strong SQL proficiency.
• Python experience for analytical investigations and handling large‑scale text datasets.
• Understanding of large language model limitations and hallucinations.

Required skills

• SQL
• Python