Data Scientist

Ellipsis Health is creating cutting-edge AI/ML products that solve healthcare staffing issues and administrative burdens using conversational AI and our patented voice biomarker technology in the delivery of better healthcare for everyone. We are
headquartered in Silicon Valley and are funded and supported by some of the most preeminent venture capital teams.

As a Data Scientist specializing in AI Quality & Evaluation, you will play a pivotal role in ensuring the robustness, accuracy, and fairness of our AI models and systems. You will be instrumental in developing comprehensive evaluation frameworks, creating diverse and high-quality datasets, and implementing metrics to drive continuous improvement across our AI initiatives. Your work will directly contribute to accelerating customer onboarding and enhancing the overall user experience.

Ellipsis Health is located in the San Francisco Bay Area, but we are open to remote candidates for this role, as long as they are located in the U.S.

Responsibilities

• Create and Manage Datasets: Design and build high-quality datasets for training, testing and evaluating conversational AI system components.
• Develop Evaluation Frameworks: Engineer and implement advanced evaluation methodologies and metrics to accelerate customer onboarding. You’ll also establish robust evaluation metrics for our AI systems.
• Analyze and Improve AI Quality: Investigate the intricate relationship between LLM quality and the performance of LLM Judge systems to drive continuous improvement.
• Collaborate and Innovate: Work closely with AI/ML Engineers and Product Managers to integrate your findings into product enhancements. You’ll stay current with the latest advancements in AI evaluation, particularly in Large Language Models (LLMs) and speech systems, contributing to our overall technical strategy.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Science, Statistics, Mathematics, or a related quantitative field.
• Proficiency in programming languages such as Python (with libraries like Pandas, NumPy, Scikit-learn, TensorFlow, PyTorch).
• Solid understanding of statistical analysis, experimental design, and A/B testing methodologies.
• Strong problem-solving skills and a proactive approach to identifying and addressing data quality and model performance issues.
• Excellent communication and interpersonal skills, with the ability to collaborate effectively in a fast-paced environment.

Nice to Have

• Experience with Natural Language Processing (NLP) techniques and understanding of LLM architectures and their evaluation challenges.
• Experience with Speech Technologies and their evaluation is a significant advantage.
• Experience in a regulated industry (e.g., healthcare, finance). Familiarity with data governance principles, particularly concerning PII/PHI.
• Experience with MLOps practices and deploying models to production.
• Familiarity with cloud platforms (AWS, Azure, GCP).

Salary and Benefits
We offer competitive salary and benefits, including 401k matching up to a certain percentage of your salary, health, vision, and dental insurance, and very flexible paid time off. The typical salary range for this role is $60,000 to $120,000 USD. The amount offered will be determined by a variety of factors including but not necessarily limited to your individual skills, qualifications, and past experience relative to the requirements of the role.

Background Checks
As a health technology company, we reserve the right to run a background check on any applicant to which we extend an offer and to re-perform any such check at any time during the course of employment. Please know that there is no set policy on rejecting candidates because of certain background check results, and that we look at a candidate as a whole before making any decisions. We comply with all “ban the box” laws in applicable jurisdictions.

Assistance
If you have a disability or otherwise require any assistance whatsoever in the application or recruitment process, please feel free to submit a request to careers@ellipsishealth.com.

Apply Now | Please email resume to careers@ellip