- Support development of an AI/ML platform leveraging generative AI to relieve user burden across the Centers for Medicare and Medicaid Services (CMS)
- Design and maintain scalable automated test frameworks for web applications, APIs, and AI-driven features
- Develop automated pipelines for model validation, bias detection, and drift monitoring in ML-based systems
- Define and execute functional, regression, performance, and AI-specific test plans
- Partner with AI/ML engineers to create test datasets and simulate real-world user behavior and edge cases
- Implement CI/CD-integrated quality gates for AI release reliability and fairness metrics
- Mentor junior engineers and establish best practices in intelligent automation and AI testing
- Advocate for ethical AI standards and contribute to responsible AI discussions
Requirements
- 5+ years of experience in software QA/automation engineering, with at least 1 year focused on AI/ML product testing
- Strong proficiency in at least one automation framework (e.g., Selenium, Cypress, Playwright) and scripting language (Python preferred)
- Hands-on experience with testing AI models, including validation metrics, dataset versioning, and model explainability
- Familiarity with tools like MLflow, TensorBoard, or similar for AI observability
- Understanding of model bias, fairness audits, and compliance considerations in government or regulated environments
- Experience working in Agile DevSecOps environments with modern CI/CD tooling (Jenkins, GitHub Actions, GitLab, etc.)
- Ability to obtain and maintain a Public Trust security clearance
- Acceptable results of a background investigation
- Strong collaboration skills with clear communication across engineering, data science, and product teams
Core Competencies
Demonstrates expertise in AI/ML product testing, including automated testing frameworks, model validation, and ethical AI standards. Proficient in CI/CD practices and collaboration across engineering and data science teams.
Highest-signal resume keywords
- AI/ML Product Testing
- Automated Testing Frameworks
- Model Validation Metrics
- CI/CD Tooling
- Collaboration Skills
Hard Skills
- Software QA Engineering
- Automation Engineering
- Selenium
- Cypress
- Playwright
- Python
- Model Explainability
- Dataset Versioning
- Bias Detection
- Drift Monitoring
Soft Skills
- Collaboration
- Clear Communication
- Mentoring
Certifications & Qualifications
- Public Trust Security Clearance
Industry Keywords
- Agile DevSecOps
- Compliance Considerations
- Ethical AI Standards
- Fairness Audits
Tools & Technologies
- MLflow
- TensorBoard
- Jenkins
- GitHub Actions
- GitLab
#J-18808-Ljbffr
Automation Test Engineer Arbeitgeber: Jobtailor
Als Front Office Supervisor in unserem dynamischen Team bieten wir Ihnen die Möglichkeit, in einem unterstützenden und freundlichen Arbeitsumfeld zu wachsen. Wir legen großen Wert auf die berufliche Entwicklung unserer Mitarbeiter und bieten regelmäßige Schulungen sowie die Chance, Verantwortung zu übernehmen. Unsere Lage ermöglicht es Ihnen, Teil einer lebendigen Gemeinschaft zu sein, während Sie gleichzeitig die Standards unseres Franchise-Partners einhalten und unseren Gästen einen unvergesslichen Aufenthalt bieten.
