AI/ML - Evaluation Scientist, Siri Data at Apple (Cupertino, CA)

Would you like to play a part in the next revolution in human-computer interaction? Contribute to a product that is redefining mobile and desktop computing, and work with the people who built the intelligent assistant that helps millions of people get things done just by asking?
The vision for the Siri Data Organization is to improve Siri by using data as the voice of our customers. Within this organization the mission of the Analytics team is to inform the evolution of Siri through measurement and analysis of the user experience. Part of this mission is achieved through human evaluation; as an Evaluation Scientist, you will drive how we design our evaluation tasks and guidelines.

The Siri Data Organization is seeking a talented Evaluation Scientist to drive methodologies for measuring how Siri is performing for our users as part of our evaluation program. Your will help curate data sets and metrics that will be used across Siri, and will impact key decisions on the Siri product!

You will develop and own various evaluation tasks for curating data sets from our human evaluators.
This will include:
- working with engineering teams to bring the tasks to life
- ensuring that the tasks are designed to elicit valid and reliable labels
- partnering cross-functionally to ensure that the tasks are providing data that meets the Siri organizations needs
- Author and collaborate on evaluation guidelines to ensure the human evaluators understand how to apply the guidelines and interact with the evaluation workflows
- Conduct analyses to inform evaluation task designs and guidelines improvements

Please let the company know you found this position via so we can keep providing you with quality jobs.

See related AI/ML jobs