Andromeda Sadiradra - Audio Rater - Generalist - English (Ireland)

Welo is an AI services company building data solutions to improve machine learning models. The team works on specialized evaluation projects that shape how AI systems interact with users. This role sits within Welo Data, their quality assurance division.

You'll work as an Audio Rater on the Andromeda Sadiradra project, evaluating voice conversations between people and AI agents. The work is fully remote for contractors based in Ireland, with a flexible 10 hours per week commitment starting as soon as possible. Pay is 27.30 dollars per hour.

Your main responsibilities

Listen to recorded interactions between users and AI voice agents, then evaluate the AI's responses using two separate rating scales. The first scale measures content quality—assessing whether responses are helpful, relevant, and contextually appropriate. The second evaluates prosody, or how natural, well-paced, and expressive the agent's voice delivery sounds. For each rating you assign on the 1 to 5 scales, you'll write a brief, specific justification explaining your score. Consistency across your ratings matters, so you'll need to maintain clear standards throughout your work.

To succeed in this role, you need strong listening skills and the ability to pick up on subtle details in audio quality and conversational flow. You should be comfortable making consistent judgments and defending them with clear reasoning. Written communication is important—your justifications need to be concise but descriptive enough that others understand your perspective. You'll also need reliable availability to commit to roughly 10 hours each week on an ongoing basis.

This is contract work, so you'll have the flexibility of freelance arrangements while contributing to AI model improvement. English fluency for Irish contractors is required. If you enjoy detailed analytical work and have an ear for both content quality and vocal delivery, this could be a good fit for your schedule.