AI Benchmarking Lead, Performance Benchmarking Evaluation
Amazon is seeking an AI Benchmarking Lead to support their Seller Assistant Gen-AI copilot by ensuring evaluation reliability and accuracy. The role involves auditing model performance, calibrating quality standards, and driving SOP improvements across international markets. Candidates will use SQL and Python to analyze data and improve evaluation metrics. This position reports within the ML data operations team at a massive scale.
50k new jobs listed every day. Install TAL to find more jobs like this.
Experience
1-3 years
Function
Quality Assurance
Work mode
Onsite, India
Company
Tier 1
What you will work on
Amazon is seeking an AI Benchmarking Lead to support their Seller Assistant Gen-AI copilot by ensuring evaluation reliability and accuracy. The role involves auditing model performance, calibrating quality standards, and driving SOP improvements across international markets. Candidates will use SQL and Python to analyze data and improve evaluation metrics. This position reports within the ML data operations team at a massive scale.
TAL's take
High-impact AI benchmarking role at Amazon working on large-scale Gen-AI products with clear performance metrics.
The JD provides a very clear breakdown of benchmarking responsibilities, team scale, and specific toolsets required for the AI quality role.
Salaries at Amazon
25.8 LPA average
Based on 3,341 Grapevine salary entries for Amazon.
Information Technology
0 - 2 years | L3
7 LPA average
Range: 2 - 12 LPA
Other
0 - 2 years | L3
3 LPA average
Range: 2 - 4 LPA
Human Resources
0 - 2 years | L3
6 LPA average
Range: 5 - 7 LPA
Accounting
0 - 2 years | L3
2 LPA average
Range: 2 - 2 LPA
Must haves
- Bachelor's degree or equivalent
- Experience in natural language data labeling or annotation
- Proficiency in MS Excel
- Basic understanding of SQL and Python
- Strong verbal and written communication skills in English
Tools and skills
Nice to have: sql, python.
About the company
Global FAANG company with a strong engineering presence in India.
Posts mentioning Amazon
Amazon - L5/6 salary range
Hi, what is the salary range for category manager - L5 and L6 roles at Amazon. What would the split be between fixed, bonus & rsu?
Amazon IN
I am currently a Junior Product Manager (L5) at Amazon IN. Looking to switch to some other role within Amazon (context - I am a business student and always wanted to try Marketing/Finance roles). Any recommendations on which roles can be good?
Amazon Sde 1 hiring
Any Amazonions here ? When will hiring again start for sde 1 and 2 ?