🔍 Read the full analysis: Egyptian And Nigerian English Join The Open ASR Leaderboard on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Hugging Face has added two private Liva AI evaluation splits for Egyptian- and Nigerian-accented English to its Open ASR Leaderboard. The conversational recordings total 2.89 hours; the announcement does not detail speaker numbers or show how the additions affect leaderboard scores.
Hugging Face has added two private English speech evaluation splits from Liva AI to its Open ASR Leaderboard, covering Egyptian- and Nigerian-accented English, as detailed in the original analysis. The conversational recordings total 2.89 hours, expanding the accents evaluated while giving leaderboard users new comparisons between US and non-US speech, and between English accent groups defined by Kachru’s Three Circles framework.
The two new evaluation sets are named Liva AI Conversational EG and Liva AI Conversational NG. Hugging Face says the Egyptian split contains 1.34 hours of audio and the Nigerian split contains 1.55 hours. Both consist of spontaneous conversation rather than scripted speech. Their transcripts are punctuated and cased, and retain disfluencies, according to the leaderboard announcement.
The splits are private evaluation data, not publicly available benchmark datasets. Hugging Face says private data can make evaluations more trustworthy by reducing developers’ ability to tune systems directly to publicly available test examples. The announcement does not provide scores for the new splits or explain whether existing leaderboard rankings changed as a result.
The leaderboard’s Private tab will show two accent comparisons: US versus non-US, and Inner Circle versus Outer and Expanding Circles. The non-US grouping includes the UK, Australia, Canada, India, Nigeria and Egypt. Under the current presentation, the Inner Circle grouping includes the US, UK, Australia and Canada, while India, Nigeria and Egypt appear in the Outer and Expanding grouping.
Egyptian and Nigerian English join the Open ASR Leaderboard
Hugging Face added private Liva AI speech splits to broaden accent evaluation. The short recordings add coverage, while leaving questions about how representative the results are.
What was added
Both datasets contain spontaneous conversation with punctuated, cased transcripts that retain disfluencies.
Liva AI Conversational EG
A private evaluation split for Egyptian-accented English.
Liva AI Conversational NG
A private evaluation split for Nigerian-accented English.
New ways to compare accents
The leaderboard’s Private tab will show results across two groupings.
- Non-US includes the UK, Australia, Canada, India, Nigeria and Egypt.
- Inner Circle compared with Outer and Expanding Circles, based on Kachru’s framework.
- Hugging Face plans to separate the latter circles as more accents are added.
Why private evaluation matters
Keeping test examples private may make it harder to tune systems directly to known benchmark material.
Users can inspect recognition results for two additional English varieties.
Scores describe these specific short test sets, not all speakers in either country.
What the New Accent Tests Add
The additions give users a way to examine speech-recognition results for Egyptian and Nigerian varieties of English, which were not included in the leaderboard’s previously listed accent groupings. Recognition performance can vary across speech varieties, so a single overall score may not show how a system works for every group of speakers. Separate results can help users identify differences that an aggregate score might obscure.
The new splits also have clear limits. Together they contain less than three hours of speech, and the announcement does not establish whether the recordings represent the range of English speakers in Egypt or Nigeria. The leaderboard results will measure performance on these particular test sets; they should not, on the information provided, be treated as a complete measure of recognition quality across either country.
Keeping evaluation material private may make it harder to optimize a model against the exact test recordings. But that feature does not by itself establish the quality or breadth of the sample. Without information about speakers and recording methods, users have limited grounds to judge how widely the scores apply.
Products Worth Considering
Samsung Galaxy Tab A7 Lite 8.7-Inch 32GB Wi-Fi Tablet, Gray
DJI Mic 2 Wireless Microphone System (2 Transmitters, Receiver, Charging Case)
Kindle Paperwhite 16GB (2024) – 7 Glare-Free Display, Weeks Battery Life – Black
As an affiliate, we earn on qualifying purchases.
The Leaderboard’s Earlier Accent Expansions
The Liva AI additions continue an effort by Hugging Face to broaden the speech represented in the Open ASR Leaderboard’s private evaluations. The announcement says the platform previously added 11 private splits from Appen and DataOceanAI, covering scripted and conversational speech across five accents. It later added a private split for Indian-accented English in collaboration with Voice Arena.
The new grouping draws on Kachru’s Three Circles framework, which categorizes varieties of English as Inner Circle, Outer Circle or Expanding Circle. Hugging Face credits Hume AI with the idea for the leaderboard comparison. The current display combines the Outer and Expanding Circles; the announcement says the platform plans to separate them as more accents are added.
The update expands the leaderboard’s listed coverage, but it does not describe a change to the recognition models being evaluated or provide a new overall ranking. Its reported development is the addition of test data and the accompanying way to compare results by accent group.
““We’re excited to announce a new collaboration with Liva AI to add even more private data with two new accents: Egyptian and Nigerian.””
— Hugging Face
What the Sample Details Do Not Show
The announcement does not say when the recordings were collected, how many speakers contributed, or how those speakers were selected. It also provides no breakdown by region, age or other characteristics that could help readers judge the range of voices represented. No independent assessment of the splits’ representativeness is included in the supplied material.
Hugging Face has not reported baseline scores or ranking effects associated with the additions in the announcement. It is also unclear whether the recordings or fuller dataset documentation may become available later; the splits are described as private. The size and composition of the test sets, and the way results will affect users’ interpretation of leaderboard performance, remain open questions.
Planned Changes to Accent Groups
Hugging Face says it plans to continue adding accents and languages and to separate the Outer and Expanding Circle groupings when more accents are available. The announcement gives no date or threshold for that change. For now, users can find the new comparisons in the leaderboard’s Private tab.
Further information about speaker composition, collection methods and per-accent scores would help readers assess how broadly the new evaluations apply and what they change in practice. The announcement does not set a timeline for such details or for additional data releases.
Key Questions
Which English accents are covered by the new splits?
The private Liva AI splits cover Egyptian-accented English and Nigerian-accented English.
How much speech do the new evaluation sets contain?
The Egyptian split contains 1.34 hours and the Nigerian split contains 1.55 hours, for a combined 2.89 hours.
What kind of recordings are included?
Both sets contain spontaneous conversational audio. Hugging Face says the transcripts are punctuated and cased and include disfluencies.
Can developers download the new datasets?
No public download is described. Hugging Face identifies them as private evaluation splits used by the leaderboard, rather than publicly available benchmark datasets.
Do the new splits show how well speech recognition works across Egypt and Nigeria?
They provide evaluations on these specific recordings, but the announcement gives no speaker counts or demographic and regional breakdowns. With 2.89 hours total and limited information about sample composition, the results cannot be assumed to represent all English speakers in either country.
Primary source: Hugging Face · via ThorstenMeyerAI.com
