Ayb 0.5 Brings a Size Ladder to Armenian-Language AI
Metric-AI's Ayb 0.5 release offers three Armenian-adapted model sizes. The important story is not one leaderboard position, but a practical path for evaluating tokenization, language coverage, instruction following, and deployment tradeoffs.

Treat Ayb's three model sizes as candidates for task-specific testing, with explicit evaluation across Armenian varieties, grounding, instruction following, safety, and serving cost.
Ayb 0.5 is a new family of open models adapted for Armenian, with 0.8B, 4B, and 9B parameter variants. Metric-AI describes an adaptation pipeline built on Qwen3.5 that combines Armenian vocabulary expansion, continued pretraining on Armenian and English, instruction tuning, and initial post-training. The release matters because it gives teams several capacity points instead of forcing every Armenian-language use case into one model size.
That range also makes Ayb a useful reminder: language coverage is not a single capability. A model can tokenize Armenian efficiently yet struggle with factual recall, follow short instructions but lose constraints in longer conversations, or perform well in Eastern Armenian while being unreliable for Western Armenian and regional varieties. Choosing a model therefore requires evaluation that reflects the intended speakers and workflow, not merely a global benchmark rank.
Three sizes imply three different jobs
The smallest model is the natural candidate for constrained pipelines: classification, routing, text normalization, or a language component adjacent to OCR and speech systems. Its appeal is operational, but a small parameter count does not by itself prove that it will meet a latency or memory target on a particular device. Those properties must be measured with the exact runtime, quantization, context length, and batch pattern planned for production.
The 4B version occupies the middle ground for assistants and general text tasks. It may be the sensible first evaluation target when a team wants better language generation without immediately accepting the serving cost of the largest variant. The 9B model is positioned for more demanding conversations, complex instructions, and tool use. Bigger still does not mean universally safer or more accurate; it means evaluators should test whether the added capability produces enough value to justify its infrastructure cost.
A simple selection process starts with the smallest plausible model. Run the same representative task set across all three, record quality failures as well as throughput and memory, and move upward only when the smaller option misses a requirement that matters to users. This makes model size a testable engineering choice rather than a proxy for quality.
Read the benchmarks as a map, not a verdict
Metric-AI reports zero-shot results on ArmBench, which spans 24 tasks, and Armenian IFEval, an instruction-following evaluation. The release reports ArmBench overall scores of 0.5300, 0.5828, and 0.6164 for the 0.8B, 4B, and 9B models respectively. It also reports 0.729 for Ayb-9B on Armenian IFEval, with thinking mode disabled.
Those numbers help compare the family under a shared setup, but they do not answer whether a model is ready for a particular product. Benchmark aggregates can hide uneven behavior across reading comprehension, correction, factual knowledge, and multi-step reasoning. They also cannot represent every register, dialect, domain vocabulary, or prompt style an application will encounter.
A stronger local evaluation should separate at least four dimensions: comprehension of supplied Armenian text, factual answers that require outside knowledge, adherence to output constraints, and generation quality judged by fluent speakers. Keep a distinct slice for adversarial prompts and false premises. For retrieval-based systems, score whether answers remain grounded in the provided passage and whether the model declines to fill gaps with plausible details.
Tokenization is foundational, but only the first layer
Expanding a tokenizer with Armenian-specific vocabulary can reduce fragmentation, where a word is represented by many small pieces. That can make better use of a finite context window and may improve the efficiency of training and inference. Yet token counts alone do not demonstrate semantic understanding. The continued-pretraining and instruction-tuning stages are what connect those representations to useful behavior, and their coverage determines which language varieties and domains are served well.
Teams can inspect this boundary with a compact audit. Compare token counts for common Armenian sentences, names, dates, inflections, and mixed Armenian-English text. Then test meaning-preserving variations: change word order, substitute synonyms, introduce typographical noise, and use transliterated input. Stable answers across these changes are more informative than a polished response to one canonical prompt.
Deployment needs explicit language and safety boundaries
The authors say training was predominantly in Eastern Armenian and warn that Western Armenian, regional dialects, and Armenian written in Latin script are less reliable. They also identify uneven recent-event knowledge, incorrect details, limited multi-step reasoning, and unfinished preference tuning as limitations. These are product requirements, not footnotes. An interface should tell users which varieties were tested, offer a correction or escalation path, and avoid presenting confident prose as verification.
For higher-stakes domains, a refusal in a selected demonstration is not evidence of comprehensive safety. Teams need repeatable tests covering medical, legal, financial, privacy, and self-harm scenarios in natural Armenian phrasing. Human review and domain-specific safeguards remain necessary where mistakes can cause harm.
Ayb 0.5 broadens the set of tools available for Armenian-language experimentation. Its most constructive use will come from pairing that access with evaluations designed by Armenian speakers, transparent reporting by language variety, and deployment decisions based on observed task performance rather than headline rankings.


