← LLM Models#32Open SourceGoogle
View on HF →
Gemma 2 27B
Google's open model family. Excellent instruction following at 27B.
Official page →📖Context8K tokens
🚀Speed120 tok/s
💵Input price$0.650/1M
💸Output price$0.650/1M
🧠Parameters27B
Benchmarks
Gray bar = dataset averageMMLU57 academic subjects
75.2%avg 81.8
GPQA DiamondPhD-level science questions
38.4%avg 50.4
HumanEvalPython code generation
74.0%avg 79.9
SWE-Bench VerifiedReal GitHub engineering tasks
14.7%avg 32.4
Composite ScoreScore = MMLU×20% + GPQA×30% + HumanEval×25% + SWE-Bench×25%48.7
HF Open LLM Leaderboard
IFEval · BBH · MATH · GPQA · MuSR · MMLU-Pro
Average
36.2
IFEvalInstruction following
79.8%avg 79.9
BBHBig Bench Hard
49.3%avg 48.7
MATH Lvl 5Competition mathematics
23.9%avg 37.8
GPQAGraduate-level science Q&A
16.7%avg 12.9
MuSRMultistep soft reasoning
9.1%avg 10.7
MMLU-ProProfessional knowledge
38.4%avg 40.0
More from Google
API Providers
Details
- Provider
- Parameters
- 27B
- Context
- 8K tokens
- Speed
- 120 tok/s
- Open Source
- Yes
- License
- Open (varies)
- Released
- -