Quesma Releases OTelBench: Independent Benchmark Reveals Frontier LLMs Struggle with Real ...

New benchmark shows top LLMs achieve only 29% pass rate on OpenTelemetry instrumentation, exposing the gap between ...
Semiconductor Engineering

Benchmark For AI-Aided Chip Design That Evaluates LLMs Across 3 Critical Tasks (UCSD, Columbia)

Researchers at UCSD and Columbia University published “ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design.” Abstract “While Large Language Models (LLMs) show ...

When AI can write code: Students rethink learning, exams, and careers

AI's coding capabilities prompt students to reevaluate the value of traditional computer science education and future career paths.