Hacker News
· 2026-10-30
· Evals
I decided to test this claim with my app, NoMoreHappyPath (NMHP), against Claude on a real project.Here is the test:I built Acremuse, a commercial real estate analysis platform, with Claude...
Hacker News
· 2026-10-28
· Evals
Algeon is a query engine built on Apache DataFusion that exposes NVIDIA's cuGraph, cuVS, and cuML as SQL table functions. It doesn't write its GPU kernels. Instead, it takes what NVIDIA already...
Anthropic Research
· 2026-10-23
· Evals
In a UK AISI evaluation, Mythos 5 took unsanctioned actions that impacted real people and organizations. In this post, we investigate these behaviors from Claude Mythos 5 further, to understand...
Hacker News
· 2026-10-21
· Evals
Article URL: https://www.aurorasre.ai/benchmark Comments URL: https://news.ycombinator.com/item?id=50180570 Points: 1 # Comments: 0
Hugging Face
· 2026-10-21
· Evals
Hugging Face
· 2026-10-20
· Evals
Anthropic Research
· 2026-10-16
· Evals
We gave Claude plain-language specifications for eight quantum circuits, along with an automated evaluator to check and score its designs. Working largely on its own, Claude improved on the best...
Databricks Blog
· 2026-10-15
· Evals
At Databricks, we continuously develop and evaluate new AI research techniques to...
Hacker News
· 2026-10-11
· Evals
Code has become extremely cheap to produce, if you have a 5x claude subscription, you can produce or at least try to produce any piece of software you want. And, to be honest, I actually like...
Hacker News
· 2026-10-11
· Evals
Article URL: https://arxiv.org/abs/2609.37315 Comments URL: https://news.ycombinator.com/item?id=50043658 Points: 1 # Comments: 0
Hacker News
· 2026-10-10
· Evals
Hey, I'm Akhil, co-founder of Tessary. An open-source agent reliability platform that monitors every production trace, detects issues that sampled evals miss, and investigates to find the root...
Anthropic Research
· 2026-10-09
· Evals
Investigating unintended model actions in our evaluations and internal use
Claude Blog
· 2026-10-08
· Evals
Bradley Axen, Head of AI Capabilities at Block, on how his team evaluates new large language models, how Claude Fable 5 orchestrates company-wide code migrations, and why safeguards are a critical...
Claude Blog
· 2026-10-05
· Evals
See how Cresta built Conductor, an agent that builds other agents, on the Claude Agent SDK, and how the team evaluates it with every new Claude model.
Claude Blog
· 2026-09-28
· Evals
Principles for designing evals and hillclimbing against them without fooling yourself, and how the claude-api skill's build-eval and hillclimb commands put them to work.
MLflow Blog
· 2026-09-25
· Evals
Test Jev on difficult answers, inspect its mistakes, and run it as an MLflow judge.
Claude Blog
· 2026-09-23
· Evals
Inside our performance sprint: the benchmarks Claude built, the loop each Slack thread ran, and the guardrails that let us ship 3,000 changes safely.
Hacker News
· 2026-09-22
· Evals
Article URL: https://benchmarkheaven.com/jev-models Comments URL: https://news.ycombinator.com/item?id=49800574 Points: 1 # Comments: 1
Hugging Face
· 2026-09-22
· Evals
MLflow Blog
· 2026-09-22
· Evals
Compare Jev with GPT, Claude, and DeepSeek as an LLM judge using MLflow across quality, cost, and latency.
Databricks Blog
· 2026-09-17
· Evals
The five criteria for evaluating a database for AI agents are branch isolation, serverless...
Claude Blog
· 2026-09-17
· Evals
Working at the frontier: How Balyasny Asset Management evaluates and governs Claude Fable 5
Claude Blog
· 2026-09-17
· Evals
Balyasny Asset Management (BAM) Chief AI Officer Charlie Flanagan on why the firm uses Claude Fable 5 and the role of safeguards in deploying frontier intelligence safely and reliably across the...
Hacker News
· 2026-09-11
· Evals
In the summer YC RFS, Aaron Epstien said ‘The next trillion users on the internet won't be people, they'll be AI agents. And now is the time to "Make Something Agents Want".’The “Make something...
Anthropic Research
· 2026-09-10
· Evals
Anthropic’s Frontier Red Team developed new evaluations to measure AI capabilities in tactical intelligence targeting and conventional weapons development.
Databricks Blog
· 2026-09-09
· Evals
Zepto's Push for Reliable, Real-Time Customer SupportZepto is one of India's fastest-growing...
Hacker News
· 2026-09-03
· Evals
This all started a few days ago when my wife's article:https://en.wikipedia.org/wiki/Jackson%E2%80%93Benton_brawl_o...got picked up to be on the homepage of wikipedia. I tapped on the view history...
Hacker News
· 2026-09-03
· Evals
Article URL: https://hotdogbenchmark.lol/ Comments URL: https://news.ycombinator.com/item?id=49549107 Points: 1 # Comments: 1
Claude Blog
· 2026-09-02
· Evals
The architecture, latency & cost techniques, and eval practices for agents that make it easier to buy and sell online.
Hugging Face
· 2026-09-01
· Evals
Hacker News
· 2026-08-28
· Evals
Article URL: https://condensedconcepts.blogspot.com/2026/08/questions-to-consider-when-evaluating-ai.html Comments URL: https://news.ycombinator.com/item?id=49483366 Points: 2 # Comments: 0
Anthropic Research
· 2026-08-28
· Evals
We had Claude autonomously train models to improve their performance on several public benchmarks that measure 10 categories of alignment failure. For all 10, Claude found fixes that improved the...
Databricks Blog
· 2026-08-27
· Evals
The MotivationMore and more enterprises are now asking agents to work with their...
Hacker News
· 2026-08-25
· Evals
Article URL: https://www.tbench.ai/ Comments URL: https://news.ycombinator.com/item?id=49430706 Points: 1 # Comments: 0
Hacker News
· 2026-08-25
· Evals
Article URL: https://ai.bedvibe.studio/retrieval-coverage/ Comments URL: https://news.ycombinator.com/item?id=49430614 Points: 1 # Comments: 0
Hacker News
· 2026-08-20
· Evals
I'm the developer behind CartoType. I’ve been working on bridging local language models with offline mapping, and I have just put together a demo of a completely offline Spatial RAG pipeline...
Databricks Blog
· 2026-08-18
· Evals
This year, Databricks hosted the inaugural Grounded Reasoning Cup, a first-of-its-kind...
Claude Blog
· 2026-08-13
· Evals
Securing the frontier: How JetBrains evaluates and deploys Claude Fable 5
Claude Blog
· 2026-08-13
· Evals
JetBrains Agent Systems CTO Vladislav Tankov on how the company evaluates frontier models against its private repositories, when his team reaches for Claude Fable 5, and why he treats safeguards...
Hacker News
· 2026-08-09
· Evals
Article URL: https://antigma.ai/eval Comments URL: https://news.ycombinator.com/item?id=49229621 Points: 1 # Comments: 0
Hacker News
· 2026-08-09
· Evals
Article URL: https://venturebeat.com/orchestration/qwen-3-8-max-and-claude-opus-5-show-why-raw-benchmark-scores-dont-predict-the-bill Comments URL: https://news.ycombinator.com/item?id=49229526...
Databricks Blog
· 2026-08-07
· Evals
AI assistants use language models, data retrieval, and reasoning to understand requests...
Hacker News
· 2026-08-06
· Evals
Article URL: https://felonybench.org/ Comments URL: https://news.ycombinator.com/item?id=49194758 Points: 2 # Comments: 0
Hacker News
· 2026-08-05
· Evals
For many everyday images, background removal works well. The remaining challenge is images with fine hair, fur, glass, transparent objects, motion blur, low-contrast subjects, or cluttered...
MLflow Blog
· 2026-08-02
· Evals
How to build measurable, testable, and continuously improving AI agent capabilities.
Hacker News
· 2026-07-27
· Evals
I built this after noticing that AI-agent PRs are hard to review. There are many tools available but it does not show the reason why the code can be problematic in case of agent.A normal diff can...
Hacker News
· 2026-07-25
· Evals
I’ve recently learned about neuromorphic computing while thinking about the core inefficiencies of current deep learning.Today AI relies heavily on super-dense layers, continuous computation of...
Hacker News
· 2026-07-23
· Evals
Article URL: https://apply.workable.com/toloka-ai/j/07A6121198 Comments URL: https://news.ycombinator.com/item?id=49019347 Points: 1 # Comments: 0
Claude Blog
· 2026-07-17
· Evals
Nate Schmidt's job at Cursor is to evaluate frontier models against their ability to tackle long-running, real-world engineering problems. Here’s why–and how–Claude Fable 5 changed the calculus on...
Hugging Face
· 2026-07-16
· Evals
Hacker News
· 2026-07-14
· Evals
Article URL: https://arxiv.org/abs/2607.11871 Comments URL: https://news.ycombinator.com/item?id=48904369 Points: 1 # Comments: 0
Claude Blog
· 2026-07-13
· Evals
Hebbia builds research and diligence software for financial professionals, and tests every new model against finance evals tied to expert outcomes. In testing, Claude Fable 5 posted the biggest...
Databricks Blog
· 2026-07-08
· Evals
At Databricks, the way we build software is changing quickly as we aggressively adopt...
MLflow Blog
· 2026-07-08
· Evals
MLflow Review Queues turn AI trace review into a ticketing system with assignments, status, and human evaluations — so you can stop tracking degradations and misbehavior in spreadsheets.
Hacker News
· 2026-07-06
· Evals
Article URL: https://flower.ai/benchmarks/flowerbench/ Comments URL: https://news.ycombinator.com/item?id=48803428 Points: 2 # Comments: 1
Hacker News
· 2026-07-06
· Evals
I'm looking for this myself but figured it's good to have an actual discussion about this. I'm pretty new to the benchmarking side of LLMs.For example, I looked at eyeballvull [1]. It seems...
Hugging Face
· 2026-06-30
· Evals
Hugging Face
· 2026-06-30
· Evals
Databricks Blog
· 2026-06-23
· Evals
Retrieval Augmented Generation (RAG) is an AI architecture pattern that connects...
Hacker News
· 2026-06-22
· Evals
Article URL: https://github.com/arsenis-cmd/clai-benchmarks Comments URL: https://news.ycombinator.com/item?id=48629740 Points: 2 # Comments: 0
Hugging Face
· 2026-06-18
· Evals
Hacker News
· 2026-06-15
· Evals
There's always a generous look at the code you've come up with. But the pantheon is different. The pantheon is made by turning multiple sub-agents, and when the scorer scores them one by one and...
Hugging Face
· 2026-06-12
· Evals
Hugging Face
· 2026-06-09
· Evals
Hacker News
· 2026-05-31
· Evals
I've been running Crypto Predictions for over a month with a limited number of coins and models (some models in the same family the produce close results have been removed)I think you guys might...
MLflow Blog
· 2026-05-28
· Evals
Use Google ADK's trajectory, response-match, and LLM-judge evaluators as MLflow scorers for end-to-end agent evaluation.
Hugging Face
· 2026-05-27
· Evals
Hacker News
· 2026-05-22
· Evals
Article URL: https://modelrift.com/blog/openscad-llm-benchmark/ Comments URL: https://news.ycombinator.com/item?id=48234090 Points: 1 # Comments: 0
Hacker News
· 2026-05-20
· Evals
a few weeks ago, I built professor goose, a socratic ai tutor built around the feynman idea that if you can’t explain it, you don’t understand it. the way professor goose works specifically is you...
Hacker News
· 2026-05-20
· Evals
Article URL: https://github.com/serenedb/serenedb/tree/main/libs/iresearch Comments URL: https://news.ycombinator.com/item?id=48205526 Points: 6 # Comments: 1
Hacker News
· 2026-05-16
· Evals
Article URL: https://github.com/albedan/ai-ml-gpu-bench Comments URL: https://news.ycombinator.com/item?id=48158245 Points: 1 # Comments: 0
Databricks Blog
· 2026-05-08
· Evals
Recently announced Genie Code is Databricks’ autonomous AI partner purpose built for data work. ...
Hacker News
· 2026-05-03
· Evals
Article URL: https://www.nist.gov/news-events/news/2026/05/caisi-evaluation-deepseek-v4-pro Comments URL: https://news.ycombinator.com/item?id=47994920 Points: 1 # Comments: 0
Hacker News
· 2026-05-02
· Evals
Hi HN — author here. iesna.eu is a browser-based ecosystem for working with photometric data: parsing standard luminaire files (LDT/EULUMDAT, IES LM-63, Oxytech, ATLA-S001), running design...
Hacker News
· 2026-05-01
· Evals
Article URL: https://arxiv.org/abs/2604.27906 Comments URL: https://news.ycombinator.com/item?id=47972683 Points: 1 # Comments: 0
Hugging Face
· 2026-04-29
· Evals
Anthropic Research
· 2026-04-29
· Evals
Apr 29, 2026ScienceEvaluating Claude’s bioinformatics research capabilities with BioMysteryBench
Hacker News
· 2026-04-25
· Evals
I shipped a wiki layer for AI agents that uses markdown + git as the source of truth, with a bleve (BM25) + SQLite index on top. No vector or graph db yet.It runs locally in ~/.wuphf/wiki/ and you...
MLflow Blog
· 2026-04-22
· Evals
Shipping your first AI agent or LLM application feels fulfilling until you have to make changes because it does not work as you intended. Most of us start the same way: we test a few prompts, the...
MLflow Blog
· 2026-04-21
· Evals
MLflow AI Gateway now supports configurable guardrails that use LLM judges to block or sanitize harmful content, PII, and custom policy violations before they reach your users or your models.
Hacker News
· 2026-04-13
· Evals
every "best backup plugin" comparison is affiliate content with no real data. so i built a site that runs actual backup/restore operations in docker containers and publishes real benchmark...
Hacker News
· 2026-04-13
· Evals
Article URL: https://github.com/Evol-ai/SkillCompass Comments URL: https://news.ycombinator.com/item?id=47749727 Points: 1 # Comments: 0
Hacker News
· 2026-04-08
· Evals
Article URL: https://penfieldlabs.substack.com/p/milla-jovovich-just-released-an-ai Comments URL: https://news.ycombinator.com/item?id=47687208 Points: 2 # Comments: 0
LangChain Blog
· 2026-04-08
· Evals
By Vivek Trivedy, Product Manager💡TL;DR: We can build better agents by building better harnesses. But to autonomously build a “better” harness, we need a strong learning signal to “hill-climb” on....
Hacker News
· 2026-04-06
· Evals
Article URL: https://psycnet.apa.org/fulltext/2027-12675-001.html Comments URL: https://news.ycombinator.com/item?id=47658237 Points: 1 # Comments: 0
Google Research
· 2026-04-03
· Evals
Generative AI
LangChain Blog
· 2026-04-02
· Evals
💡TL;DR: Open models like GLM-5 and MiniMax M2.7 now match closed frontier models on core agent tasks — file operations, tool use, and instruction following — at a fraction of the cost and latency....
Google Research
· 2026-03-31
· Evals
Algorithms & Theory
LangChain Blog
· 2026-03-27
· Evals
A practical checklist for agent evaluation: error analysis, dataset construction, grader design, offline & online evals, and production readiness.
LangChain Blog
· 2026-03-26
· Evals
Discover how Kensho, S&P Global’s AI innovation engine, leveraged LangGraph to create its Grounding framework–a unified agentic access layer solving fragmented financial data retrieval at enterprise scale.
LangChain Blog
· 2026-03-26
· Evals
💡TLDR: The best agent evals directly measure an agent behavior we care about. Here's how we source data, create metrics, and run well-scoped, targeted experiments over time to make agents more...
Hacker News
· 2026-03-24
· Evals
What it isKnitting is a from-scratch concurrency framework for JavaScript built on shared memory. Instead of manually wiring message protocols, you just call functions across threads.Why I built...
Hugging Face
· 2026-03-24
· Evals
MLflow Blog
· 2026-03-24
· Evals
AI coding agents edit files, run commands, and browse the web autonomously, but what are they actually doing? Learn how to trace every step, evaluate output quality, and control LLM spending for...
MLflow Blog
· 2026-03-24
· Evals
AI coding agents edit files, run commands, and browse the web autonomously, but what are they actually doing? Learn how to trace every step, evaluate output quality, and control LLM spending for...
MLflow Blog
· 2026-03-24
· Evals
AI coding agents edit files, run commands, and browse the web autonomously, but what are they actually doing? Learn how to trace every step, evaluate output quality, and control LLM spending for...
MLflow Blog
· 2026-03-23
· Evals
How to test Claude Code skills using MLflow tracing and LLM judges, and create a self-improvement loop where Claude Code refines its own skills.
MLflow Blog
· 2026-03-23
· Evals
How to test Claude Code skills using MLflow tracing and LLM judges, and create a self-improvement loop where Claude Code refines its own skills.
MLflow Blog
· 2026-03-23
· Evals
How to test Claude Code skills using MLflow tracing and LLM judges, and create a self-improvement loop where Claude Code refines its own skills.
MLflow Blog
· 2026-03-21
· Evals
Bring MLflow experiment tracking and execution tracing to Inspect AI evaluations with the inspect-mlflow package.
MLflow Blog
· 2026-03-18
· Evals
Making your agents work reliably in production requires observability, evaluation, version control, and governance. Learn how MLflow brings all four together as the only complete open source AI platform.
MLflow Blog
· 2026-03-18
· Evals
Making your agents work reliably in production requires observability, evaluation, version control, and governance. Learn how MLflow brings all four together as the only complete open source AI platform.
MLflow Blog
· 2026-03-18
· Evals
Making your agents work reliably in production requires observability, evaluation, version control, and governance. Learn how MLflow brings all four together as the only complete open source AI platform.
Databricks Blog
· 2026-03-17
· Evals
Retrieval underpins modern AI systems, and the quality of the embedding model determines...
Hacker News
· 2026-03-16
· Evals
Article URL: https://www.youtube.com/watch?v=tz5wALHhhds Comments URL: https://news.ycombinator.com/item?id=47396503 Points: 1 # Comments: 0
Hugging Face
· 2026-03-13
· Evals
Hacker News
· 2026-03-12
· Evals
As anyone with an internet connection knows, there’s been a lot of buzz about how AI is going to reshape the workforce for the past 3 years and layoffs due to “AI” have already started, the most...
Databricks Blog
· 2026-03-11
· Evals
Databricks is excited to announce the acquisition of Quotient AI, an innovator in...
Hacker News
· 2026-03-08
· Evals
Article URL: https://arxiv.org/abs/2603.03823 Comments URL: https://news.ycombinator.com/item?id=47295537 Points: 8 # Comments: 1
Hacker News
· 2026-03-08
· Evals
dlgo is an LLM inference engine written in Go. CPU path has zero dependencies beyond the standard library. GPU path uses Vulkan compute — no CUDA required.I benchmarked it against Ollama using the...
Hacker News
· 2026-03-08
· Evals
I was researching the personal finance market and initially found only a few obvious companies.After digging more, I found a much longer tail of startups I’d never heard of, including products...
Anthropic Engineering
· 2026-03-06
· Evals
Evaluating Opus 4.6 on BrowseComp, we found cases where the model recognized the test, then found and decrypted answers to it—raising questions about eval integrity in web-enabled environments.
Anthropic Engineering
· 2026-03-06
· Evals
Evaluating Opus 4.6 on BrowseComp, we found cases where the model recognized the test, then found and decrypted answers to it—raising questions about eval integrity in web-enabled environments.
LangChain Blog
· 2026-03-05
· Evals
By Robert XuRecently at LangChain we’ve been building skills to help coding agents like Codex, Claude Code, and Deep Agents CLI work with our ecosystem: namely, LangChain and LangSmith. This is...
MLflow Blog
· 2026-03-04
· Evals
Score agent plans, tool calls, and reasoning with TruLens GPA framework through mlflow.genai.evaluate().
LangChain Blog
· 2026-03-04
· Evals
We’re releasing a CLI along with our first set of skills to give AI coding agents expertise in the LangSmith ecosystem. This includes adding tracing to agents, understanding their execution,...
LangChain Blog
· 2026-03-04
· Evals
We’re releasing our first set of skills to give AI coding agents expertise in the open source LangChain ecosystem. This includes building agents with LangChain, LangGraph, and Deep Agents. On our...
MLflow Blog
· 2026-03-04
· Evals
Score agent plans, tool calls, and reasoning with TruLens GPA framework through mlflow.genai.evaluate().
MLflow Blog
· 2026-03-04
· Evals
Score agent plans, tool calls, and reasoning with TruLens GPA framework through mlflow.genai.evaluate().
MLflow Blog
· 2026-03-02
· Evals
High-level summary: problems, approaches, and takeways for better RAG with MLflow
Hamel Husain's Blog
· 2026-03-02
· Evals
Evals Skills for Coding Agents
MLflow Blog
· 2026-03-02
· Evals
High-level summary: problems, approaches, and takeways for better RAG with MLflow
MLflow Blog
· 2026-03-02
· Evals
High-level summary: problems, approaches, and takeways for better RAG with MLflow
MLflow Blog
· 2026-02-27
· Evals
Add fast, deterministic safety validation to MLflow evaluation pipelines using Guardrails AI scorers. No LLM required.
MLflow Blog
· 2026-02-27
· Evals
Add fast, deterministic safety validation to MLflow evaluation pipelines using Guardrails AI scorers. No LLM required.
MLflow Blog
· 2026-02-27
· Evals
Add fast, deterministic safety validation to MLflow evaluation pipelines using Guardrails AI scorers. No LLM required.
LangChain Blog
· 2026-02-26
· Evals
You can't monitor agents like traditional software. Inputs are infinite, behavior is non-deterministic, and quality lives in the conversations themselves. This article explains what to monitor,...
MLflow Blog
· 2026-02-24
· Evals
MLflow 3.10 introduces multi-turn evaluation and conversation simulation so you can score entire conversations, test agent changes with reproducible scenarios, and catch failures that only surface...
MLflow Blog
· 2026-02-24
· Evals
MLflow 3.10 introduces multi-turn evaluation and conversation simulation so you can score entire conversations, test agent changes with reproducible scenarios, and catch failures that only surface...
MLflow Blog
· 2026-02-24
· Evals
MLflow 3.10 introduces multi-turn evaluation and conversation simulation so you can score entire conversations, test agent changes with reproducible scenarios, and catch failures that only surface...
LangChain Blog
· 2026-02-22
· Evals
You can't build reliable agents without understanding how they reason, and you can't validate improvements without systematic evaluation.
LangChain Blog
· 2026-02-18
· Evals
Learn how monday Service developed an eval-driven development framework for their customer-facing service agents.
Hugging Face
· 2026-02-12
· Evals
Anthropic Engineering
· 2026-02-05
· Evals
Infrastructure configuration can swing agentic coding benchmarks by several percentage points—sometimes more than the leaderboard gap between top models.\n\n
Anthropic Research
· 2026-02-05
· Evals
Evaluating and mitigating the growing risk of LLM-discovered 0-days
Anthropic Engineering
· 2026-02-05
· Evals
Agentic coding benchmarks are often treated as precise measurements of relative model capability. However, we’ve found that infrastructure configuration alone can produce meaningful differences in...
MLflow Blog
· 2026-02-03
· Evals
Introducing MemAlign, a new framework that aligns LLMs with human feedback via a lightweight dual-memory system, achieving competitive or better quality than state-of-the-art prompt optimizers, at...
MLflow Blog
· 2026-02-03
· Evals
Introducing MemAlign, a new framework that aligns LLMs with human feedback via a lightweight dual-memory system, achieving competitive or better quality than state-of-the-art prompt optimizers, at...
MLflow Blog
· 2026-02-03
· Evals
Introducing MemAlign, a new framework that aligns LLMs with human feedback via a lightweight dual-memory system, achieving competitive or better quality than state-of-the-art prompt optimizers, at...
MLflow Blog
· 2026-01-29
· Evals
Improve your agents using MLflow's extensive, industry-leading suite of high-quality LLM judges.
MLflow Blog
· 2026-01-29
· Evals
Improve your agents using MLflow's extensive, industry-leading suite of high-quality LLM judges.
MLflow Blog
· 2026-01-29
· Evals
Improve your agents using MLflow's extensive, industry-leading suite of high-quality LLM judges.
Hugging Face
· 2026-01-27
· Evals
Hugging Face
· 2026-01-21
· Evals
Anthropic Engineering
· 2026-01-21
· Evals
What we learned from three iterations of a performance engineering take-home that Claude keeps beating.
Anthropic Engineering
· 2026-01-21
· Evals
What we learned from three iterations of a performance engineering take-home that Claude keeps beating.
Hamel Husain's Blog
· 2026-01-15
· Evals
LLM Evals: Everything You Need to Know
Anthropic Engineering
· 2026-01-09
· Evals
The capabilities that make agents useful also make them difficult to evaluate. The strategies that work across deployments combine techniques to match the complexity of the systems they measure. \n
Anthropic Engineering
· 2026-01-09
· Evals
Guidance on how to design more rigorous and useful evals for agents across a range of agent architectures and use cases in real-world deployment.
MLflow Blog
· 2025-10-15
· Evals
How to quickly prototype an agent using the Claude Agent SDK then instrument and evaluate it with MLflow
Hamel Husain's Blog
· 2025-10-01
· Evals
Selecting The Right AI Evals Tool
Google Research
· 2025-09-24
· Evals
Generative AI
Hugging Face
· 2025-09-18
· Evals
MLflow Blog
· 2025-09-15
· Evals
How to easily create custom evaluators that understand the semantics of your domain and automatically align with human experts
Anthropic Engineering
· 2025-09-11
· Evals
Agents are only as effective as the tools we give them. We share how to write high-quality tools and evaluations, and how you can boost performance by using Claude to optimize its tools for itself.
MLflow Blog
· 2025-08-30
· Evals
Building GenAI tools presents a unique set of challenges. As we evaluate accuracy, iterate on prompts, and enable collaboration, we often encounter bottlenecks that slow down our progress toward...
Google Research
· 2025-08-26
· Evals
Generative AI
MLflow Blog
· 2025-08-11
· Evals
In MLflow 3.2, we introduced the concept of assessments, which are the quality evaluations and trace annotations that are crucial for understanding and improving your AI applications. With the...
Hugging Face
· 2025-08-01
· Evals
Hugging Face
· 2025-07-17
· Evals
Anthropic Research
· 2025-07-15
· Evals
Detailed cyber evaluations of Claude 4
Hugging Face
· 2025-07-04
· Evals
Hamel Husain's Blog
· 2025-06-23
· Evals
Inspect AI, An OSS Python Library For LLM Evals
Anthropic Research
· 2025-06-16
· Evals
SHADE-Arena: Evaluating sabotage and monitoring in LLM agents
Hugging Face
· 2025-06-06
· Evals
Google Research
· 2025-05-14
· Evals
Data Mining & Modeling
Hugging Face
· 2025-04-16
· Evals
Hugging Face
· 2025-02-28
· Evals
Hugging Face
· 2025-02-04
· Evals
Anthropic Engineering
· 2025-01-06
· Evals
SWE-bench is an AI evaluation benchmark that assesses a model's ability to complete real-world software engineering tasks.
Hugging Face
· 2024-12-17
· Evals
Hugging Face
· 2024-12-04
· Evals
Hugging Face
· 2024-11-19
· Evals
Anthropic Research
· 2024-11-19
· Evals
A statistical approach to model evaluations
Hamel Husain's Blog
· 2024-10-29
· Evals
Using LLM-as-a-Judge For Evaluation: A Complete Guide
Anthropic Research
· 2024-10-18
· Evals
Sabotage evaluations for frontier models
Anthropic Engineering
· 2024-09-19
· Evals
For an AI model to be useful in specific contexts, it often needs access to background knowledge.
Hugging Face
· 2024-07-25
· Evals
Hugging Face
· 2024-07-01
· Evals
Hugging Face
· 2024-06-18
· Evals
Hugging Face
· 2024-05-24
· Evals
Hugging Face
· 2024-04-19
· Evals
Hugging Face
· 2024-04-16
· Evals
Hamel Husain's Blog
· 2024-03-29
· Evals
Your AI Product Needs Evals
Hugging Face
· 2024-02-27
· Evals
Hugging Face
· 2024-02-20
· Evals
Hugging Face
· 2024-02-02
· Evals
Anthropic Research
· 2023-12-07
· Evals
Evaluating and Mitigating Discrimination in Language Model Decisions
Anthropic Research
· 2023-10-04
· Evals
Challenges in evaluating AI systems
Anthropic Research
· 2022-12-19
· Evals
Discovering Language Model Behaviors with Model-Written Evaluations
Hugging Face
· 2022-10-24
· Evals
Hugging Face
· 2022-10-03
· Evals
Google Developers Blog
·
· Evals
To bridge the gap between static model knowledge and rapidly evolving software practices, Google DeepMind developed a "Gemini API developer skill" that provides agents with live documentation and...
Google Developers Blog
·
· Evals
The newly introduced continuous checkpointing feature in Orbax and MaxText is designed to optimize the balance between reliability and performance during model training, addressing issues with...
Google Developers Blog
·
· Evals
Google Cloud has introduced the Agents CLI, a specialized tool designed to bridge the gap between local development and production-grade AI agent deployment. The CLI provides coding assistants...
Google Developers Blog
·
· Evals
LiteRT is a production-ready framework designed to help mobile developers unlock the power of Neural Processing Units (NPUs), overcoming the performance and battery limitations of traditional CPU...
Google Developers Blog
·
· Evals
Google announced the transition from assistive AI to independent agents, highlighting the launch of the Gemini 3.5 series and major updates to its Antigravity agent-first development platform. For...
Google Developers Blog
·
· Evals
Building AI agents often leaves developers uncertain if prompt tweaks to fix single errors will accidentally cause widespread regressions in production. To bridge this gap, Google has introduced a...
Google Developers Blog
·
· Evals
Google's open-source TPU microbenchmark suite provides developers with granular performance metrics across Network, Compute, HBM, Host Transfer, and Attention components to validate real-world...
Google Developers Blog
·
· Evals
Agent Platform's evaluation service is now generally available, providing developers with a unified engine to measure agent quality consistently across local development experiments and live...
Google Developers Blog
·
· Evals
Moving live voice agents from demo to production requires rigorous, automated testing to handle the unpredictability of real multi-turn conversations. ADK now provides native live evaluation,...
Google Developers Blog
·
· Evals
Google Cloud has natively integrated TPU support into the vLLM serving engine, allowing developers to elastically scale high-demand embedding pipelines using Google Kubernetes Engine (GKE). To...
Google Developers Blog
·
· Evals
While end-to-end benchmarks like SWE-bench provide broad performance scores for AI agents, they are often expensive, slow, and lack the root-cause diagnostics needed to explain exactly where an...
Google Developers Blog
·
· Evals
This blog post explores how to transition AI agents from static, build-time security controls to dynamic runtime governance using the Gemini Enterprise Agent Platform. It highlights three primary...
Google Developers Blog
·
· Evals
The "autofinetune" project introduces an autonomous research loop that fully automates LLM post-training workflows, including Supervised Fine-Tuning (SFT) and Reinforcement Learning via GRPO. By...
Google Developers Blog
·
· Evals
The MaxText team successfully reproduced AI2’s OLMo 3 7B language model from scratch on Google Cloud TPUs using JAX/XLA, precisely matching the original PyTorch-on-GPU reference across...
Google Developers Blog
·
· Evals
EmbeddingGemma 2 is a new 740M open-weight multimodal model that maps text, images, video, and audio into a unified vector space for privacy-first, on-device retrieval. Developers can easily...
Google Developers Blog
·
· Evals
EmbeddingGemma 2 is a compact, open-source multimodal embedding model that maps text, code, images, video, and audio into a unified 768-dimensional space. Developers can use the...
Google Developers Blog
·
· Evals
A practical engineering guide and reference architecture for establishing continuous evaluation pipelines for multi-turn AI agents using Gemini Enterprise Agent Platform, Google ADK, and...
Google Developers Blog
·
· Evals
This post announces native PyTorch backends for vLLM and SGLang powered by TorchTPU, enabling developers to run and optimize serving workloads using familiar PyTorch tools. Built in collaboration...
Google Developers Blog
·
· Evals
The Agents CLI Extension System enables developers to use frameworks like LangChain, LangGraph, or custom enterprise stacks while retaining a unified deployment and evaluation workflow. It details...
Google Developers Blog
·
· Evals
This post walks through the Agents CLI evaluation loop for systematically improving agent performance through continuous testing and grading. It demonstrates how to catch prompt regressions,...
No articles match your filters.