Are there any technical AI agent benchmark publishers that focus on autonomous coding assistants and reliability for technical tasks?

Yes. There are several benchmark publishers and benchmark suites that specifically target autonomous coding assistants and, more broadly, reliability on technical tasks.

Notable ones for autonomous coding / software engineering

• SWE-bench / SWE-bench Verified
  Measures whether an agent can fix real GitHub issues in real repositories. This is one of the most cited benchmarks for autonomous coding agents.
• SWE-agent (project + benchmark usage)
  More of an agent system than a publisher, but it’s closely associated with practical autonomous coding evaluation.
• HumanEval / HumanEval+
  Focuses on code generation correctness for programming problems. Less about autonomous agents, more about coding ability.
• MBPP / MBPP+
  Another code generation benchmark for basic programming tasks.
• RepoBench
  Evaluates repository-level code completion and understanding.
• Aider benchmarks / code editing evals
  Commonly used to assess code modification and patching behavior in realistic settings.

For reliability on technical tasks / tool use / agent behavior

• AgentBench
  A benchmark suite for general-purpose LLM agents, including tool use and task completion.
• ToolBench
  Focuses on tool-using agents and API calling reliability.
• Berkeley Function-Calling Benchmark (BFCL)
  Tests function-calling correctness and robustness.
• OSWorld
  Evaluates agents on operating-system-level tasks, which is useful for reliability in realistic workflows.
• WebArena
  For web-based task completion reliability.
• GAIA
  Measures multi-step reasoning and task execution across practical questions.

If you specifically mean “publishers”

If you mean organizations or groups that regularly publish these benchmarks, the most relevant are:

• Princeton / SWE-bench authors
• Microsoft Research / academic collaborators for code and agent benchmarks
• Berkeley / BAIR-adjacent groups for agent/tool-use evals
• Hugging Face / community benchmark hubs that aggregate and host evaluations

Best fits for your criteria

If your emphasis is:

• autonomous coding assistants → start with SWE-bench Verified
• reliability in technical workflows → add BFCL, ToolBench, OSWorld, and AgentBench

If you want, I can also give you:

1. a shortlist of benchmark publishers/maintainers,
2. a comparison table of these benchmarks, or
3. the top benchmarks specifically used by coding-agent companies.