BigCode project

BigCodeBench

BigCode project · Function-level code generation. What it measures: Programming task pass rate. Complete and Instruct use different prompts; Full and Hard use different task sets. Match them before comparing results.

Publisher: BigCode project
Scope: Function-level code generation
What it measures: Programming task pass rate
Source checked: Oct 3, 2026

Open original source Evaluation method

Scope

A model benchmark describes the tested model and task. It does not measure the whole tool, its interface, or every available plan.

The source repository was archived on 20 July 2026.

Tools in this category

A model benchmark describes the tested model and task. It does not measure the whole tool, its interface, or every available plan.

Bolt.new

AI coding tools

A browser-based prompt-to-app builder optimized for fast prototypes.

G2 4.3 / 5

96 reviewsSnapshot: Oct 3, 2026

Open tool Details

Cursor

AI coding tools

A familiar code editor centered on codebase context and agentic editing.

G2 4.6 / 5

323 reviewsSnapshot: Oct 3, 2026

Open tool Details

GitHub Copilot

AI coding tools

An AI coding assistant integrated across GitHub and popular development environments.

G2 4.4 / 5

399 reviewsSnapshot: Oct 3, 2026

Open tool Details

Replit

AI coding tools

A browser-based development platform with an agent, hosting and databases in one workspace.

G2 4.4 / 5

414 reviewsSnapshot: Oct 3, 2026

Open tool Details

Tabnine

AI coding tools

Tabnine’s platform page now leads to Tricentis AI Workspace. Current Tabnine access and plans remain unverified.

G2 4.2 / 5

72 reviewsSnapshot: Oct 3, 2026

Open tool Details

Windsurf

AI coding tools

Windsurf now continues as Devin Desktop: a full IDE with local and cloud agent management.

G2 4.3 / 5

35 reviewsSnapshot: Oct 3, 2026

Open tool Details