k4black/codebleu

Combines syntactic AST matching and semantic data-flow analysis with n-gram scoring to evaluate code generation quality across 10 programming languages. Built on tree-sitter for language-agnostic parsing, it decomposes scores into four components (n-gram match, weighted n-gram match, syntax match, dataflow match) that can be individually weighted. Integrates with HuggingFace's evaluate library for streamlined evaluation workflows.

130 stars and 5,089 monthly downloads. Used by 1 other package. No commits in the last 6 months. Available on PyPI.