microsoft/monitors4codegen
Code and Data artifact for NeurIPS 2023 paper - "Monitor-Guided Decoding of Code LMs with Static Analysis of Repository Context". `multispy` is a lsp client library in Python intended to be used to build applications around language servers.
Provides a Monitor-Guided Decoding framework that uses Language Server Protocol clients (via the `multilspy` library) to apply static analysis constraints during LM inference, enforcing type safety and semantic validity across Java, Rust, C#, and Python. The approach intercepts token generation to filter invalid identifiers and method calls by querying live language servers for repository-wide type information, achieving 19-25% compilation rate improvements without model retraining. Includes the PragmaticCode dataset (100 real-world repos with full build environments) and DotPrompts benchmark (10k+ method-completion examples) enabling evaluation on compilation success alongside traditional metrics.
280 stars. No commits in the last 6 months.
Stars
280
Forks
34
Language
Python
License
MIT
Category
Last pushed
Aug 08, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/microsoft/monitors4codegen"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
aalok-sathe/surprisal
A unified interface for computing surprisal (log probabilities) from language models! Supports...
EvolvingLMMs-Lab/lmms-engine
A simple, unified multimodal models training engine. Lean, flexible, and built for hacking at scale.
FunnySaltyFish/Better-Ruozhiba
【逐条处理完成】人为审核+修改每一条的弱智吧精选问题QA数据集
reasoning-machines/pal
PaL: Program-Aided Language Models (ICML 2023)
YutongWang1216/DocMTAgent
Code and data releases for the paper -- DelTA: An Online Document-Level Translation Agent Based...