Program Synthesis Python Packages

evalplus

Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024

multilspy

multilspy is a lsp client library in Python intended to be used to build applications around language servers.

leeroo-kapso

Kapso by Leeroo: Long-running agents that optimize AI and Data systems, and learn from every experience. #1 open-source on MLE-Bench; ALE-Bench; RelBench

geneticengine

Program Synthesis framework that features heuristics methods (e.g., Genetic Programming) and grammars are defined in Python dataclasses

bigcodebench

[ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

exform

Reshape text by example: show it a couple of before⇒after lines, it infers the transform and applies it to the whole file/stream — or emits a portable awk/python one-liner. Deterministic, offline, no regex, no LLM.

asts

Programmatic modification and evaluation of software

autogoal

A Python framework for program synthesis with a focus on Automated Machine Learning.

arcle

A Gymnasium-based Environment of the Abstraction and Reasoning Corpus (ARC)

rulesmith

DSPy searches for a readable rules file: field checks settle the easy cases for free, and a small judge model answers the rest with calibrated confidence. Same accuracy, a fraction of the model calls.

optiverse

A Python library for evolving code and algorithms using Large Language Models (LLMs)

o1op

Deterministic code synthesis operator — zero LLM in the loop. 8-color algebraic type system, 7-pass causal inference, 7-layer verification, 9-phase autonomous engagement. Substrate validated across 9 IEEE 2026 papers + IBM z/OS production assessment.

program-synthesis

Program Synthesis

tcgen

Program synthesis tool. Read a list of strings and create a generator that sample strings from the same distribution as input strings. Focused on test cases of competitive programming problems.

psynlp

Python module with Program Synthesis Techniques for NLP

evoeval

EvoEval: Evolving Coding Benchmarks via LLM

popper-ilp

An inductive logic programming system