Dataset Generation Python Packages

nfstream

NFStream: a Flexible Network Data Analysis Framework.

kiln-ai

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

datasetrising

Toolchain for creating custom datasets and training Stable Diffusion (1.x, 2.x, XL) models and LoRAs

kiln-server

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

dabook

Compile any book-like document into a *traceable, validated, editable Book Graph*, then export that graph as ML-ready datasets (raw / RAG / LLM / QA / code / dialogue) — resumable, multi-book, parallel, with a live localhost control room.

ragscore

⚡️ The "1-Minute RAG Audit" — Generate QA datasets & evaluate RAG systems in Colab, Jupyter, or CLI. Privacy-first, async, visual reports.

barred

Unofficial implementation of BARRED: generate faithful, diverse labeled datasets (arXiv:2604.25203)

scalexi

scalexi is a versatile open-source Python library, optimized for Python 3.11+, focuses on facilitating low-code development and fine-tuning of diverse Large Language Models (LLMs).

lightningrod-ai

Better AI Predictions ⚡

human-eval-rust

SigilDERG Data Production is an enterprise-grade Rust pipeline that crawls crates, runs rigorous scans (Clippy, Geiger, license checks), and generates instruction-style JSONL shards. It features semantic chunking, configurable splits, observability, and seamless SigilDERG ecosystem integration.

wildedge-sdk

Python SDK for WildEdge

bpycv

Computer vision utils for Blender (generate instance annoatation, depth and 6D pose by one line code)

auromat

AUROra MApping Toolkit - Python library / CLI tools for creating and working with georeferenced images for aurora research.

chatette

A powerful dataset generator for Rasa NLU, inspired by Chatito

pyslice

Data set templating library for model dataset creation and model running.

stopes

A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB team.

ks-llm-ranker

Ranking Large Language Models using the Principle of Least Action! Built during my time at Knit Space, Hubbali under the guidance Prof. Prakash Hegade.

scpscraper

A Python library designed for scraping data from the SCP wiki.

six2one

six2one is a command-line tool that allows you to fetch broadly, and export narrowly — you've never met an e621 downloader quite like this one.

iftg

IFTG (ImageFromTextGenerator) is a Python package that simplifies creating robust datasets for OCR models. Generate images from text, apply over 10 built-in noise effects, and customize fonts and layouts. IFTG supports all languages and offers endless noise combinations, including custom noise creation.

carnopy

Reproducible thermophysical datasets from scientific backends with visualization, provenance, and leakage-aware preparation for physics-informed machine-learning and engineering workflows.

pyreports

pyreports is a python library that allows you to create complex report from various sources

toolsgen

A modular Python library for synthesizing tool-calling datasets from JSON tool definitions using an LLM-as-a-judge pipeline. Designed for OpenAI-compatible APIs.

autosynth

Agentic synthetic-data generation framework inspired by Meta FAIR's Autodata / Agentic Self-Instruct.