Data Ingestion Python Packages

oneagent-sdk

Enables custom tracing of Python applications in Dynatrace

ingestr

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

bruin-sdk

Bruin Python SDK — eliminate boilerplate in Bruin Python assets

tracebloc-ingestor

tracebloc data pipeline for training/test dataset setup

omniload

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

druid-to-pinot-migrator

CLI and library to translate Apache Druid ingestion specs into Apache Pinot schemas, table configs, and ingestion jobs.

comet-hunter

An image ingestion and processing system, designed to assist in the discovery of comets.

llm-markdownify

Convert PDFs, images to high-quality Markdown using Vision LLMs.

sequor

Build complete API integrations with YAML and SQL. Rapid development without vendor lock-in and per-row costs.

feedunify

A high-performance, unifying library for data ingestion pipelines from multiple sources.

nanostream

Modules for ETL Pipelines

dataforge-dfg

Data Forge is a high-performance, CLI-first data integration tool designed to streamline the lifecycle of data from ingestion to transformation. Built with Python, it provides a robust framework for handling both ETL and ELT workflows with a focus on automation, reliability, and developer experience.

metalpipe

Modules for ETL Pipelines

bors

A highly flexible and versatile service integration framework.

sdmxflow

Download SDMX datasets into a reproducible, append-only on-disk layout for data warehouse refresh workflows.

skeem

Infer SQL DDL statements from tabular data.

ragpipe-ai

RAG in 3 functions. Ingest any data source into vector databases.