Datalake Python Packages

lakefs-sdk

lakeFS - Data version control for your data lake | Git for data

587K 6K 482

starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

563K 12K 3K

lakefs

lakeFS - Data version control for your data lake | Git for data

467K 6K 482

deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

113K 9K 724

pandasai

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

35K 24K 2K

lakefs-client

lakeFS - Data version control for your data lake | Git for data

11K 6K 482

pan-cortex-data-lake

Python idiomatic SDK for Cortex™ Data Lake.

7K 48 22

pandasai-openai

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

6K 24K 2K

dlt-iceberg

An Iceberg destination for DLT that supports REST catalogs

6K 14 6

pandasai-litellm

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

4K 24K 2K

apache-gravitino

World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.

3K 3K 951

ftm-lakehouse

Data standard and archive storage for structured FollowTheMoney data, leaked data, private and public document collections.

3K 9 1

pancloud

Python idiomatic SDK for Cortex™ Data Lake.

2K 48 22

omniload

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

2K 5 4

zingg

Scalable master data management, identity resolution, entity resolution, and deduplication using ML

1K 1K 178

tarn

An insanely customizable framework for key-value storage 💾

1K 3 0

dlt-filesystem

omniload is a polyglot data loader based on dlt. Copy data between any source and any destination.

1K 5 4

dm-job-lib

Better organize data in data lake and build ETL pipeline with Web UI tool.

702 10 2

aws-insurancelake-etl

This solution helps you deploy ETL processes and data storage resources to create an Insurance Lake using Amazon S3 buckets for storage, AWS Glue for data transformation, and AWS CDK Pipelines. It is originally based on the AWS blog Deploy data lake ETL jobs using CDK Pipelines, and complements the InsuranceLake Infrastructure project

690 38 16