Large Scale Python Packages

camel-oasis

🏝️ OASIS: Open Agent Social Interaction Simulations with One Million Agents.

colossalai

Making large AI models cheaper, faster and more accessible

midisimx

Greatly improved, enhanced, and streamlined fork of midisim for calculating, searching, and analyzing MIDI-to-MIDI similarity at scale

es-pandas

Read, write and update large scale pandas DataFrame with Elasticsearch

torchdr

TorchDR - PyTorch Dimensionality Reduction

colossalai-nightly

Making large AI models cheaper, faster and more accessible

parl

A high-performance distributed training framework for Reinforcement Learning

ms-mint-app

Web-based post-processing tool for LC-MS metabolomics data. Extract, visualize, and analyze targeted metabolites with interactive browser-based interface.

brayns

Visualizer for large-scale and interactive ray-tracing of neurons

facedb

A package designed for efficient face recognition across extensive photo collections, optimized for large-scale processing.

quick-batch

ultra simple command line tool for docker-scaling batch processing

fleet-x

飞桨大模型开发套件，提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。

custom-colossalai

Making large AI models cheaper, faster and more accessible

redis-bulk-cleaner

Deletes keys from Redis database in bulk.

pyccea

A Python package of cooperative co-evolutionary algorithms for feature selection in high-dimensional data.

ms-mint-app2

MS-MINT is a Python application for processing, analyzing, and visualizing large liquid chromatography-mass spectrometry (LC-MS) metabolomics datasets

data-selection

DSIR large-scale data selection framework for language model training

neorl

NeuroEvolution Optimization with Reinforcement Learning

ringer

Large-scale data structures hosted on the file-system

fleet-lightning

飞桨大模型开发套件，提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。

ls-mcmc

A light-weight library for large-scale Markov Chain Monte Carlo sampling

gigl

Gigantic Graph Learning (GiGL) Framework: Large-scale training and inference for Graph Neural Networks

xdeeprank

An eXtensible Package of Deep Learning based Ranking Models for Large-scale Industrial Recommender System with Tensorflow

plsc

Paddle Large Scale Classification Tools，supports ArcFace, CosFace, PartialFC, Data Parallel + Model Parallel. Model includes ResNet, ViT, Swin, DeiT, CaiT, FaceViT, MoCo, MAE, ConvMAE, CAE.