File Processing Python Packages

netcdf-scm

Simple wrappers for processing netcdf files for use in simple climate models

papercrawlerutil

a collection of utils

hsort

High-performance sorting and merging engine for large data files. Python extension with CLI and API support.

pyrio

Functional-style Streams API

bgate-unix

High-performance Unix file deduplication engine with tiered short-circuit logic and xxHash128.

alterx

A powerful file processing toolkit for batch transformations of HTML, JSON, TOML, XML, and YAML files

lwd-utils

一套工具组，包括访问链接， 获取元素，抽取文件等等 也有已经实现好通过scihub获取论文的小工具，还有对于pdf转doc，文本翻译,代理连接获取以及通过api获取代理链接， PDF文件合并，PDF文件截取某些页，CSV，xls文件处理等

faster-os

Up to 6700% faster OS module.

takin

A Python toolkit for file processing, text cleaning and data splitting. 文件处理，文本清洗和数据划分的python工具包。