Aws Glue Python Packages

awswrangler

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

aws-glue-schema-registry

Use the AWS Glue Schema Registry in Python projects.

glue-utils

glue-utils makes AWS Glue jobs less repetitive, more type-safe, and easier to reason about in production.

runner-dbt-aws-airflow

Run dbt projects on AWS Glue (Spark Job, Interactive Session, Python Shell) and EMR (Serverless, on-EC2) from Apache Airflow. Declarative YAML routing, task-collapse, OpenLineage.

piicatcher

Scan databases and data warehouses for PII data. Tag tables and columns in data catalogs like Amundsen and Datahub

pyspark-tools

SQL To Pyspark converter

aws-parquet

a toolkit that provides an object-oriented interface for working with parquet datasets on AWS

lakecli

A CLI to manage and monitor permissions in AWS Lake Formation