Metadata-Version: 2.4
Name: dlt-fabric
Version: 1.29.0.post9
Summary: Maintained fork of dlt with fixes for the Microsoft Fabric Warehouse destination
Project-URL: Homepage, https://github.com/sdebruyn/dlt
Project-URL: Repository, https://github.com/sdebruyn/dlt
Project-URL: Upstream dlt, https://github.com/dlt-hub/dlt
Author-email: Sam Debruyn <sam@debruyn.dev>
Maintainer-email: Sam Debruyn <sam@debruyn.dev>
License-Expression: Apache-2.0
License-File: LICENSE.txt
Keywords: etl
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries
Classifier: Typing :: Typed
Requires-Python: <3.15,>=3.10
Requires-Dist: click>=7.1
Requires-Dist: fsspec>=2022.4.0
Requires-Dist: gitpython>=3.1.29
Requires-Dist: giturlparse>=0.10.0
Requires-Dist: humanize>=4.4.0
Requires-Dist: jsonpath-ng>=1.8.0
Requires-Dist: orjson!=3.10.1,!=3.9.11,!=3.9.12,!=3.9.13,!=3.9.14,<4,>=3.6.7; platform_python_implementation != 'PyPy' and sys_platform != 'emscripten'
Requires-Dist: orjson>=3.10.1; platform_python_implementation != 'PyPy' and sys_platform != 'emscripten'
Requires-Dist: orjson>=3.11.0; python_version > '3.13'
Requires-Dist: packaging>=21.1
Requires-Dist: pathvalidate>=2.5.2
Requires-Dist: pendulum>=2.1.2
Requires-Dist: pendulum>=3.0.0; python_version > '3.13'
Requires-Dist: pluggy>=1.3.0
Requires-Dist: pytz>=2022.6
Requires-Dist: pywin32>=306; sys_platform == 'win32'
Requires-Dist: pyyaml>=5.4.1
Requires-Dist: requests>=2.26.0
Requires-Dist: requirements-parser>=0.5.0
Requires-Dist: rich-argparse>=1.6.0
Requires-Dist: semver>=3.0.0
Requires-Dist: setuptools>=65.6.0
Requires-Dist: simplejson>=3.17.5
Requires-Dist: sqlglot>=25.4.0
Requires-Dist: tenacity>=8.0.2
Requires-Dist: tomlkit>=0.11.3
Requires-Dist: typing-extensions>=4.8.0
Requires-Dist: tzdata>=2022.1
Requires-Dist: win-precise-time>=1.4.2; os_name == 'nt' and python_version < '3.13'
Provides-Extra: athena
Requires-Dist: botocore>=1.28; extra == 'athena'
Requires-Dist: pyarrow>=16.0.0; extra == 'athena'
Requires-Dist: pyathena>=2.9.6; extra == 'athena'
Requires-Dist: s3fs>=2022.4.0; extra == 'athena'
Provides-Extra: az
Requires-Dist: adlfs>=2024.7.0; extra == 'az'
Provides-Extra: azure-key-vault
Requires-Dist: azure-core>=1.31.0; extra == 'azure-key-vault'
Requires-Dist: azure-identity>=1.12.0; extra == 'azure-key-vault'
Requires-Dist: azure-keyvault-secrets>=4.7.0; extra == 'azure-key-vault'
Provides-Extra: bigquery
Requires-Dist: db-dtypes>=1.2.0; extra == 'bigquery'
Requires-Dist: gcsfs>=2022.4.0; extra == 'bigquery'
Requires-Dist: google-cloud-bigquery>=2.26.0; extra == 'bigquery'
Requires-Dist: grpcio>=1.50.0; extra == 'bigquery'
Requires-Dist: pyarrow>=16.0.0; extra == 'bigquery'
Provides-Extra: cli
Requires-Dist: cron-descriptor>=1.2.32; extra == 'cli'
Requires-Dist: pip>=23.0.0; extra == 'cli'
Requires-Dist: pipdeptree<2.10,>=2.9.3; extra == 'cli'
Provides-Extra: clickhouse
Requires-Dist: adlfs>=2024.7.0; extra == 'clickhouse'
Requires-Dist: clickhouse-connect>=0.7.7; extra == 'clickhouse'
Requires-Dist: clickhouse-driver>=0.2.7; extra == 'clickhouse'
Requires-Dist: gcsfs>=2022.4.0; extra == 'clickhouse'
Requires-Dist: pyarrow>=16.0.0; extra == 'clickhouse'
Requires-Dist: s3fs>=2022.4.0; extra == 'clickhouse'
Provides-Extra: databricks
Requires-Dist: databricks-sdk>=0.38.0; extra == 'databricks'
Requires-Dist: databricks-sql-connector>=2.9.3; (python_version <= '3.12') and extra == 'databricks'
Requires-Dist: databricks-sql-connector>=3.6.0; (python_version >= '3.13') and extra == 'databricks'
Requires-Dist: databricks-zerobus-ingest-sdk>=1.2.0; ((platform_system == 'Linux' and (platform_machine == 'x86_64' or platform_machine == 'aarch64')) or (platform_system == 'Windows' and (platform_machine == 'AMD64' or platform_machine == 'x86_64'))) and extra == 'databricks'
Provides-Extra: dbml
Requires-Dist: pydbml; extra == 'dbml'
Provides-Extra: deltalake
Requires-Dist: deltalake>=0.25.1; extra == 'deltalake'
Requires-Dist: pyarrow>=16.0.0; extra == 'deltalake'
Provides-Extra: dremio
Requires-Dist: pyarrow>=16.0.0; extra == 'dremio'
Provides-Extra: duckdb
Requires-Dist: duckdb>=0.9; extra == 'duckdb'
Provides-Extra: ducklake
Requires-Dist: duckdb>=1.2.0; extra == 'ducklake'
Requires-Dist: pyarrow>=16.0.0; extra == 'ducklake'
Provides-Extra: fabric
Requires-Dist: adlfs>=2024.7.0; extra == 'fabric'
Requires-Dist: mssql-python>=1.8.0; extra == 'fabric'
Requires-Dist: pyarrow>=16.0.0; extra == 'fabric'
Provides-Extra: filesystem
Requires-Dist: botocore>=1.28; extra == 'filesystem'
Requires-Dist: s3fs>=2022.4.0; extra == 'filesystem'
Provides-Extra: gcp
Requires-Dist: db-dtypes>=1.2.0; extra == 'gcp'
Requires-Dist: gcsfs>=2022.4.0; extra == 'gcp'
Requires-Dist: google-cloud-bigquery>=2.26.0; extra == 'gcp'
Requires-Dist: grpcio>=1.50.0; extra == 'gcp'
Provides-Extra: gs
Requires-Dist: gcsfs>=2022.4.0; extra == 'gs'
Provides-Extra: hf
Requires-Dist: huggingface-hub>=1.4.1; extra == 'hf'
Requires-Dist: pyarrow>=21.0.0; extra == 'hf'
Provides-Extra: http
Requires-Dist: aiohttp>3.9.0; extra == 'http'
Provides-Extra: hub
Requires-Dist: dlthub-client>=0.28.1; extra == 'hub'
Requires-Dist: dlthub<0.30,>=0.29.0; extra == 'hub'
Provides-Extra: lance
Requires-Dist: duckdb>=1.4.3; extra == 'lance'
Requires-Dist: ibis-framework>=12.0.0; extra == 'lance'
Requires-Dist: lancedb>=0.33.0; extra == 'lance'
Requires-Dist: pyarrow>=16.0.0; extra == 'lance'
Requires-Dist: pylance<8,>=6.0.1; extra == 'lance'
Provides-Extra: lancedb
Requires-Dist: duckdb>=1.4.3; extra == 'lancedb'
Requires-Dist: ibis-framework>=12.0.0; extra == 'lancedb'
Requires-Dist: lancedb>=0.22.0; extra == 'lancedb'
Requires-Dist: pyarrow>=16.0.0; extra == 'lancedb'
Provides-Extra: motherduck
Requires-Dist: duckdb>=0.9; extra == 'motherduck'
Requires-Dist: pyarrow>=16.0.0; extra == 'motherduck'
Provides-Extra: mssql
Requires-Dist: mssql-python>=1.8.0; extra == 'mssql'
Provides-Extra: oracle
Requires-Dist: oracledb>=3.4.1; extra == 'oracle'
Provides-Extra: parquet
Requires-Dist: pyarrow>=16.0.0; extra == 'parquet'
Provides-Extra: polars
Requires-Dist: polars>=1.0.0; extra == 'polars'
Provides-Extra: postgis
Requires-Dist: psycopg2-binary>=2.9.1; extra == 'postgis'
Provides-Extra: postgres
Requires-Dist: psycopg2-binary>=2.9.1; extra == 'postgres'
Provides-Extra: pyiceberg
Requires-Dist: pyarrow>=16.0.0; extra == 'pyiceberg'
Requires-Dist: pyiceberg-core>=0.6.0; extra == 'pyiceberg'
Requires-Dist: pyiceberg>=0.9.1; extra == 'pyiceberg'
Requires-Dist: sqlalchemy>=1.4; extra == 'pyiceberg'
Provides-Extra: qdrant
Requires-Dist: qdrant-client>=1.8; (python_version > '3.13') and extra == 'qdrant'
Requires-Dist: qdrant-client[fastembed]>=1.8; (python_version <= '3.13') and extra == 'qdrant'
Provides-Extra: redshift
Requires-Dist: psycopg2-binary>=2.9.1; extra == 'redshift'
Provides-Extra: s3
Requires-Dist: botocore>=1.28; extra == 's3'
Requires-Dist: s3fs>=2022.4.0; extra == 's3'
Provides-Extra: sftp
Requires-Dist: paramiko<5,>=3.3.0; extra == 'sftp'
Provides-Extra: snowflake
Requires-Dist: snowflake-connector-python>=3.5.0; extra == 'snowflake'
Requires-Dist: snowflake-connector-python>=4.4.0; (python_version >= '3.14') and extra == 'snowflake'
Provides-Extra: sql-database
Requires-Dist: sqlalchemy>=1.4; extra == 'sql-database'
Provides-Extra: sqlalchemy
Requires-Dist: alembic>1.10.0; extra == 'sqlalchemy'
Requires-Dist: sqlalchemy>=1.4; extra == 'sqlalchemy'
Provides-Extra: synapse
Requires-Dist: adlfs>=2024.7.0; extra == 'synapse'
Requires-Dist: mssql-python>=1.8.0; extra == 'synapse'
Requires-Dist: pyarrow>=16.0.0; extra == 'synapse'
Provides-Extra: weaviate
Requires-Dist: weaviate-client<5.0.0,>=4.0.0; extra == 'weaviate'
Description-Content-Type: text/markdown

# dlt-fabric

`dlt-fabric` is a maintained fork of [dlt](https://github.com/dlt-hub/dlt) with fixes for the Microsoft Fabric Warehouse destination and the related MS SQL family of destinations (mssql, synapse).

dlt's Fabric, mssql, and synapse destinations have open issues around authentication and reliability that are not yet released upstream. This fork carries the fixes on top of each dlt release so they can be used today, while the changes work their way through upstream review.

## What this fork carries

This fork applies the following changes on top of the corresponding upstream dlt release:

- [dlt-hub/dlt#4147](https://github.com/dlt-hub/dlt/pull/4147): Microsoft Entra ID authentication for the mssql, synapse, and fabric destinations, an injectable pre-fetched `access_token` or externally constructed `azure_credential`, and `authentication = "fab_notebookutils"` for pipelines running inside a Fabric notebook. Supersedes the closed [#4140](https://github.com/dlt-hub/dlt/pull/4140).
- [dlt-hub/dlt#4141](https://github.com/dlt-hub/dlt/pull/4141): migration of the mssql, synapse, and fabric destinations from `pyodbc` to the `mssql-python` driver, which bundles its own client libraries so no system ODBC install is needed.
- [dlt-hub/dlt#4142](https://github.com/dlt-hub/dlt/pull/4142): a staging-optimized replace strategy for the Fabric destination, including a fix that makes concurrent multi-table-chain loads safe.
- [dlt-hub/dlt#4258](https://github.com/dlt-hub/dlt/pull/4258): a `mssql_arrow` backend for `sql_database` that uses `mssql-python`'s native Arrow C Data Interface for zero-copy batch extraction.
- [dlt-hub/dlt#4259](https://github.com/dlt-hub/dlt/pull/4259): correct nvarchar-to-varchar length scaling in the Fabric type mapper (character precision x 4 for UTF-8 byte semantics).
- [dlt-hub/dlt#4260](https://github.com/dlt-hub/dlt/pull/4260): allow `time` columns through the Parquet load path on Fabric (the inherited Synapse rejection does not apply).
- [dlt-hub/dlt#4261](https://github.com/dlt-hub/dlt/pull/4261): map SQL Server `MONEY` and `SMALLMONEY` types to `decimal(19,4)` and `decimal(10,4)` in `sql_database` schema inference.
- [dlt-hub/dlt#4275](https://github.com/dlt-hub/dlt/pull/4275): an Azure Key Vault configuration provider for loading secrets and config from Azure Key Vault, with `DefaultAzureCredential` fallback and a `dlt[azure_key_vault]` extra.
- [dlt-hub/dlt#4302](https://github.com/dlt-hub/dlt/pull/4302): route OneLake filesystem configurations to a dedicated client that strips trailing separators before directory probes, which OneLake answers with `403 AuthenticationFailed`.
- [dlt-hub/dlt#4307](https://github.com/dlt-hub/dlt/pull/4307): widen `tinyint` to `smallint` in the Fabric type mapper, since Fabric Warehouse has no `tinyint` and rejects the inherited SQL Server mapping.

These are proposed as pull requests against upstream dlt. Until they are merged and released, this fork is rebased onto each new dlt release to stay current.

## How the branches fit together

Everything is developed as small branches that each become one upstream pull request. Two of them
are stacked, because their content genuinely depends on another PR; the rest sit directly on
`devel`.

```
upstream/devel
├── feat/mssql-access-token-credential         #4147  Entra ID auth, access_token/azure_credential, fab_notebookutils
│   ├── feat/azure-key-vault-provider          #4275  AzureKeyVaultProvider (uses the NotebookUtils credential)
│   └── feat/mssql-python-driver               #4141  pyodbc -> mssql-python migration
│       └── feat/2-mssql-python-arrow-batches  #4258  mssql_arrow backend
├── feat/fabric-staging-optimized              #4142  staging-optimized replace via DDL transactions
├── fix/3-nvarchar-utf8-length                 #4259
├── fix/4-fabric-time-parquet                  #4260
├── fix/5-money-decimal-precision              #4261
├── fix/8-onelake-directory-probes             #4302
└── fix/4306-fabric-tinyint-smallint           #4307
```

`#4141` is stacked on `#4147` rather than branched from `devel` so the Entra ID authentication
exists once. When it was branched independently, both carried their own copy of the same two auth
commits, which meant every change had to be made twice and the integration branch inherited the
duplication.

GitHub cannot express this as a real stacked PR: a pull request's base must live in the upstream
repository, and these branches live in the fork. Each PR body therefore names the PR it depends on,
and the diff shown on GitHub includes the parent's commits until the parent lands.

## Rebuilding the integration and release branches

`integrated-all-prs` is every PR merged together, for testing the combination. It **starts from
`devel`**. Rebuild it with plain merges; the stacking means the two leaves carry everything below
them:

```sh
git checkout integrated-all-prs && git reset --hard devel
git merge --no-ff feat/2-mssql-python-arrow-batches   # brings #4147 and #4141 along
git merge --no-ff feat/azure-key-vault-provider       # shares #4147, so this merges cleanly
git merge --no-ff feat/fabric-staging-optimized
git merge --no-ff fix/3-nvarchar-utf8-length
git merge --no-ff fix/4-fabric-time-parquet
git merge --no-ff fix/5-money-decimal-precision
git merge --no-ff fix/8-onelake-directory-probes
git merge --no-ff fix/4306-fabric-tinyint-smallint
```

Conflicts here are almost always two branches appending tests to the same file; keep both sides.

`dlt-fabric` is the published branch and **starts from `upstream/master`**, the latest stable
release. Do not merge `integrated-all-prs` into it: that branch sits on `devel`, so merging it
would drag every unreleased `devel` commit into the release. Cherry-pick the feature commits
instead, which is exactly the set that is in the integration branch but not in `devel`:

```sh
git checkout dlt-fabric && git reset --hard upstream/master
git rev-list --reverse --no-merges devel..integrated-all-prs | while read c; do git cherry-pick "$c"; done
git cherry-pick <package as dlt-fabric> <resolve version and package name>
```

Verify afterwards that nothing from `devel` slipped in:

```sh
git rev-list upstream/master..dlt-fabric | while read c; do
  git merge-base --is-ancestor "$c" devel && git log -1 --oneline "$c"
done   # must print nothing
```

## Versioning and release

`pyproject.toml` sets `name = "dlt-fabric"` and the version of the upstream release this fork sits
on, with a `.postN` suffix per fork release. `dlt/version.py` resolves the distribution name so
`dlt.__version__` keeps working under the renamed package. Publish with `make publish-library`.

## Installation

`dlt-fabric` is a drop-in replacement for `dlt`. Install it instead of the upstream package:

```bash
pip install dlt-fabric
# or
uv add dlt-fabric
```

Then use it exactly as you would use `dlt`:

```python
import dlt
```

Both packages install the same `dlt` import path, so `dlt-fabric` cannot be installed alongside the upstream `dlt` package in the same environment.

## Documentation

This fork does not maintain separate documentation. For everything beyond the fixes listed above, the upstream resources apply directly:

- Documentation and usage: https://dlthub.com/docs
- Upstream project: https://github.com/dlt-hub/dlt
