v4.0

There have been important changes in the main function pangres.upsert:

Breaking changes

1. The first argument engine has been be renamed to con and now accepts engines and connections
2. The argument chunksize now defaults to None. Like in pandas.DataFrame.to_sql we will attempt to insert all rows by default. Previously the default was 10000 rows.
3. There will be no more automatic adjustments to the chunksize the user passes in pangres.upsert even if we can predict that it will raise an Exception due to database limitations.

E.g. inserting 100000 rows at once in a SQlite database with pangres will necessarily raise an Exception down the line because we need to pass NUMBER_OF_ROWS * NUMBER_OF_COLUMNS parameters and the maximum of parameters allowed in SQLite is 32766 (for version >= 3.32.0, otherwise it is 999).

I have made a new utility function pangres.adjust_chunksize that you can use before calling pangres.upsert if you want to make sure the chunksize is not too big.

New Features

Now that pangres.upsert accept connections objects this will give you more control when using it when it comes to connections and transactions.

See transaction control demo notebook.