fidesctl sync MVP
#764
Assignees
Comments
ThomasLaPiana
pushed a commit
that referenced
this issue
Aug 17, 2022
#764) Co-authored-by: Christopher Calhoun <[email protected]>
ThomasLaPiana
added a commit
that referenced
this issue
Aug 23, 2022
* Bump fideslib from 2.0.3 to 2.0.4 (#643) * Bump fideslib from 2.0.3 to 2.0.4 * Update CHANGELOG.md Co-authored-by: Paul Sanders <[email protected]> * Deploy UI within fidesops (#625) * Switch to Chakra image instead of nextjs * checking in progress towards static build * Mount static file directory * Update Dockerfile * Add dockerignore file * Fix file path * Update Dockerfile * Add admin ui config model * Conditionally build the UI * Fix env_prefix * Remove hello_world placeholder * Remove if logic from Dockerfile * Fix issues with BASE_URL * Refactor Head into component & update URNs * Fix import order * Update test URL path * Fix isort lint issue * Fix unit test failure * Update docs and changelog * Add ConnectionConfig Search [#609] (#641) * Add search on connectionconfig field that examines name, key, and description fields. - Add field ConnectionConfig.description - Allow ConnectionConfig.description to be updated via PATCH /connectionconfig * Add search to the guides and update changelog. * New description key returned in response. * Bump downrev. * Add ordering assertion to test. * Add `AdminUiSettings` to helper method (#647) * Add `AdminUiSettings` to helper method * format code * Update changelog * Add Prettier CI check and format UI code to be consistent (#655) * Run prettier on whole codebase * Add prettier CI check * Update format commands * Add prettier formatting to the privacy center * format admin ui tests too * Add privacy center format CI check * Update changelog * Align formatting with fidesctl codebase * Add Default Policies [#624] (#654) * WIP: Add a data migration that autogenerates a default 'download' and 'delete' policy. * Bump downrev and add logging for data migration steps. * Try importing models from db.base. * Use the connection currently being used to emit SQL to the database. * Skip adding default policies to test database. * Refactor so we're querying in SQL and not relying on SQLAlchemy models. * Add default policies to policy guides and clarify that the TESTING env variable is primarily set by pytest. * Update docs/fidesops/docs/guides/policies.md Co-authored-by: Cole Isaac <[email protected]> Co-authored-by: Cole Isaac <[email protected]> * Sort ConnectionConfig Name Ascending [#668] (#672) * Sort by connection name ascending. * Update changelog. * Install MSSQL Dependencies by Default [#635] (#664) * Have MSSQL dependencies be installed by default in the dockerfile by flipping the variable's meaning to SKIP_MSSQL_INSTALLATION. * Update changelog. * Add ConnectionConfig Filters (#675) * Add connectionconfig filtering on connection_type (can search multiple) and whether the connection is disabled. * Update changelog. * Ignore mypy error - we intentionally want the default value to be a Query so FastAPI interprets as a query param and not a request body per FastAPI docs. * Remove unused union. * Add connection config filtering on system_type and test_status. * No elif after return. * Add manual system type for connectionconfig filtering. * Adding support for default values for connector params (#661) * 559-endpoint-log-events - Adds middleware for calling analytics events for each endpoint (#622) * add formatting requirement to squlalchemy uri option (#685) * add formatting requirement to squlalchemy uri option * add port * [SaaS Connector] Zendesk - Ticket Endpoints (access) (#677) Co-authored-by: HamzaWaseemOnBench <[email protected]> * add number of masked records into masking endpoint log message (#692) * add number of masked records into masking endpoint log message * update changelog * update to use proper python convention for list lenght * tweak log message with optional plural Co-authored-by: Adam Sachs <[email protected]> * Datastore management landing page (#674) * Refactor routes into enums and create connections page * Test switching back to double quotes * Convert back to double quotes * Add placeholder connection filters * Set up api scaffolding * Get basic grid going * Initial grid card styling * Fix simple eslint issues * Add development config back in * Finish draft of card * Add working test button and landing page * Add pagination and small fixes * Fix testing issues * Add auth tests for datastore connection page * run formatter * Update changelog * update the create_test_data command to add connectionconfigs * Disable create buttons & fix text overflow * Update filter dropdown values * Fix test timestamp bug * Remove development variable * Fix imports * Update button hover color * remove commented out code * fix typo * Remove Saas Option Co-authored-by: Sean Preston <[email protected]> * Delete Datastore [Frontend] (#683) * Link Delete Connection menu item with confirmation modal, patterned off of user delete modal. * Update changelog. * Vertically center modals. * Add a spinner while collection is being deleted. Only close modal after successful deletion. * Prevent closing modal while we're still making the delete request. * 546 - [Admin UI] Change "Policy Name" to "Request Type" on SR list page (#696) * 546 - [Admin UI] Change "Policy Name" to "Request Type" on SR list page * Rollback file change * Updated CHANGELOG.md file Co-authored-by: Christopher Calhoun <[email protected]> * Enable/Disable Datastores [Frontend] (#693) * Add the ability to enable/disable a connectionconfig. * Fix other location to prevent from closing modal while in progress. * Update changelog. * 659 Add Postgres and Redis to health endpoint (#690) * adds db and redis to health endpoint, untested * updates typing, suppresses sqlalchemy and alembic logs for healthchecks * adds to makefile, updates docs and tests * format * adds check for cache enabled, update tests * format * standardize health endpoint structure * Execute Privacy Requests with Celery (#621) Updates the way privacy requests are dispatched into processing from a background process into a Celery task * 512 db redis health (#686) * catches exceptions from db/redis and provides better errors on startup * move health endpoint changes to separate branch * update changelog * catch appropriate redis err, stop app upon db and cache connection failure * adds check for cache enabled * Datastore Connection Filtering (#691) * Refactor routes into enums and create connections page * Test switching back to double quotes * Convert back to double quotes * Add placeholder connection filters * Set up api scaffolding * Get basic grid going * Initial grid card styling * Fix simple eslint issues * Add development config back in * Finish draft of card * Add working test button and landing page * Add pagination and small fixes * Fix testing issues * Add auth tests for datastore connection page * run formatter * Update changelog * update the create_test_data command to add connectionconfigs * Disable create buttons & fix text overflow * Update filter dropdown values * Fix test timestamp bug * Remove development variable * Add working filter dropdowns * Add outside click hook & polish things * Fix imports * Update changelog * Update button hover color * remove commented out code * fix typo * Remove Saas Option * Fix welcome screen bug * Remove edit button * Fix lint and formatting issues * removes commented-out code Co-authored-by: Sean Preston <[email protected]> Co-authored-by: eastandwestwind <[email protected]> * update domain -> host config references in stripe and sentry test infrastructure (#698) Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> * Clean up postman collection (#704) * Revoke a Pending Privacy Request [#525] (#592) * Add the ability to cancel a pending task. The celery task is not actually cancelled yet. - Track cancel reason, datetime cancelled, and add a new cancelled status. * Add drp revoke request to postman collection. * Add drp revoke docs. * Update down_rev after rebase. * Fix incorrect check. * Restore new canceled state. * Check that the privacy request is not canceled right before starting execution. This is really our last chance to check before we start executing the graph in dask. The use case here might be it was canceled shortly after it was approved. * Attempt to revoke a queued celery task if we cancel it before it starts executing. * Prettier. * Changelog updated. * Add a few unit tests around how triggering the run_privacy_request_task with a cancelled task id doesn't do anything and how you can't approve a canceled privacy request. * Fix SQLAlchemy logging to console - logging in migration propagates to the rest of the application. * Refresh session instead of creating a new one. * Add 200 character limit. * Add some assertions that db.refresh is doing what we think it's doing. * Update CHANGELOG.md, bump to `fideslib==2.1.0` (#705) * bump version of fideslib * updates changelog pre-release * Update datastore connection filters to support clearing (#701) * Update connection filters to clear * Use new common header function for analytics * Update prepareHeaders function to make type checker happy * Update changelog * Management UI updates (#702) * reorder ui docs * admi ui * subject request detail finalization, datastore stub * config edit * comment out docs not implemented * remove tracking * changelog * remove stray comment, add Canceled status * Remove stray hyphen (#709) * Reduce docker image size (#707) * Reduce docker image size * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Bump next-auth from 4.3.2 to 4.5.0 in /clients/privacy-center (#694) Bumps [next-auth](https://github.com/nextauthjs/next-auth) from 4.3.2 to 4.5.0. - [Release notes](https://github.com/nextauthjs/next-auth/releases) - [Changelog](https://github.com/nextauthjs/next-auth/blob/main/CHANGELOG.md) - [Commits](https://github.com/nextauthjs/next-auth/compare/[email protected]@v4.5.0) --- updated-dependencies: - dependency-name: next-auth dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update tutorial directions to install fidesdemo from the root directory (#715) * update tutorial directions to install fidesdemo from the root directory * changelog * [SaaS Connector] Salesforce (access) (#676) Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Hamza W <[email protected]> * Parallelize CI safe checks (#717) * Parallelize CI safe checks * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Make reading of environment variables case insensitive (#712) * Make reading of environment variables case insensitive * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Add dependabot (#718) * Add dependabot * Update CHANGELOG * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Bump requests-mock from 1.8.0 to 1.9.3 (#732) Bumps [requests-mock](https://github.com/jamielennox/requests-mock) from 1.8.0 to 1.9.3. - [Release notes](https://github.com/jamielennox/requests-mock/releases) - [Commits](https://github.com/jamielennox/requests-mock/compare/1.8.0...1.9.3) --- updated-dependencies: - dependency-name: requests-mock dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump packaging from 20.9 to 21.3 (#733) Bumps [packaging](https://github.com/pypa/packaging) from 20.9 to 21.3. - [Release notes](https://github.com/pypa/packaging/releases) - [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pypa/packaging/compare/20.9...21.3) --- updated-dependencies: - dependency-name: packaging dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump types-redis from 4.2.6 to 4.3.2 (#729) Bumps [types-redis](https://github.com/python/typeshed) from 4.2.6 to 4.3.2. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-redis dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump faker from 8.12.1 to 13.14.0 (#730) Bumps [faker](https://github.com/joke2k/faker) from 8.12.1 to 13.14.0. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](https://github.com/joke2k/faker/compare/v8.12.1...v13.14.0) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 2 to 4 (#724) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v2...v4) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 2 to 3 (#723) Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github/codeql-action from 1 to 2 (#725) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 1 to 2. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/codeql-action/compare/v1...v2) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump docker/login-action from 1 to 2 (#722) Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 2. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](https://github.com/docker/login-action/compare/v1...v2) --- updated-dependencies: - dependency-name: docker/login-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-node from 2 to 3 (#726) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 2 to 3. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/setup-node dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [#716] update datastore documentation (#742) * Adds `fidesops worker` command to start the Celery worker [#663] (#673) * add fidesops worker cmd * rename Dockerfiles, adds worker specific dockerfile with no frontend * formatting * update container names, add separate dockerfile for worker * pass worker args through to worker_main * rename containers fidesops -> webserver, celery -> worker * fix imports * add return type * update dockerfile * describe changelog * specify same argv for all celery instantiation * add logging to cli * add back .egg * specify dockerfile to build from * 708 - fix console warning in disable connections (#750) * removes unneeded code in disable connection * Bump fideslib from version 2.1.0 to 2.1.1 (#721) * Bump fideslib from version 2.1.0 to 2.1.1 * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Password Hashing update (#749) * Fix no such container error with docker-compose (#758) Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * 1.6.1 release (#763) * Update CHANGELOG for release * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * Bump psycopg2-binary from 2.9.1 to 2.9.3 (#783) Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.1 to 2.9.3. - [Release notes](https://github.com/psycopg/psycopg2/releases) - [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS) - [Commits](https://github.com/psycopg/psycopg2/commits) --- updated-dependencies: - dependency-name: psycopg2-binary dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2021.10.0 to 2022.6.1 (#781) Bumps [dask](https://github.com/dask/dask) from 2021.10.0 to 2022.6.1. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2021.10.0...2022.6.1) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump alembic from 1.6.5 to 1.8.0 (#780) Bumps [alembic](https://github.com/sqlalchemy/alembic) from 1.6.5 to 1.8.0. - [Release notes](https://github.com/sqlalchemy/alembic/releases) - [Changelog](https://github.com/sqlalchemy/alembic/blob/main/CHANGES) - [Commits](https://github.com/sqlalchemy/alembic/commits) --- updated-dependencies: - dependency-name: alembic dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump sqlalchemy-bigquery from 1.3.0 to 1.4.4 (#779) Bumps [sqlalchemy-bigquery](https://github.com/googleapis/python-bigquery-sqlalchemy) from 1.3.0 to 1.4.4. - [Release notes](https://github.com/googleapis/python-bigquery-sqlalchemy/releases) - [Changelog](https://github.com/googleapis/python-bigquery-sqlalchemy/blob/main/CHANGELOG.md) - [Commits](https://github.com/googleapis/python-bigquery-sqlalchemy/compare/v1.3.0...v1.4.4) --- updated-dependencies: - dependency-name: sqlalchemy-bigquery dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pytest-cov from 2.11.1 to 3.0.0 (#787) Bumps [pytest-cov](https://github.com/pytest-dev/pytest-cov) from 2.11.1 to 3.0.0. - [Release notes](https://github.com/pytest-dev/pytest-cov/releases) - [Changelog](https://github.com/pytest-dev/pytest-cov/blob/master/CHANGELOG.rst) - [Commits](https://github.com/pytest-dev/pytest-cov/compare/v2.11.1...v3.0.0) --- updated-dependencies: - dependency-name: pytest-cov dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump apscheduler from 3.8.0 to 3.9.1 (#789) Bumps [apscheduler](https://github.com/agronholm/apscheduler) from 3.8.0 to 3.9.1. - [Release notes](https://github.com/agronholm/apscheduler/releases) - [Changelog](https://github.com/agronholm/apscheduler/blob/3.9.1/docs/versionhistory.rst) - [Commits](https://github.com/agronholm/apscheduler/compare/3.8.0...3.9.1) --- updated-dependencies: - dependency-name: apscheduler dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump types-pyyaml from 6.0.8 to 6.0.9 (#791) Bumps [types-pyyaml](https://github.com/python/typeshed) from 6.0.8 to 6.0.9. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pyyaml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [#681] add documentation on fideslog use (#751) * [#681] add documentation on fideslog use * changelog * update fidesops use wording * Fideslib models (#700) * WIP * WIP * WIP * WIP * Use available exceptions from fideslib * Fix failing tests * Fix policy tests * Remove debugging code * Fix failing tests * Fix failing tests * Fix failiing tests * Run black and isort * Make pylint in docker happy * Clean up migrations * Move downgrade point of table renames * Remove Dockerfile temp workaround and fix pylint errors * Fix failing tests * Remove jwt.py Co-authored-by: Paul Sanders <[email protected]> * 660 Add support for multiple statuses to be selected for filtering subject requests (#802) Co-authored-by: Dawn Pattison <[email protected]> * Resolve issue with MyPy seeing files in fidesops as missing imports (#719) Co-authored-by: Paul Sanders <[email protected]> * 249 saas connector zendesk ticket erasure (#775) Co-authored-by: Hamza W <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> * Fixing `check-migration` command (#806) * Fix issue requiring separate install of snowflake-connector-python (#807) * Fix issue requiring separate install of snowflake-connector-python * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * 513 - [Admin UI] Update Subject Request status filter to be a multise… (#764) Co-authored-by: Christopher Calhoun <[email protected]> * 706 Adds SaaS connection type to SaaS yaml config (#748) * Adds Saas type to saas yaml config * To allow data migration that queries on a connectiontype enum to work, update previous schema migrations that used "alter type" to update connection type to rename the enum type, create a new enum with the new types, and then delete the old enum type. Co-authored-by: Dawn Pattison <[email protected]> * Make `worker` node optional (#770) * default fidesops to running the worker and webserver on same container * default to using a worker, add docker config for worker * update changelog * USE_DEDICATED_WORKER -> WORKER_ENABLED * add basic descriptions for celery vars to docs * remove unused import * add Make command for a server + worker * GET Available Connectors [#706] (#768) * Adds Saas type to saas yaml config * alter postman collection * updates changelog * lint fixes * Add endpoint to surface all available connectors including database options and saas options. * Exclude custom and manual types from list of available connectors. - Add docs and postman collection. * Update changelog. * Remove committed ANALYTICS_ID. * Import ClientDetail from fideslib instead of fidesops. * Fix import order. Co-authored-by: eastandwestwind <[email protected]> * Endpoint: Return Secrets for a Connector Type [#753] (#795) * Adds Saas type to saas yaml config * alter postman collection * updates changelog * lint fixes * Add endpoint to surface all available connectors including database options and saas options. * Exclude custom and manual types from list of available connectors. - Add docs and postman collection. * Update changelog. * Add an endpoint to fetch the types of secrets that should be supplied for a given connection type. - Relocate "load_config" which we use to load saas config yamls, now that we have another use case beyond unit tests. * Dynamically override the SaaSSchema docstring for a given saas connector type, so the description isn't abstract. - Update changelog - Add docs - Add endpoint to postman collection * Add missing import. * Add a request method to docs. * Update docstring. * Remove committed ANALYTICS_ID. * Import ClientDetail from fideslib instead of fidesops. * Fix import order. * Restore removed items in changelog. Co-authored-by: eastandwestwind <[email protected]> * Add fixture to clear tables between test (#680) * Add fixture to clear tables between test runs * Update CHANGELOG * Add missing ordering to customer_details logs query in test. * update import path Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Dawn Pattison <[email protected]> Co-authored-by: Sean Preston <[email protected]> * Replace user authentication routes with fideslib routes (#811) Co-authored-by: Paul Sanders <[email protected]> * test fixing publish_docs ci action (#818) * test fixing publish_docs ci action * update branch target to * try another commit to see if publish_docs is triggered * revert change to branch target * allow publish docs on test branch (#819) * allow publish docs on test branch * remove main * trying new empty commit * reverting to main * Temporarly disable paths * Revert temporary test * Add makefile * pull latest Co-authored-by: Paul Sanders <[email protected]> * Update Celery config defaults (#808) * update celery config defaults, set to redis settings if none provided * rename REDIS_CONNECTION_URL to CONNECTION_URL * add var to config reference * make DB index optional, use f'string * adds support for redis USER * add default for db_index in format string * set correct default for USER * Bump next-auth from 4.5.0 to 4.9.0 in /clients/privacy-center (#823) Bumps [next-auth](https://github.com/nextauthjs/next-auth) from 4.5.0 to 4.9.0. - [Release notes](https://github.com/nextauthjs/next-auth/releases) - [Changelog](https://github.com/nextauthjs/next-auth/blob/main/CHANGELOG.md) - [Commits](https://github.com/nextauthjs/next-auth/compare/[email protected]@v4.9.0) --- updated-dependencies: - dependency-name: next-auth dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fideslib to handle base64 encoded password (#820) * Handle hashed password * Update CHANGELOG * Fix failing test and use UserPasswordReset schema from fideslib * Restore fidesops.toml Co-authored-by: Paul Sanders <[email protected]> * update local deployment for serving the ui [#644] (#827) * update local deployment for serving the ui * changelog * Skip Masking of Uvicorn Logs [#766] (#831) * Update get_fides_log_record_factory to skip masking of all uvicorn logs. * Update changelog. * 832 - Create new user gives HTTP 422 Unprocessable Entity exception (#833) * 832 - [User Management] Create new user gives HTTP 422 Unprocessable Entity exception * Updated CHANGELOG.md file * [Admin UI] Change Login Page Wording (#774) * sign in to instead of sign into Changing the wording of the admin ui page to improve grammatical accuracy. * fides admin-ui wording changelog * added link to pr * fix typo on subject identities * updated changelog * typo * Fix bug in client with no scopes (#830) Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: eastandwestwind <[email protected]> * 599 clipboard icon (#838) * replace clipboard icon * temp fix for auth * revert last change * changelog * adds concurrency to unsafe check jobs (#835) * Reduce docker image size (#846) * Reduce docker image size * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * [#743] Store provided identity data in application database (#834) * adds identity fields to PrivacyRequest model * store identity data inside database * update changelog * add identities in test data command * store identities provided via the DRP creation endpoint * black + isort * store provided identity data in request creation from onetrust * remove deprecated migration * adds new provided identity table * use new provided identity table * add docstring, remove comment * update DRP privacy request creation to use ProvidedIdentity model * update identity creation in test data command * use persisted identity in OneTrust * update test to use persisted identity * isort update * use enums * optionally receive a salt in hash_value cmd * use a constant salt for provided identity hashing * remove import * use typehints * update typedef * use enum in dict * Bump faker from 13.14.0 to 13.15.0 (#848) Bumps [faker](https://github.com/joke2k/faker) from 13.14.0 to 13.15.0. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](https://github.com/joke2k/faker/compare/v13.14.0...v13.15.0) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump unidecode from 1.2.0 to 1.3.4 (#849) Bumps [unidecode](https://github.com/kmike/text-unidecode) from 1.2.0 to 1.3.4. - [Release notes](https://github.com/kmike/text-unidecode/releases) - [Commits](https://github.com/kmike/text-unidecode/commits) --- updated-dependencies: - dependency-name: unidecode dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update fastapi-pagination[sqlalchemy] requirement (#852) Updates the requirements on [fastapi-pagination[sqlalchemy]](https://github.com/uriyyo/fastapi-pagination) to permit the latest version. - [Release notes](https://github.com/uriyyo/fastapi-pagination/releases) - [Commits](https://github.com/uriyyo/fastapi-pagination/compare/0.8.3...0.9.3) --- updated-dependencies: - dependency-name: fastapi-pagination[sqlalchemy] dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump black from 22.3.0 to 22.6.0 (#855) Bumps [black](https://github.com/psf/black) from 22.3.0 to 22.6.0. - [Release notes](https://github.com/psf/black/releases) - [Changelog](https://github.com/psf/black/blob/main/CHANGES.md) - [Commits](https://github.com/psf/black/compare/22.3.0...22.6.0) --- updated-dependencies: - dependency-name: black dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Updating SaaSType enum (#857) * Serve UI from root (#720) * Refactor user management UI code (#839) * Refactor pages to use the same Layout * Add UserForm.tsx * Add Yup validation schema * Integrate refactored form to New route * Styling * WIP refactor EditUserForm * Fix EditUserForm typescript errors * Refactor handleSubmit * Small fixes to EditUserForm * Make call to get scopes earlier so form will be populated on first render * Update changelog * Search for `PrivacyRequest` based on hashed identity [#765] (#847) * adds identity fields to PrivacyRequest model * store identity data inside database * update changelog * add identities in test data command * store identities provided via the DRP creation endpoint * black + isort * store provided identity data in request creation from onetrust * remove deprecated migration * adds new provided identity table * use new provided identity table * add docstring, remove comment * update DRP privacy request creation to use ProvidedIdentity model * update identity creation in test data command * use persisted identity in OneTrust * update test to use persisted identity * isort update * use enums * optionally receive a salt in hash_value cmd * use a constant salt for provided identity hashing * remove import * use typehints * update typedef * use enum in dict * test for exact match search * added exact match search to request status api * import order * update CHANGELOG * documentation reorganization and page standardization (#858) * copy changes to fix docs outlining [#427] and standardize formatting [DOX-216] * Fix `create_test_data` (#862) * Fix create_test_data * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * Bump pre-commit from 2.9.3 to 2.20.0 (#853) Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 2.9.3 to 2.20.0. - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](https://github.com/pre-commit/pre-commit/compare/v2.9.3...v2.20.0) --- updated-dependencies: - dependency-name: pre-commit dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Return persisted identities in `get_request_status` view (#860) * 671 - [Datastore Management] Include Icon with datastore connection (FE) (#845) * 671 - [Datastore Management] Include Icon with datastore connection (FE) * 671 - [Datastore Management] Include Icon with datastore connection (FE) * Updated CHANGELOG.md file * 671 - [Datastore Management] Include Icon with datastore connection (FE) Resolved import statement issues * 671 - [Datastore Management] Include Icon with datastore connection (FE) Resolved import statement issues * 671 - [Datastore Management] Include Icon with datastore connection (FE) Resolved import statement issues * 671 - [Datastore Management] Include Icon with datastore connection (FE) Resolved import statement issues * 671 - [Datastore Management] Include Icon with datastore connection (FE) 1. Updated connector svg images 2. Added default style to Spinner component * 671 - [Datastore Management] Include Icon with datastore connection (FE) Deleted unnecessary file * 671 - [Datastore Management] Include Icon with datastore connection (FE) Updated MySQL and Outreach .svg files * Renamed connectors folder to connector-logos * Resolved misspelling error in import statement * Code review feedback * Code review feedback * Populate dataset (#844) * Correct test name for mypy in safe_pr_checks.yml (#875) Co-authored-by: Paul Sanders <[email protected]> * Adds `celery.toml` for loading custom Celery config [#821] (#865) * adds option to configure EVENT_QUEUE_PREFIX for celery * provide the option to specify a default queue name too * update celery config to load in from its own config toml file * updates changelog * update value for event_queue_prefix * test celery config overrides * include config_path arg * add type def * add config path to execution settings * correct values * add celery configuration to docs (#872) * update config reference * additional documentation on celery configs [#755] * add celery.toml reference * define optional celery config and overrides * reword config links * link to lowercase settings * v1.6.2 Release Checklist (#881) * updates changelog * update ERD * use star * splits IMAGE_NAME into COMPOSE_SERVICE_NAME for docker compose services as naming has diverged (#884) * belated changelog push (#885) * Move root-level docker files into docker/ subdir (#877) * Move root-level docker files into docker/ subdir * move all of the compose files * remove root aux compose files, refactor away the no-db and worker compose files * unify the app and worker dockerfiles into a single file that leverages build stages * move python scripts into a subdir, fix paths in compose integration files * fix the script tests * use python sleep instead of system sleep, fix script paths * remove the analytics_id that accicentally got committed * updated changelog * move the sample sql data to a subdir of docker/ so it can be mounted * update the teardown command and fix the integration files * fix more path typos * more desperate tweaks * fix mysql/mariadb/mongo tests * added an additional build step if mssql not there, all tests passing * fix an accidental lowercasing * Apply suggestions from code review Co-authored-by: Paul Sanders <[email protected]> * Update CHANGELOG.md Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern (#873) * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern Refactored the Datastore Management column border layout design to match Figma. * Updated CHANGELOG.md file * 45 - [Datastore Management] Keep datastore cards in 1/3 screen pattern * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern * Fixed import statement * 793 upgrade password hash (#876) * Update config.py * Update crypto imports to fideslib and remove legacy file and tests * Update Identity salt * checkpoint, lowercasing all of the things * update more config values to lowercase * fix linting errors, fix config validators * update the configs to use lowercase keys * lowercase the allowed keys * Update create_test_data.py * bump fideslib version * Lowercase config variables * Fix linting issues * Fix some test failures * Fix application fixtures * Remove old celery config options * Sort script imports * Update changelog Co-authored-by: Thomas <[email protected]> * Feat: Vault for secrets (#869) * Replace config/gh secrets with secrets from vault * sorting and update changelog * fix path for importing test helpers * more sorting * run ci again with empty commit * update unsafe_pr_checks * allow for no vault vars for unit tests, adds back back for saas config toml vars * check for client * sort * init client to None * fix imports * fix pytest markers to better indicate which tests rely on actual secrets * Bump pandas from 1.3.3 to 1.4.3 (#896) Bumps [pandas](https://github.com/pandas-dev/pandas) from 1.3.3 to 1.4.3. - [Release notes](https://github.com/pandas-dev/pandas/releases) - [Changelog](https://github.com/pandas-dev/pandas/blob/main/RELEASE.md) - [Commits](https://github.com/pandas-dev/pandas/compare/v1.3.3...v1.4.3) --- updated-dependencies: - dependency-name: pandas dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add dependabot label to dependabot PRs (#898) Co-authored-by: Paul Sanders <[email protected]> * [SaaS Connector] Salesforce (erasure) (#888) * 252 saas connector sendgrid (#883) * initial sendgrid saas connector integration. access only, contacts only * erasure (update) support for sendgrid * remove unused imports * update sendgrid test fixture to expect 404 response status code because of ignore_errors enhancement * Fixing import order and cleaning up the retry logic for consistency * add sendgrid env var support to makefile and unsafe_pr_checks config. remove DELETE endpoint per PR comments * Added delete endpoint for contacts * Fixing data_path for contacts endpoint * Reverting search query to improve performance and avoid server timeouts * Updated delete endpoint request, used request instead of SaaSRequest in tests * updated imports after check suggestion * Updated code after review * Removed unused variables, imports * Restoring Makefile * Fixed import cryptographic_util error * Misc fixes * Updated Changelog file * Updated Changelog for unreleased section and pulled main * Updated Changelog and added Sendgrid in unreleased section * Updated Changelog and added Sendgrid in unreleased section with link * Updated Changelog and added Sendgrid in added section after Adam's suggestion Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> Co-authored-by: Hamza W <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> * 747 - Users should be able to click on the full field of a dropdown-type filter to open up the dropdown (#903) * 747 - Users should be able to click on the full field of a dropdown-type filter to open up the dropdown * Disabled eslint import/extensions rule for certain files with an alias path in the import statement * Updated CHANGELOG.md file * Resolved jest unit tests from failing * Removed eslint-disable import/extensions declarations * Bump types-redis from 4.3.2 to 4.3.4 (#895) Bumps [types-redis](https://github.com/python/typeshed) from 4.3.2 to 4.3.4. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-redis dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Adds saas config base info to connection config responses (#904) * adds saas config base info to connection config responses * update changelog * add properties to base class * use diff naming to avoid recursion * update saas_config prop for unit tests * another occurance of saas config in unit test * use pydantic alias for type field * sort * revert change to saas type naming * unused import * init parent class so that props are avail on self * revert to using props * Adding privacy_request_id placeholder (#911) * Bump mypy from 0.961 to 0.971 (#914) Bumps [mypy](https://github.com/python/mypy) from 0.961 to 0.971. - [Release notes](https://github.com/python/mypy/releases) - [Commits](https://github.com/python/mypy/compare/v0.961...v0.971) --- updated-dependencies: - dependency-name: mypy dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pydash from 5.0.2 to 5.1.0 (#920) Bumps [pydash](https://github.com/dgilland/pydash) from 5.0.2 to 5.1.0. - [Release notes](https://github.com/dgilland/pydash/releases) - [Changelog](https://github.com/dgilland/pydash/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/dgilland/pydash/compare/v5.0.2...v5.1.0) --- updated-dependencies: - dependency-name: pydash dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update boto3 requirement from ~=1.18.14 to ~=1.24.34 (#917) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.18.14...1.24.34) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2022.6.1 to 2022.7.0 (#915) Bumps [dask](https://github.com/dask/dask) from 2022.6.1 to 2022.7.0. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2022.6.1...2022.7.0) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix integration set up scripts for postgres and mariadb - casing has changed for config variables. (#921) * zendesk and salesforce connection docs (#908) * Adobe Campaign access and erasure (#905) * Updated tutorial to match latest fidesdemo (#772) * Correct build arg variable name (#925) * Correct build arg variable name * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Default `FIDESOPS__ADMIN_UI__ENABLED` to `True` (#936) * serve AdminUI by default * updates changelog * Update python docker base image from slim-buster to slim-bullseye (#928) * Update python docker base image from slim-buster to slim-bullseye * Update CHANGELOG * Remove ipython from dev-requirements.txt Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * Update boto3 requirement from ~=1.24.34 to ~=1.24.36 (#939) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.24.34...1.24.36) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump faker from 13.15.0 to 13.15.1 (#941) Bumps [faker](https://github.com/joke2k/faker) from 13.15.0 to 13.15.1. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](https://github.com/joke2k/faker/compare/v13.15.0...v13.15.1) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump types-ujson from 5.2.0 to 5.4.0 (#947) Bumps [types-ujson](https://github.com/python/typeshed) from 5.2.0 to 5.4.0. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-ujson dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add db vs saas to connection type api (#937) * add db vs saas to connection type api * adds changelog line * mypy * fix test * format * more tests * formatting * adds system type query param * format * adjust test saas search * correct query param * Use Nox as the build tool instead of Make (#919) * Create noxfile.py * update the dockerfile with more stages * add GitPython as a dev requirement (used by nox builds) * add the noxfiles, all docker builds work * fix the "make compose-build" command * remove the worker docker stage due to it being redundant with prod * create a unified compose file for integrations * update the password prompt to be accurate * copy/pasta run_infrastructure into the noxfiles dir so it can be called directly via python * add create_user and seed_test_data to the nox utils * docs commands work * get the generic dev command working * add db commands to utils * clean up mypy configuration in pyproject.toml and remove config from setup.cfg * simplify some of the CI targets and start updating the pytest targets * cleanup run_infrastructure * update ci_suite and other CI nox targets * add nox as a dev-requirement * get dev commands working, sans quickstart * tweak to the compose_down constant * get the new pylint target passing * remove analytics_id * updated the changelog * Apply suggestions from code review Co-authored-by: Paul Sanders <[email protected]> * fix the xenon command * add a few tweaks to the worker dev command * fix an import issue * update dockerignore * move mssql to the bottom of the datastore list Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * 863 - Retry a DSR (FE) (#938) * Update docs docker base image from slim-buster to slim-bullseye (#949) * Changed Debian base image * Update the python version to be the same as used in the fidesops app image Co-authored-by: Paul Sanders <[email protected]> * Updated changelog Co-authored-by: Dave Quinlan <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * Experimenting with fixes for the failing MSSQL CI tests (#918) * Change docker password environment variable to MSSQL_SA_PASSWORD * Revert password environment variable name * Set MSSQL user to root in docker-compose * Revert setting root user in docker-compose * Change environment variable name from REQUIRE_MSSQL to SKIP_MSSQL_INSTALLATION * Add healthcheck to mssql compose file * Modify healthcheck command * Revert healthcheck * Try for more loging information * Another try for logging * Another try for logging * Try running only mssql tests to avoid timeout * Revert mssql only flag * Extend time out to try to get logs * Revert extra logging and extended timeout * Set network mode to host * Make mssql run on its own * Remove network from docker-compose and only run mssql in integration * Increase integration test logging * Revert mssql only * Use cache for docker * Fix workflow error * Fix workflow error * Fix target * Fix make traget * Fix make traget * Revert cache * Verify that mssql is running from pytest fixture * Add restart to mssql * Revert rester in docker-compose * Revert wait for mssql in pytest fixture * Wait for mssql to be ready before adding test data * Add count of retries * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Delete custom GitHub issue templates (#955) We've defined generic issue templates for all Ethyca repos here: https://github.com/ethyca/.github/tree/main/.github/ISSUE_TEMPLATE Removing the templates from this repo allows it to automatically pull in the organization templates for consistency. * Bump sqlalchemy-redshift from 0.8.8 to 0.8.10 (#940) Bumps [sqlalchemy-redshift](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift) from 0.8.8 to 0.8.10. - [Release notes](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/releases) - [Changelog](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/blob/main/CHANGES.rst) - [Commits](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/compare/0.8.8...0.8.10) --- updated-dependencies: - dependency-name: sqlalchemy-redshift dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Move tests into an "ops" subdir (#935) * Create __init__.py * move all of the test files down one dir into `ops` * update test paths * update paths where needed * update the changelog * Dispatch a repository event on new published releases (#945) * Add release dispatch event action * Update `CHANGELOG.md` * Reduce # of clients connected to the application db [#810] (#944) * Reduce number of open connections: - Limit task concurrency to two per worker. - Create one Engine per celery process which opens up a connection pool. Create one Session per celery process and use that session across privacy requests. - Close the session after the privacy request has finished executing. This just resets the session and returns connections back to the pool. It can be reused. - Remove unnecessary places where session is closed manually because the session is being used as a context manager and is already closed through that. - Pass the same Session that the privacy request is using through to TaskResources to be re-used to create ExecutionLogs instead of opening up a new Session. - Don't close the session when passing it into the Execution Log, wait until the entire privacy request is complete/exited. * Define "self" for run_privacy_task - it's the task itself. For mypy's benefits, define that the session is a context manager. * Make a session non-optional for graph_task.run_access_request, graph_task.run_erasure, and for instantiating taskResources * Use missing db fixture. * Add missing db resource. * Update test to reflect new behavior that disabling a datasource while a request is in progress can cause related collections to be skipped once the current session is expired and the connection config has the most recent state. Because the same Session that is being used to run the PrivacyRequest is now being used for ExecutionLogs, the process of saving an ExecutionLog runs a session.commit() which expires the Session and causes the ConnectionConfig to have the most recent state the next time it is accessed. * Update CHANGELOG. * enable worker by default in our dockerfile (#958) * add extra steps to make clean (#767) * Push `dev` image on pushes to `main` (#956) * Update publish_to_dockerhub.yml * add a dev step and use nox * update the changelog * Move Client Code into an `ops` subdir (#964) * Move Client Code into an `ops` subdir * move all of the files * update the dockerfile * update package.json * update codepaths for workflow tests * Update the changelog and docs references * Update .github/dependabot.yaml * Bump gitpython from 3.1 to 3.1.27 (#971) Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1 to 3.1.27. - [Release notes](https://github.com/gitpython-developers/GitPython/releases) - [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES) - [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.0...3.1.27) --- updated-dependencies: - dependency-name: gitpython dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix the `nox -s dev` command not spinning up the webserver (#959) * Update dev_nox.py * spin up the app before running a container shell * [#927, #929, #930] sendgrid, adobe, outreach connector docs (#951) * Bump sqlalchemy-utils from 0.37.8 to 0.38.3 (#968) Bumps [sqlalchemy-utils](https://github.com/kvesteri/sqlalchemy-utils) from 0.37.8 to 0.38.3. - [Release notes](https://github.com/kvesteri/sqlalchemy-utils/releases) - [Changelog](https://github.com/kvesteri/sqlalchemy-utils/blob/master/CHANGES.rst) - [Commits](https://github.com/kvesteri/sqlalchemy-utils/compare/0.37.8...0.38.3) --- updated-dependencies: - dependency-name: sqlalchemy-utils dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pyodbc from 4.0.32 to 4.0.34 (#980) Bumps [pyodbc](https://github.com/mkleehammer/pyodbc) from 4.0.32 to 4.0.34. - [Release notes](https://github.com/mkleehammer/pyodbc/releases) - [Commits](https://github.com/mkleehammer/pyodbc/compare/4.0.32...4.0.34) --- updated-dependencies: - dependency-name: pyodbc dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2022.7.0 to 2022.7.1 (#967) Bumps [dask](https://github.com/dask/dask) from 2022.7.0 to 2022.7.1. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2022.7.0...2022.7.1) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump alembic from 1.8.0 to 1.8.1 (#989) Bumps [alembic](https://github.com/sqlalchemy/alembic) from 1.8.0 to 1.8.1. - [Release notes](https://github.com/sqlalchemy/alembic/releases) - [Changelog](https://github.com/sqlalchemy/alembic/blob/main/CHANGES) - [Commits](https://github.com/sqlalchemy/alembic/commits) --- updated-dependencies: - dependency-name: alembic dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix erroneous values in the Outreach config (#988) * Add documentation for new nox commands (#981) * Add documentation for new nox commands * changelog * missed make to nox edits * review edits * update `nox -s dev` to not open a shell, add it as a posarg option * Apply suggestions from code review * remove the analytics id * update typo * remove extra numbering Co-authored-by: Thomas <[email protected]> * Bump types-toml from 0.10.7 to 0.10.8 (#998) Bumps [types-toml](https://github.com/python/typeshed) from 0.10.7 to 0.10.8. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-toml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fideslog from 1.2.1 to 1.2.2 (#996) Bumps [fideslog](https://github.com/ethyca/fideslog) from 1.2.1 to 1.2.2. - [Release notes](https://github.com/ethyca/fideslog/releases) - [Commits](https://github.com/ethyca/fideslog/compare/v1.2.1...v1.2.2) --- updated-dependencies: - dependency-name: fideslog dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Added Choose your connection feature (#987) * Added Choose your connection feature 760-Add a Connection - Select a connector to configure (front end) 866-Add a Connection - Front End layout structure * Updated CHANGELOG.md file * Fix lint issues * Fix build issue * Hide overflow Co-authored-by: Andrew Jackson <[email protected]> * saas request overrides (#986) * initial cut of saas request overrides. include mailchimp as an example and test case. minor refactor of some of the saas request execution to enable smoother override * fix rebase issue by moving saas override tests into ops subdir * import path updates to resolve conflicts caused by rebase * add session parameter into graph task calls to fix saas override integration tests caused by rebase * update changelog * tweaks to saas connector overrides and associated tests * expose override factory register as module variable for clenaer decorator calls Co-authored-by: Adam Sachs <[email protected]> * Update OAuth strategy to be able to perform local testing (#962) * Update boto3 requirement from ~=1.24.36 to ~=1.24.42 (#1001) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.24.36...1.24.42) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fastapi[all] from 0.78.0 to 0.79.0 (#1002) Bumps [fastapi[all]](https://github.com/tiangolo/fastapi) from 0.78.0 to 0.79.0. - [Release notes](https://github.com/tiangolo/fastapi/releases) - [Commits](https://github.com/tiangolo/fastapi/compare/0.78.0...0.79.0) --- updated-dependencies: - dependency-name: fastapi[all] dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update config reference to use lowercase values (#952) * [#909] update config reference to use lowercase values * changelog * missed uppercase variables * update usages of False to false in connector docs * Added Auth0 Connector (#991) * add pagination back to connection types endpoints (#1019) * add pagination back to connection types endpoints * check for pagination in system_type search Co-authored-by: Paul Sanders <[email protected]> * Subject Request Events and Logs Section (#1018) * Fix small issue with eslint config * Get initial drawer working * Add jest config to eslintignore * Fix small css issue * Refactor components an…
ThomasLaPiana
pushed a commit
that referenced
this issue
Sep 26, 2022
#764) Co-authored-by: Christopher Calhoun <[email protected]>
ThomasLaPiana
added a commit
that referenced
this issue
Sep 27, 2022
* Delete Datastore [Frontend] (#683) * Link Delete Connection menu item with confirmation modal, patterned off of user delete modal. * Update changelog. * Vertically center modals. * Add a spinner while collection is being deleted. Only close modal after successful deletion. * Prevent closing modal while we're still making the delete request. * 546 - [Admin UI] Change "Policy Name" to "Request Type" on SR list page (#696) * 546 - [Admin UI] Change "Policy Name" to "Request Type" on SR list page * Rollback file change * Updated CHANGELOG.md file Co-authored-by: Christopher Calhoun <[email protected]> * Enable/Disable Datastores [Frontend] (#693) * Add the ability to enable/disable a connectionconfig. * Fix other location to prevent from closing modal while in progress. * Update changelog. * 659 Add Postgres and Redis to health endpoint (#690) * adds db and redis to health endpoint, untested * updates typing, suppresses sqlalchemy and alembic logs for healthchecks * adds to makefile, updates docs and tests * format * adds check for cache enabled, update tests * format * standardize health endpoint structure * Execute Privacy Requests with Celery (#621) Updates the way privacy requests are dispatched into processing from a background process into a Celery task * 512 db redis health (#686) * catches exceptions from db/redis and provides better errors on startup * move health endpoint changes to separate branch * update changelog * catch appropriate redis err, stop app upon db and cache connection failure * adds check for cache enabled * Datastore Connection Filtering (#691) * Refactor routes into enums and create connections page * Test switching back to double quotes * Convert back to double quotes * Add placeholder connection filters * Set up api scaffolding * Get basic grid going * Initial grid card styling * Fix simple eslint issues * Add development config back in * Finish draft of card * Add working test button and landing page * Add pagination and small fixes * Fix testing issues * Add auth tests for datastore connection page * run formatter * Update changelog * update the create_test_data command to add connectionconfigs * Disable create buttons & fix text overflow * Update filter dropdown values * Fix test timestamp bug * Remove development variable * Add working filter dropdowns * Add outside click hook & polish things * Fix imports * Update changelog * Update button hover color * remove commented out code * fix typo * Remove Saas Option * Fix welcome screen bug * Remove edit button * Fix lint and formatting issues * removes commented-out code Co-authored-by: Sean Preston <[email protected]> Co-authored-by: eastandwestwind <[email protected]> * update domain -> host config references in stripe and sentry test infrastructure (#698) Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> * Clean up postman collection (#704) * Revoke a Pending Privacy Request [#525] (#592) * Add the ability to cancel a pending task. The celery task is not actually cancelled yet. - Track cancel reason, datetime cancelled, and add a new cancelled status. * Add drp revoke request to postman collection. * Add drp revoke docs. * Update down_rev after rebase. * Fix incorrect check. * Restore new canceled state. * Check that the privacy request is not canceled right before starting execution. This is really our last chance to check before we start executing the graph in dask. The use case here might be it was canceled shortly after it was approved. * Attempt to revoke a queued celery task if we cancel it before it starts executing. * Prettier. * Changelog updated. * Add a few unit tests around how triggering the run_privacy_request_task with a cancelled task id doesn't do anything and how you can't approve a canceled privacy request. * Fix SQLAlchemy logging to console - logging in migration propagates to the rest of the application. * Refresh session instead of creating a new one. * Add 200 character limit. * Add some assertions that db.refresh is doing what we think it's doing. * Update CHANGELOG.md, bump to `fideslib==2.1.0` (#705) * bump version of fideslib * updates changelog pre-release * Update datastore connection filters to support clearing (#701) * Update connection filters to clear * Use new common header function for analytics * Update prepareHeaders function to make type checker happy * Update changelog * Management UI updates (#702) * reorder ui docs * admi ui * subject request detail finalization, datastore stub * config edit * comment out docs not implemented * remove tracking * changelog * remove stray comment, add Canceled status * Remove stray hyphen (#709) * Reduce docker image size (#707) * Reduce docker image size * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Bump next-auth from 4.3.2 to 4.5.0 in /clients/privacy-center (#694) Bumps [next-auth](https://github.com/nextauthjs/next-auth) from 4.3.2 to 4.5.0. - [Release notes](https://github.com/nextauthjs/next-auth/releases) - [Changelog](https://github.com/nextauthjs/next-auth/blob/main/CHANGELOG.md) - [Commits](https://github.com/nextauthjs/next-auth/compare/[email protected]@v4.5.0) --- updated-dependencies: - dependency-name: next-auth dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update tutorial directions to install fidesdemo from the root directory (#715) * update tutorial directions to install fidesdemo from the root directory * changelog * [SaaS Connector] Salesforce (access) (#676) Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Hamza W <[email protected]> * Parallelize CI safe checks (#717) * Parallelize CI safe checks * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Make reading of environment variables case insensitive (#712) * Make reading of environment variables case insensitive * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Add dependabot (#718) * Add dependabot * Update CHANGELOG * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Bump requests-mock from 1.8.0 to 1.9.3 (#732) Bumps [requests-mock](https://github.com/jamielennox/requests-mock) from 1.8.0 to 1.9.3. - [Release notes](https://github.com/jamielennox/requests-mock/releases) - [Commits](https://github.com/jamielennox/requests-mock/compare/1.8.0...1.9.3) --- updated-dependencies: - dependency-name: requests-mock dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump packaging from 20.9 to 21.3 (#733) Bumps [packaging](https://github.com/pypa/packaging) from 20.9 to 21.3. - [Release notes](https://github.com/pypa/packaging/releases) - [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pypa/packaging/compare/20.9...21.3) --- updated-dependencies: - dependency-name: packaging dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump types-redis from 4.2.6 to 4.3.2 (#729) Bumps [types-redis](https://github.com/python/typeshed) from 4.2.6 to 4.3.2. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-redis dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump faker from 8.12.1 to 13.14.0 (#730) Bumps [faker](https://github.com/joke2k/faker) from 8.12.1 to 13.14.0. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](https://github.com/joke2k/faker/compare/v8.12.1...v13.14.0) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 2 to 4 (#724) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v2...v4) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 2 to 3 (#723) Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/checkout/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump github/codeql-action from 1 to 2 (#725) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 1 to 2. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/codeql-action/compare/v1...v2) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump docker/login-action from 1 to 2 (#722) Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 2. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](https://github.com/docker/login-action/compare/v1...v2) --- updated-dependencies: - dependency-name: docker/login-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-node from 2 to 3 (#726) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 2 to 3. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/v2...v3) --- updated-dependencies: - dependency-name: actions/setup-node dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [#716] update datastore documentation (#742) * Adds `fidesops worker` command to start the Celery worker [#663] (#673) * add fidesops worker cmd * rename Dockerfiles, adds worker specific dockerfile with no frontend * formatting * update container names, add separate dockerfile for worker * pass worker args through to worker_main * rename containers fidesops -> webserver, celery -> worker * fix imports * add return type * update dockerfile * describe changelog * specify same argv for all celery instantiation * add logging to cli * add back .egg * specify dockerfile to build from * 708 - fix console warning in disable connections (#750) * removes unneeded code in disable connection * Bump fideslib from version 2.1.0 to 2.1.1 (#721) * Bump fideslib from version 2.1.0 to 2.1.1 * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Password Hashing update (#749) * Fix no such container error with docker-compose (#758) Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * 1.6.1 release (#763) * Update CHANGELOG for release * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * Bump psycopg2-binary from 2.9.1 to 2.9.3 (#783) Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.1 to 2.9.3. - [Release notes](https://github.com/psycopg/psycopg2/releases) - [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS) - [Commits](https://github.com/psycopg/psycopg2/commits) --- updated-dependencies: - dependency-name: psycopg2-binary dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2021.10.0 to 2022.6.1 (#781) Bumps [dask](https://github.com/dask/dask) from 2021.10.0 to 2022.6.1. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2021.10.0...2022.6.1) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump alembic from 1.6.5 to 1.8.0 (#780) Bumps [alembic](https://github.com/sqlalchemy/alembic) from 1.6.5 to 1.8.0. - [Release notes](https://github.com/sqlalchemy/alembic/releases) - [Changelog](https://github.com/sqlalchemy/alembic/blob/main/CHANGES) - [Commits](https://github.com/sqlalchemy/alembic/commits) --- updated-dependencies: - dependency-name: alembic dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump sqlalchemy-bigquery from 1.3.0 to 1.4.4 (#779) Bumps [sqlalchemy-bigquery](https://github.com/googleapis/python-bigquery-sqlalchemy) from 1.3.0 to 1.4.4. - [Release notes](https://github.com/googleapis/python-bigquery-sqlalchemy/releases) - [Changelog](https://github.com/googleapis/python-bigquery-sqlalchemy/blob/main/CHANGELOG.md) - [Commits](https://github.com/googleapis/python-bigquery-sqlalchemy/compare/v1.3.0...v1.4.4) --- updated-dependencies: - dependency-name: sqlalchemy-bigquery dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pytest-cov from 2.11.1 to 3.0.0 (#787) Bumps [pytest-cov](https://github.com/pytest-dev/pytest-cov) from 2.11.1 to 3.0.0. - [Release notes](https://github.com/pytest-dev/pytest-cov/releases) - [Changelog](https://github.com/pytest-dev/pytest-cov/blob/master/CHANGELOG.rst) - [Commits](https://github.com/pytest-dev/pytest-cov/compare/v2.11.1...v3.0.0) --- updated-dependencies: - dependency-name: pytest-cov dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump apscheduler from 3.8.0 to 3.9.1 (#789) Bumps [apscheduler](https://github.com/agronholm/apscheduler) from 3.8.0 to 3.9.1. - [Release notes](https://github.com/agronholm/apscheduler/releases) - [Changelog](https://github.com/agronholm/apscheduler/blob/3.9.1/docs/versionhistory.rst) - [Commits](https://github.com/agronholm/apscheduler/compare/3.8.0...3.9.1) --- updated-dependencies: - dependency-name: apscheduler dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump types-pyyaml from 6.0.8 to 6.0.9 (#791) Bumps [types-pyyaml](https://github.com/python/typeshed) from 6.0.8 to 6.0.9. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pyyaml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [#681] add documentation on fideslog use (#751) * [#681] add documentation on fideslog use * changelog * update fidesops use wording * Fideslib models (#700) * WIP * WIP * WIP * WIP * Use available exceptions from fideslib * Fix failing tests * Fix policy tests * Remove debugging code * Fix failing tests * Fix failing tests * Fix failiing tests * Run black and isort * Make pylint in docker happy * Clean up migrations * Move downgrade point of table renames * Remove Dockerfile temp workaround and fix pylint errors * Fix failing tests * Remove jwt.py Co-authored-by: Paul Sanders <[email protected]> * 660 Add support for multiple statuses to be selected for filtering subject requests (#802) Co-authored-by: Dawn Pattison <[email protected]> * Resolve issue with MyPy seeing files in fidesops as missing imports (#719) Co-authored-by: Paul Sanders <[email protected]> * 249 saas connector zendesk ticket erasure (#775) Co-authored-by: Hamza W <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> * Fixing `check-migration` command (#806) * Fix issue requiring separate install of snowflake-connector-python (#807) * Fix issue requiring separate install of snowflake-connector-python * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * 513 - [Admin UI] Update Subject Request status filter to be a multise… (#764) Co-authored-by: Christopher Calhoun <[email protected]> * 706 Adds SaaS connection type to SaaS yaml config (#748) * Adds Saas type to saas yaml config * To allow data migration that queries on a connectiontype enum to work, update previous schema migrations that used "alter type" to update connection type to rename the enum type, create a new enum with the new types, and then delete the old enum type. Co-authored-by: Dawn Pattison <[email protected]> * Make `worker` node optional (#770) * default fidesops to running the worker and webserver on same container * default to using a worker, add docker config for worker * update changelog * USE_DEDICATED_WORKER -> WORKER_ENABLED * add basic descriptions for celery vars to docs * remove unused import * add Make command for a server + worker * GET Available Connectors [#706] (#768) * Adds Saas type to saas yaml config * alter postman collection * updates changelog * lint fixes * Add endpoint to surface all available connectors including database options and saas options. * Exclude custom and manual types from list of available connectors. - Add docs and postman collection. * Update changelog. * Remove committed ANALYTICS_ID. * Import ClientDetail from fideslib instead of fidesops. * Fix import order. Co-authored-by: eastandwestwind <[email protected]> * Endpoint: Return Secrets for a Connector Type [#753] (#795) * Adds Saas type to saas yaml config * alter postman collection * updates changelog * lint fixes * Add endpoint to surface all available connectors including database options and saas options. * Exclude custom and manual types from list of available connectors. - Add docs and postman collection. * Update changelog. * Add an endpoint to fetch the types of secrets that should be supplied for a given connection type. - Relocate "load_config" which we use to load saas config yamls, now that we have another use case beyond unit tests. * Dynamically override the SaaSSchema docstring for a given saas connector type, so the description isn't abstract. - Update changelog - Add docs - Add endpoint to postman collection * Add missing import. * Add a request method to docs. * Update docstring. * Remove committed ANALYTICS_ID. * Import ClientDetail from fideslib instead of fidesops. * Fix import order. * Restore removed items in changelog. Co-authored-by: eastandwestwind <[email protected]> * Add fixture to clear tables between test (#680) * Add fixture to clear tables between test runs * Update CHANGELOG * Add missing ordering to customer_details logs query in test. * update import path Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Dawn Pattison <[email protected]> Co-authored-by: Sean Preston <[email protected]> * Replace user authentication routes with fideslib routes (#811) Co-authored-by: Paul Sanders <[email protected]> * test fixing publish_docs ci action (#818) * test fixing publish_docs ci action * update branch target to * try another commit to see if publish_docs is triggered * revert change to branch target * allow publish docs on test branch (#819) * allow publish docs on test branch * remove main * trying new empty commit * reverting to main * Temporarly disable paths * Revert temporary test * Add makefile * pull latest Co-authored-by: Paul Sanders <[email protected]> * Update Celery config defaults (#808) * update celery config defaults, set to redis settings if none provided * rename REDIS_CONNECTION_URL to CONNECTION_URL * add var to config reference * make DB index optional, use f'string * adds support for redis USER * add default for db_index in format string * set correct default for USER * Bump next-auth from 4.5.0 to 4.9.0 in /clients/privacy-center (#823) Bumps [next-auth](https://github.com/nextauthjs/next-auth) from 4.5.0 to 4.9.0. - [Release notes](https://github.com/nextauthjs/next-auth/releases) - [Changelog](https://github.com/nextauthjs/next-auth/blob/main/CHANGELOG.md) - [Commits](https://github.com/nextauthjs/next-auth/compare/[email protected]@v4.9.0) --- updated-dependencies: - dependency-name: next-auth dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fideslib to handle base64 encoded password (#820) * Handle hashed password * Update CHANGELOG * Fix failing test and use UserPasswordReset schema from fideslib * Restore fidesops.toml Co-authored-by: Paul Sanders <[email protected]> * update local deployment for serving the ui [#644] (#827) * update local deployment for serving the ui * changelog * Skip Masking of Uvicorn Logs [#766] (#831) * Update get_fides_log_record_factory to skip masking of all uvicorn logs. * Update changelog. * 832 - Create new user gives HTTP 422 Unprocessable Entity exception (#833) * 832 - [User Management] Create new user gives HTTP 422 Unprocessable Entity exception * Updated CHANGELOG.md file * [Admin UI] Change Login Page Wording (#774) * sign in to instead of sign into Changing the wording of the admin ui page to improve grammatical accuracy. * fides admin-ui wording changelog * added link to pr * fix typo on subject identities * updated changelog * typo * Fix bug in client with no scopes (#830) Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: eastandwestwind <[email protected]> * 599 clipboard icon (#838) * replace clipboard icon * temp fix for auth * revert last change * changelog * adds concurrency to unsafe check jobs (#835) * Reduce docker image size (#846) * Reduce docker image size * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * [#743] Store provided identity data in application database (#834) * adds identity fields to PrivacyRequest model * store identity data inside database * update changelog * add identities in test data command * store identities provided via the DRP creation endpoint * black + isort * store provided identity data in request creation from onetrust * remove deprecated migration * adds new provided identity table * use new provided identity table * add docstring, remove comment * update DRP privacy request creation to use ProvidedIdentity model * update identity creation in test data command * use persisted identity in OneTrust * update test to use persisted identity * isort update * use enums * optionally receive a salt in hash_value cmd * use a constant salt for provided identity hashing * remove import * use typehints * update typedef * use enum in dict * Bump faker from 13.14.0 to 13.15.0 (#848) Bumps [faker](https://github.com/joke2k/faker) from 13.14.0 to 13.15.0. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](https://github.com/joke2k/faker/compare/v13.14.0...v13.15.0) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump unidecode from 1.2.0 to 1.3.4 (#849) Bumps [unidecode](https://github.com/kmike/text-unidecode) from 1.2.0 to 1.3.4. - [Release notes](https://github.com/kmike/text-unidecode/releases) - [Commits](https://github.com/kmike/text-unidecode/commits) --- updated-dependencies: - dependency-name: unidecode dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update fastapi-pagination[sqlalchemy] requirement (#852) Updates the requirements on [fastapi-pagination[sqlalchemy]](https://github.com/uriyyo/fastapi-pagination) to permit the latest version. - [Release notes](https://github.com/uriyyo/fastapi-pagination/releases) - [Commits](https://github.com/uriyyo/fastapi-pagination/compare/0.8.3...0.9.3) --- updated-dependencies: - dependency-name: fastapi-pagination[sqlalchemy] dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump black from 22.3.0 to 22.6.0 (#855) Bumps [black](https://github.com/psf/black) from 22.3.0 to 22.6.0. - [Release notes](https://github.com/psf/black/releases) - [Changelog](https://github.com/psf/black/blob/main/CHANGES.md) - [Commits](https://github.com/psf/black/compare/22.3.0...22.6.0) --- updated-dependencies: - dependency-name: black dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Updating SaaSType enum (#857) * Serve UI from root (#720) * Refactor user management UI code (#839) * Refactor pages to use the same Layout * Add UserForm.tsx * Add Yup validation schema * Integrate refactored form to New route * Styling * WIP refactor EditUserForm * Fix EditUserForm typescript errors * Refactor handleSubmit * Small fixes to EditUserForm * Make call to get scopes earlier so form will be populated on first render * Update changelog * Search for `PrivacyRequest` based on hashed identity [#765] (#847) * adds identity fields to PrivacyRequest model * store identity data inside database * update changelog * add identities in test data command * store identities provided via the DRP creation endpoint * black + isort * store provided identity data in request creation from onetrust * remove deprecated migration * adds new provided identity table * use new provided identity table * add docstring, remove comment * update DRP privacy request creation to use ProvidedIdentity model * update identity creation in test data command * use persisted identity in OneTrust * update test to use persisted identity * isort update * use enums * optionally receive a salt in hash_value cmd * use a constant salt for provided identity hashing * remove import * use typehints * update typedef * use enum in dict * test for exact match search * added exact match search to request status api * import order * update CHANGELOG * documentation reorganization and page standardization (#858) * copy changes to fix docs outlining [#427] and standardize formatting [DOX-216] * Fix `create_test_data` (#862) * Fix create_test_data * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * Bump pre-commit from 2.9.3 to 2.20.0 (#853) Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 2.9.3 to 2.20.0. - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](https://github.com/pre-commit/pre-commit/compare/v2.9.3...v2.20.0) --- updated-dependencies: - dependency-name: pre-commit dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Return persisted identities in `get_request_status` view (#860) * 671 - [Datastore Management] Include Icon with datastore connection (FE) (#845) * 671 - [Datastore Management] Include Icon with datastore connection (FE) * 671 - [Datastore Management] Include Icon with datastore connection (FE) * Updated CHANGELOG.md file * 671 - [Datastore Management] Include Icon with datastore connection (FE) Resolved import statement issues * 671 - [Datastore Management] Include Icon with datastore connection (FE) Resolved import statement issues * 671 - [Datastore Management] Include Icon with datastore connection (FE) Resolved import statement issues * 671 - [Datastore Management] Include Icon with datastore connection (FE) Resolved import statement issues * 671 - [Datastore Management] Include Icon with datastore connection (FE) 1. Updated connector svg images 2. Added default style to Spinner component * 671 - [Datastore Management] Include Icon with datastore connection (FE) Deleted unnecessary file * 671 - [Datastore Management] Include Icon with datastore connection (FE) Updated MySQL and Outreach .svg files * Renamed connectors folder to connector-logos * Resolved misspelling error in import statement * Code review feedback * Code review feedback * Populate dataset (#844) * Correct test name for mypy in safe_pr_checks.yml (#875) Co-authored-by: Paul Sanders <[email protected]> * Adds `celery.toml` for loading custom Celery config [#821] (#865) * adds option to configure EVENT_QUEUE_PREFIX for celery * provide the option to specify a default queue name too * update celery config to load in from its own config toml file * updates changelog * update value for event_queue_prefix * test celery config overrides * include config_path arg * add type def * add config path to execution settings * correct values * add celery configuration to docs (#872) * update config reference * additional documentation on celery configs [#755] * add celery.toml reference * define optional celery config and overrides * reword config links * link to lowercase settings * v1.6.2 Release Checklist (#881) * updates changelog * update ERD * use star * splits IMAGE_NAME into COMPOSE_SERVICE_NAME for docker compose services as naming has diverged (#884) * belated changelog push (#885) * Move root-level docker files into docker/ subdir (#877) * Move root-level docker files into docker/ subdir * move all of the compose files * remove root aux compose files, refactor away the no-db and worker compose files * unify the app and worker dockerfiles into a single file that leverages build stages * move python scripts into a subdir, fix paths in compose integration files * fix the script tests * use python sleep instead of system sleep, fix script paths * remove the analytics_id that accicentally got committed * updated changelog * move the sample sql data to a subdir of docker/ so it can be mounted * update the teardown command and fix the integration files * fix more path typos * more desperate tweaks * fix mysql/mariadb/mongo tests * added an additional build step if mssql not there, all tests passing * fix an accidental lowercasing * Apply suggestions from code review Co-authored-by: Paul Sanders <[email protected]> * Update CHANGELOG.md Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern (#873) * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern Refactored the Datastore Management column border layout design to match Figma. * Updated CHANGELOG.md file * 45 - [Datastore Management] Keep datastore cards in 1/3 screen pattern * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern * Fixed import statement * 793 upgrade password hash (#876) * Update config.py * Update crypto imports to fideslib and remove legacy file and tests * Update Identity salt * checkpoint, lowercasing all of the things * update more config values to lowercase * fix linting errors, fix config validators * update the configs to use lowercase keys * lowercase the allowed keys * Update create_test_data.py * bump fideslib version * Lowercase config variables * Fix linting issues * Fix some test failures * Fix application fixtures * Remove old celery config options * Sort script imports * Update changelog Co-authored-by: Thomas <[email protected]> * Feat: Vault for secrets (#869) * Replace config/gh secrets with secrets from vault * sorting and update changelog * fix path for importing test helpers * more sorting * run ci again with empty commit * update unsafe_pr_checks * allow for no vault vars for unit tests, adds back back for saas config toml vars * check for client * sort * init client to None * fix imports * fix pytest markers to better indicate which tests rely on actual secrets * Bump pandas from 1.3.3 to 1.4.3 (#896) Bumps [pandas](https://github.com/pandas-dev/pandas) from 1.3.3 to 1.4.3. - [Release notes](https://github.com/pandas-dev/pandas/releases) - [Changelog](https://github.com/pandas-dev/pandas/blob/main/RELEASE.md) - [Commits](https://github.com/pandas-dev/pandas/compare/v1.3.3...v1.4.3) --- updated-dependencies: - dependency-name: pandas dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add dependabot label to dependabot PRs (#898) Co-authored-by: Paul Sanders <[email protected]> * [SaaS Connector] Salesforce (erasure) (#888) * 252 saas connector sendgrid (#883) * initial sendgrid saas connector integration. access only, contacts only * erasure (update) support for sendgrid * remove unused imports * update sendgrid test fixture to expect 404 response status code because of ignore_errors enhancement * Fixing import order and cleaning up the retry logic for consistency * add sendgrid env var support to makefile and unsafe_pr_checks config. remove DELETE endpoint per PR comments * Added delete endpoint for contacts * Fixing data_path for contacts endpoint * Reverting search query to improve performance and avoid server timeouts * Updated delete endpoint request, used request instead of SaaSRequest in tests * updated imports after check suggestion * Updated code after review * Removed unused variables, imports * Restoring Makefile * Fixed import cryptographic_util error * Misc fixes * Updated Changelog file * Updated Changelog for unreleased section and pulled main * Updated Changelog and added Sendgrid in unreleased section * Updated Changelog and added Sendgrid in unreleased section with link * Updated Changelog and added Sendgrid in added section after Adam's suggestion Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> Co-authored-by: Hamza W <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> * 747 - Users should be able to click on the full field of a dropdown-type filter to open up the dropdown (#903) * 747 - Users should be able to click on the full field of a dropdown-type filter to open up the dropdown * Disabled eslint import/extensions rule for certain files with an alias path in the import statement * Updated CHANGELOG.md file * Resolved jest unit tests from failing * Removed eslint-disable import/extensions declarations * Bump types-redis from 4.3.2 to 4.3.4 (#895) Bumps [types-redis](https://github.com/python/typeshed) from 4.3.2 to 4.3.4. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-redis dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Adds saas config base info to connection config responses (#904) * adds saas config base info to connection config responses * update changelog * add properties to base class * use diff naming to avoid recursion * update saas_config prop for unit tests * another occurance of saas config in unit test * use pydantic alias for type field * sort * revert change to saas type naming * unused import * init parent class so that props are avail on self * revert to using props * Adding privacy_request_id placeholder (#911) * Bump mypy from 0.961 to 0.971 (#914) Bumps [mypy](https://github.com/python/mypy) from 0.961 to 0.971. - [Release notes](https://github.com/python/mypy/releases) - [Commits](https://github.com/python/mypy/compare/v0.961...v0.971) --- updated-dependencies: - dependency-name: mypy dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pydash from 5.0.2 to 5.1.0 (#920) Bumps [pydash](https://github.com/dgilland/pydash) from 5.0.2 to 5.1.0. - [Release notes](https://github.com/dgilland/pydash/releases) - [Changelog](https://github.com/dgilland/pydash/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/dgilland/pydash/compare/v5.0.2...v5.1.0) --- updated-dependencies: - dependency-name: pydash dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update boto3 requirement from ~=1.18.14 to ~=1.24.34 (#917) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.18.14...1.24.34) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2022.6.1 to 2022.7.0 (#915) Bumps [dask](https://github.com/dask/dask) from 2022.6.1 to 2022.7.0. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2022.6.1...2022.7.0) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix integration set up scripts for postgres and mariadb - casing has changed for config variables. (#921) * zendesk and salesforce connection docs (#908) * Adobe Campaign access and erasure (#905) * Updated tutorial to match latest fidesdemo (#772) * Correct build arg variable name (#925) * Correct build arg variable name * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Default `FIDESOPS__ADMIN_UI__ENABLED` to `True` (#936) * serve AdminUI by default * updates changelog * Update python docker base image from slim-buster to slim-bullseye (#928) * Update python docker base image from slim-buster to slim-bullseye * Update CHANGELOG * Remove ipython from dev-requirements.txt Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * Update boto3 requirement from ~=1.24.34 to ~=1.24.36 (#939) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.24.34...1.24.36) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump faker from 13.15.0 to 13.15.1 (#941) Bumps [faker](https://github.com/joke2k/faker) from 13.15.0 to 13.15.1. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](https://github.com/joke2k/faker/compare/v13.15.0...v13.15.1) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump types-ujson from 5.2.0 to 5.4.0 (#947) Bumps [types-ujson](https://github.com/python/typeshed) from 5.2.0 to 5.4.0. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-ujson dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add db vs saas to connection type api (#937) * add db vs saas to connection type api * adds changelog line * mypy * fix test * format * more tests * formatting * adds system type query param * format * adjust test saas search * correct query param * Use Nox as the build tool instead of Make (#919) * Create noxfile.py * update the dockerfile with more stages * add GitPython as a dev requirement (used by nox builds) * add the noxfiles, all docker builds work * fix the "make compose-build" command * remove the worker docker stage due to it being redundant with prod * create a unified compose file for integrations * update the password prompt to be accurate * copy/pasta run_infrastructure into the noxfiles dir so it can be called directly via python * add create_user and seed_test_data to the nox utils * docs commands work * get the generic dev command working * add db commands to utils * clean up mypy configuration in pyproject.toml and remove config from setup.cfg * simplify some of the CI targets and start updating the pytest targets * cleanup run_infrastructure * update ci_suite and other CI nox targets * add nox as a dev-requirement * get dev commands working, sans quickstart * tweak to the compose_down constant * get the new pylint target passing * remove analytics_id * updated the changelog * Apply suggestions from code review Co-authored-by: Paul Sanders <[email protected]> * fix the xenon command * add a few tweaks to the worker dev command * fix an import issue * update dockerignore * move mssql to the bottom of the datastore list Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * 863 - Retry a DSR (FE) (#938) * Update docs docker base image from slim-buster to slim-bullseye (#949) * Changed Debian base image * Update the python version to be the same as used in the fidesops app image Co-authored-by: Paul Sanders <[email protected]> * Updated changelog Co-authored-by: Dave Quinlan <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * Experimenting with fixes for the failing MSSQL CI tests (#918) * Change docker password environment variable to MSSQL_SA_PASSWORD * Revert password environment variable name * Set MSSQL user to root in docker-compose * Revert setting root user in docker-compose * Change environment variable name from REQUIRE_MSSQL to SKIP_MSSQL_INSTALLATION * Add healthcheck to mssql compose file * Modify healthcheck command * Revert healthcheck * Try for more loging information * Another try for logging * Another try for logging * Try running only mssql tests to avoid timeout * Revert mssql only flag * Extend time out to try to get logs * Revert extra logging and extended timeout * Set network mode to host * Make mssql run on its own * Remove network from docker-compose and only run mssql in integration * Increase integration test logging * Revert mssql only * Use cache for docker * Fix workflow error * Fix workflow error * Fix target * Fix make traget * Fix make traget * Revert cache * Verify that mssql is running from pytest fixture * Add restart to mssql * Revert rester in docker-compose * Revert wait for mssql in pytest fixture * Wait for mssql to be ready before adding test data * Add count of retries * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Delete custom GitHub issue templates (#955) We've defined generic issue templates for all Ethyca repos here: https://github.com/ethyca/.github/tree/main/.github/ISSUE_TEMPLATE Removing the templates from this repo allows it to automatically pull in the organization templates for consistency. * Bump sqlalchemy-redshift from 0.8.8 to 0.8.10 (#940) Bumps [sqlalchemy-redshift](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift) from 0.8.8 to 0.8.10. - [Release notes](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/releases) - [Changelog](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/blob/main/CHANGES.rst) - [Commits](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/compare/0.8.8...0.8.10) --- updated-dependencies: - dependency-name: sqlalchemy-redshift dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Move tests into an "ops" subdir (#935) * Create __init__.py * move all of the test files down one dir into `ops` * update test paths * update paths where needed * update the changelog * Dispatch a repository event on new published releases (#945) * Add release dispatch event action * Update `CHANGELOG.md` * Reduce # of clients connected to the application db [#810] (#944) * Reduce number of open connections: - Limit task concurrency to two per worker. - Create one Engine per celery process which opens up a connection pool. Create one Session per celery process and use that session across privacy requests. - Close the session after the privacy request has finished executing. This just resets the session and returns connections back to the pool. It can be reused. - Remove unnecessary places where session is closed manually because the session is being used as a context manager and is already closed through that. - Pass the same Session that the privacy request is using through to TaskResources to be re-used to create ExecutionLogs instead of opening up a new Session. - Don't close the session when passing it into the Execution Log, wait until the entire privacy request is complete/exited. * Define "self" for run_privacy_task - it's the task itself. For mypy's benefits, define that the session is a context manager. * Make a session non-optional for graph_task.run_access_request, graph_task.run_erasure, and for instantiating taskResources * Use missing db fixture. * Add missing db resource. * Update test to reflect new behavior that disabling a datasource while a request is in progress can cause related collections to be skipped once the current session is expired and the connection config has the most recent state. Because the same Session that is being used to run the PrivacyRequest is now being used for ExecutionLogs, the process of saving an ExecutionLog runs a session.commit() which expires the Session and causes the ConnectionConfig to have the most recent state the next time it is accessed. * Update CHANGELOG. * enable worker by default in our dockerfile (#958) * add extra steps to make clean (#767) * Push `dev` image on pushes to `main` (#956) * Update publish_to_dockerhub.yml * add a dev step and use nox * update the changelog * Move Client Code into an `ops` subdir (#964) * Move Client Code into an `ops` subdir * move all of the files * update the dockerfile * update package.json * update codepaths for workflow tests * Update the changelog and docs references * Update .github/dependabot.yaml * Bump gitpython from 3.1 to 3.1.27 (#971) Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1 to 3.1.27. - [Release notes](https://github.com/gitpython-developers/GitPython/releases) - [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES) - [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.0...3.1.27) --- updated-dependencies: - dependency-name: gitpython dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix the `nox -s dev` command not spinning up the webserver (#959) * Update dev_nox.py * spin up the app before running a container shell * [#927, #929, #930] sendgrid, adobe, outreach connector docs (#951) * Bump sqlalchemy-utils from 0.37.8 to 0.38.3 (#968) Bumps [sqlalchemy-utils](https://github.com/kvesteri/sqlalchemy-utils) from 0.37.8 to 0.38.3. - [Release notes](https://github.com/kvesteri/sqlalchemy-utils/releases) - [Changelog](https://github.com/kvesteri/sqlalchemy-utils/blob/master/CHANGES.rst) - [Commits](https://github.com/kvesteri/sqlalchemy-utils/compare/0.37.8...0.38.3) --- updated-dependencies: - dependency-name: sqlalchemy-utils dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pyodbc from 4.0.32 to 4.0.34 (#980) Bumps [pyodbc](https://github.com/mkleehammer/pyodbc) from 4.0.32 to 4.0.34. - [Release notes](https://github.com/mkleehammer/pyodbc/releases) - [Commits](https://github.com/mkleehammer/pyodbc/compare/4.0.32...4.0.34) --- updated-dependencies: - dependency-name: pyodbc dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2022.7.0 to 2022.7.1 (#967) Bumps [dask](https://github.com/dask/dask) from 2022.7.0 to 2022.7.1. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2022.7.0...2022.7.1) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump alembic from 1.8.0 to 1.8.1 (#989) Bumps [alembic](https://github.com/sqlalchemy/alembic) from 1.8.0 to 1.8.1. - [Release notes](https://github.com/sqlalchemy/alembic/releases) - [Changelog](https://github.com/sqlalchemy/alembic/blob/main/CHANGES) - [Commits](https://github.com/sqlalchemy/alembic/commits) --- updated-dependencies: - dependency-name: alembic dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix erroneous values in the Outreach config (#988) * Add documentation for new nox commands (#981) * Add documentation for new nox commands * changelog * missed make to nox edits * review edits * update `nox -s dev` to not open a shell, add it as a posarg option * Apply suggestions from code review * remove the analytics id * update typo * remove extra numbering Co-authored-by: Thomas <[email protected]> * Bump types-toml from 0.10.7 to 0.10.8 (#998) Bumps [types-toml](https://github.com/python/typeshed) from 0.10.7 to 0.10.8. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-toml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fideslog from 1.2.1 to 1.2.2 (#996) Bumps [fideslog](https://github.com/ethyca/fideslog) from 1.2.1 to 1.2.2. - [Release notes](https://github.com/ethyca/fideslog/releases) - [Commits](https://github.com/ethyca/fideslog/compare/v1.2.1...v1.2.2) --- updated-dependencies: - dependency-name: fideslog dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Added Choose your connection feature (#987) * Added Choose your connection feature 760-Add a Connection - Select a connector to configure (front end) 866-Add a Connection - Front End layout structure * Updated CHANGELOG.md file * Fix lint issues * Fix build issue * Hide overflow Co-authored-by: Andrew Jackson <[email protected]> * saas request overrides (#986) * initial cut of saas request overrides. include mailchimp as an example and test case. minor refactor of some of the saas request execution to enable smoother override * fix rebase issue by moving saas override tests into ops subdir * import path updates to resolve conflicts caused by rebase * add session parameter into graph task calls to fix saas override integration tests caused by rebase * update changelog * tweaks to saas connector overrides and associated tests * expose override factory register as module variable for clenaer decorator calls Co-authored-by: Adam Sachs <[email protected]> * Update OAuth strategy to be able to perform local testing (#962) * Update boto3 requirement from ~=1.24.36 to ~=1.24.42 (#1001) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.24.36...1.24.42) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fastapi[all] from 0.78.0 to 0.79.0 (#1002) Bumps [fastapi[all]](https://github.com/tiangolo/fastapi) from 0.78.0 to 0.79.0. - [Release notes](https://github.com/tiangolo/fastapi/releases) - [Commits](https://github.com/tiangolo/fastapi/compare/0.78.0...0.79.0) --- updated-dependencies: - dependency-name: fastapi[all] dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update config reference to use lowercase values (#952) * [#909] update config reference to use lowercase values * changelog * missed uppercase variables * update usages of False to false in connector docs * Added Auth0 Connector (#991) * add pagination back to connection types endpoints (#1019) * add pagination back to connection types endpoints * check for pagination in system_type search Co-authored-by: Paul Sanders <[email protected]> * Subject Request Events and Logs Section (#1018) * Fix small issue with eslint config * Get initial drawer working * Add jest config to eslintignore * Fix small css issue * Refactor components and get initial functionality * Fix small logic error * Fix couple of bugs and format code * Conditinally display error tag * Fix issues with merge * Format and lint * Sort imports * Update to new solution * Format code * Update changelog Co-authored-by: Sean Preston <[email protected]> * First draft of OAuth documentation (#963) * Send Errored Requests / Reprocessed Requests Info to FidesLog [#754] (#993) * Add a method to format a representation of the graph for caching in Redis and a separate method to build a summary of the differences in the graph when a privacy request is rerun to fideslog. - Adds FieldAddress.from_string method. * Add methods to cache a representation of the access graph when it is built and a separate method to retrieve it from the cache. - Also add a method to build a "rerun_access_graph" AnalyticsEvent for fideslog where applicable. * When running the access portion of the privacy request, log stats about a rerun and then cache the current access graph. - Give a different prefix when we're caching the access graph to not get mixed up with access request results. * Show skipped new edges that are directly upstream of completed nodes instead. These edges are intentionally dropped from the new graph on rerun, so want to surface this count. * Log if a privacy request fails during the "erasure" step of privacy request execution. Even though the access step is not rerun here, compare the previously cached access graph with the access graph that would have been run to determine what data has changed. * Add missing session variables. * Send an event to Fideslog when privacy request execution fails. * Add missed session - bad merge. * Update changelog. * Update docstrings. * Remove copy/paste comment. * Respond to CR comments. * Currently AnalyticsEvent.local_host cannot be None. * Update the compose file and workflows to expect an already-built image (#966) * Update the compose file and workflows to be in line with fidesctl * update the compose file to look for a specific image * rename files and update the safe PR checks * add check_migrations to the ci checks and nox * fix the failing PR checks * fix CI failures * update the unsafe checks workflow * update run_infrastructure to use the compose service name * remove the makefile and the old run_infra script * bump pylint version, pin isort, fix issues * update pytest setup path * update the changelog * make OPS_TEST_DIR a constant * fix nox missing vars * specify that the nox imports are relative imports * remove relative import paths * run isort Co-authored-by: Sean Preston <[email protected]> * Create `AuditLog` on privacy request approval (#1038) * Create approval audit logs * Add tests * disable a pylint lint * Update changelog * Update privacy_request fixture * Updating Salesforce to use OAuth2 authentication code flow (#1039) * Removing saas_config.toml (#1043) * Bump types-pyyaml from 6.0.9 to 6.0.11 (#1047) Bumps [types-pyyaml](https://github.com/python/typeshed) from 6.0.9 to 6.0.11. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pyyaml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump docker/build-push-action from 2 to 3 (#1044) Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 2 to 3. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/v2...v3) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2022.7.1 to 2022.8.0 (#1046) Bumps [dask](https://github.com/dask/dask) from 2022.7.1 to 2022.8.0. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2022.7.1...2022.8.0) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update boto3 requirement from ~=1.24.42 to ~=1.24.46 (#1045) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [C…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a specific problem?
A method is needed to sync server-side changes to the local filesystem
Describe the solution you'd like
A new
fidesctl synccommand that can write local files from server resources.Describe alternatives you've considered, if any
A description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: