Skip to content
This repository has been archived by the owner on Nov 30, 2022. It is now read-only.

Latest commit

 

History

History
542 lines (397 loc) · 34.8 KB

CHANGELOG.md

File metadata and controls

542 lines (397 loc) · 34.8 KB
Raw

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog

The types of changes are:

  • Added for new features.
  • Changed for changes in existing functionality.
  • Developer Experience for changes in developer workflow or tooling.
  • Deprecated for soon-to-be removed features.
  • Breaking Changes for updates that break public facing APIs
  • Docs for documentation only changes.
  • Removed for now removed features.
  • Fixed for any bug fixes.
  • Security in case of vulnerabilities.

Unreleased

Added

  • Add deprecation warning #1429

Changed

  • Fix redis db_index config issue #1427
  • Allow multiple masking strategies to be specified when using fides as a masking engine #1428

1.8.0

Changed

  • Refactor privacy center to be more modular #1363

Docs

  • Update docs footer links #1406

Fixed

  • Distinguish whether webhook has been visited and no fields were found, versus never visited #1339
  • Fix Redis Cache Early Expiration in Tests #1358
  • Limit values for the offset pagination strategy are now cast to integers before use #1364
  • Allow requires_input PrivacyRequests to be addressed if a webhook is deleted, disabled, or updated #1394

Added

  • api_key auth strategy for SaaS connectors #1331
  • Access support for Rollbar #1361
  • Adds a new Timescale connector #1327
  • Allow querying the non-default schema with the Postgres Connector #1375
  • Frontend - ability for users to manually enter PII to an IN PROGRESS subject request #1016
  • Enable retries on saas connectors for failures at the http request level #1376
  • Add consent request api #1387
  • Add authenticated route to get consent preferences #1402
  • Access and erasure support for Braze #1248
  • Admin UI: Persist Redux store to localStorage #1401
  • Access and update support for Firebase Auth #1339
  • Adds the Braze connector to the SaaS connector registry #1418

Removed

  • Removed query_param auth strategy as api_key auth strategy now supersedes it #1331

Developer Experience

  • Update Fidesops config with sane defaults where necessary #1393

1.8.0

Developer Experience

  • Created a docker image for the privacy center #1165
  • Adds email scopes to postman collection #1241
  • Clean up docker build #1252
  • Add Strategy abstract base class for more extensible strategy development 1254

Added

  • Adds users and owners configuration for Hubspot connector #1091
  • Foundations for a new email connector type #1142
  • Have the new email connector cache action needed for each collection #1168
  • Adds new Celery queue for async email dispatch #1173
  • Added execution_timeframe to Policy model and schema #1244
  • Added due_date to Privacy request model #1259
  • Wrap up the email connector - it sends an email with erasure instructions as part of request execution #1246
  • Added days_left field to Privacy request response #1281
  • Mapping Vault environment variables in docker-compose.yml #1275
  • Foundations for a new "manual_webhook" connector type #1267
  • Add Days left display to admin ui #1283
  • Data seeding for Datadog access tests #1269
  • Added support for one-to-many relationships for param_values in SaaS configs #1253
  • Added due_date sorting #1284
  • Added erasure endpoints for Shopify connector #1289
  • Adds ability to send email notification upon privacy request completion #1282
  • Added human readable label to ConnectionType endpoint #1297
  • Enable new manual webhooks in privacy request execution #1285
  • Add table for consent #1301
  • Adds ability to send email notification upon privacy request receipt #1303
  • Utility to update SaaS config instances based on template updates #1307
  • Added generic request sorting button #1320
  • Adds ability to send email notification upon privacy request review #1306
  • Add a Connector - upload a DB Dataset YAML #923
  • Add a Connector - DB connector configs #922
  • Add a Connector - SaaS Dataset Management (YAML method) #1090
  • Frontend - Configure a Manual entry connector #1015
  • Manual webhook test functionality #1323
  • Added change request table #1340

Changed

  • Renamed PrivacyRequestIdentity to Identity #1324
  • Use existing Celery queue for all email dispatch calls #1341

Developer Experience

  • Created a docker image for the privacy center #1165
  • Adds email scopes to postman collection #1241
  • Clean up docker build #1252
  • Add Strategy abstract base class for more extensible strategy development 1254

Docs

  • Fix analytics opt out environment variable name #1170
  • Added how to view a subject request history and reprocess a subject request #1164
  • Adds section on email communications, and exposes previously hidden guides in nav bar #1233
  • Change references to FIDESOPS__CONFIG_PATH to FIDES__CONFIG_PATH #1302

Fixed

  • Fix issue with fideslog event loop errors #1174
  • Allow passwords to be sent either base64 encode or as plaintext. #1236
  • Allow worker to start up successfully for dev and dev_with_worker nox commands #1250
  • Fix for pytest-asyncio bug #1260
  • Fix download link in privacy center #1264
  • Make admin ui work when volumes are mounted #1266
  • Fixed typo in enum value #1280
  • Remove masking of redis error log #1288
  • Logout with malformed or expired token #1305
  • The toml package is now included in the list of direct dependencies (requirements.txt) #1338
  • Fix bug where erasure counts instead of access results may be retrieved for certain collections on request retry #1349

Security

  • Bump python from 3.9.13 to 3.9.14 #1287

1.7.2

Added

  • Adds users and owners configuration for Hubspot connector #1091
  • Foundations for a new email connector type #1142
  • Adds endpoint for GET identity verification config #1221
  • Add user identification flow to privacy center #1231
  • Access support for Shopify #1220

1.7.1

Breaking Changes

The oauth2 strategy has been renamed to oauth2_authorization_code in order to make a distinction between the newly introduced oauth2_client_credentials strategy #1159

Added

  • Access support for Datadog Logs #1060
  • Adds infra for email config and dispatch #1059
  • Add an endpoint that allows you to create a Saas connector and all supporting resources with a single request #1076
  • Add a Connection - SaaS connector's configuration parameters #761
  • Saas Connector configuration - left navigation to toggle between connection params and dataset config #984
  • SaaS Connector Configuration - Testing a Connection #985
  • Add an endpoint for verifying the user's identity before queuing the privacy request. #1111
  • Adds tests for email endpoints and service #1112
  • Adds the ability to verify a subject's identity before processing a Privacy Request #1115
  • Add option to login as root user from config#1116
  • Added email templates #1123
  • Add Retry button back into the subject request detail view #1128

Developer Experience

  • When releases are published, dispatch a repository webhook event to ethyca/fidesdemo #1075

Fixed

  • HTTP headers are now preserved in requests generated from SaaS connector pagination #1069
  • Bump fideslib to fix issue where the authenticate button in the FastAPI docs did not work #1092
  • Escape the Redis user and password to make them URL friendly #1104
  • Reduced number of connections opened against app db during health checks #1107
  • Fix FIDESOPSROOT_USERANALYTICS_ID generation when env var is set #1113
  • Set localhost to None for non-endpoint events #1130
  • Fixed docs build in CI #1138
  • Added future annotations to privacy_request.py for backwards compatibility #1136

Changed

  • Update request status endpoint to return both audit and execution logs [#1068] (#1068)
  • Update backend routing to handle dynamic frontend routes #1033
  • Make connection type search case-insensitive #1133
  • Adjust log formatting to be %-style instead of f-string #1132
  • Update StorageConfig to support multiple auth methods for S3 #1140

Docs

Removed

  • Removed create superuser #1116

1.7.0

Added

  • Erasure support for Salesforce #888
  • Publish a dev tagged image on every push to main #956
  • Access and erasure support for Sendgrid contacts endpoint #883
  • Added saas config base info to connection config responses #904
  • Access and erasure support for Adobe Campaign #905
  • Added db vs saas to connection type api #937
  • Retry a DSR (FE) #863
  • Add a Connection - Select a connector to configure (front end) #760
  • Add a Connection - Front End layout structure #866
  • Enable python function overrides for SaaS connector request execution #986
  • add Events and logs section to Subject Request Details Page #1018
  • Access and erasure support for Auth0 #991
  • Start better understanding how request execution fails #993
  • Add approval AuditLogs for user and sytem approved privacy requests #1038
  • Add finished AuditLog for subject requests #1040

Changed

  • Users should be able to click on the full field of a dropdown-type filter to open up the dropdown #747
  • Reorganize the core code logic to be easier to merge with fidesctl #1058
  • Updated the python docker base image from slim-buster to slim-bullseye #928
  • Removed ipython from the docker install #928
  • Run static nox checks outside of Docker #1053
  • Serve admin UI by default #906
  • Move tests into an ops subdir #935
  • Move client code into an ops subdir #964
  • Updated the docs docker base image to be consistent with the fidesops image #949
  • Update fidesops to use bcrypt for hashing #876

Changed

  • Updated fideslog==1.1.0 #890

Developer Experience

  • Fix randomly failing mssql integration tests #918
  • Update the way make clean works #767
  • When releases are published, dispatch a repository webhook event to ethyca/fidesops-plus #945
  • Added nox commands to replace the Makefile #919
  • GitHub Actions Workflows leverage nox commands #966
  • The docker-compose.yml file no longer handles builds #966

Docs

  • Added zendesk and salesforce connection pages #908
  • Added documentation for OAuth2 configuration and usage #963
  • Added nox command documentation #981
  • Updated references to fidesops.toml to use lowercase values #952

Fixed

  • Correct build arg variable name #925
  • Reduce number of clients connected to the application db #944
  • Patch versioneer to allow editable installs #1070
  • Add setuptools to dev-requirements to fix versioneer error and revert patch #1072

1.6.3

Added

  • Adds COMPOSE_SERVICE_NAME to Makefile #884

1.6.2

Added

  • [Admin UI] Update Subject Request status filter to be a multiselect dropdown #513
  • Add support for multiple statuses to be selected for filtering subject requests #660
  • [Datastore Management] Include Icon with datastore connection (FE)#671
  • Store provided identity data in the privacy request table #743
  • [Datastore Management] Keep datastore cards in 1/3 screen pattern #745
  • Adds SaaS connection type to SaaS yaml config 748
  • Adds exact match identity search to the privacy request status endpoint #765
  • Adds endpoint to get available connectors (database and saas) #768
  • Erasure support for Zendesk #775
  • Adds endpoint to get the secrets required for different connectors #795
  • Adds Vault for secrets management #688
  • Adds privacy_request_id placeholder to use in SaaS configs #911

Changed

  • Update clipboard icon in Admin UI #838
  • Stop masking uvicorn logs by default #831
  • Bump fideslib to handle base64 encoded password #820
  • Return identity data from application DB, instead of cache #860
  • Update admin ui to be served from the root route / #720
  • Refactored the root directory and various Docker-related files #877

Developer Experience

  • Replace user authentication routes with fideslib routes #811
  • Reduce docker image size #846
  • Load Celery config overrides from a dedicated celery.toml #821

Docs

  • Backend UI deployment #827
  • Fix publish_docs CI action #818
  • Reorganize docs and standardize formatting #858
  • Changed wording on Admin UI login page #774
  • Fixed typos in Admin UI #774

Fixed

  • Resolve issue with MyPy seeing files in fidesops as missing imports #719
  • Fixed check-migrations Make command #806
  • Fix issue requiring separate install of snowflake-connector-python #807
  • [User Management] Create new user gives HTTP 422 Unprocessable Entity exception #833
  • Fix error when running make test-data 862
  • [User Management] Refactored New and Edit user pages to reduce duplicate code #839
  • Fix error when there are no scopes in ClientDetail #830

1.6.1

Added

  • fidesops worker command for running a Celery worker #673

Developer Experience

  • Add fixture to clear tables between test #680
  • Reduce the size of the docker image #707
  • Parallelize CI safe checks to reduce run time #717
  • Add dependabot to keep dependencies up to date #718
  • Make running a worker node optional #770

Changed

  • Base64 encode passwords on frontend #749

Docs

  • Updated the tutorial installation to use main in fidesdemo #715
  • Added a page on how to use the datastore UI #742
  • Added a page on implementing and opting out of fideslog analytics #751

Fixed

  • Make reading of environment variables case insensitive #712
  • Fix console warning in disable connection modal #750
  • Fix no such container error with docker-compose #758
  • Fixed issue with extending the configuration #721

1.6.0

Added

  • Subject Request Details page #563
  • Restart Graph from Failure #578
  • Redis SSL Support #611
  • Celery as a dependency for use in the execution layer #610
  • Cache and Surface Resume/Restart Instructions #591
  • Build and deploy Admin UI from webserver #625
  • Allow disabling a ConnectionConfig #637
  • Erasure support for Outreach connector #619
  • Adds searching of ConnectionConfigs #641
  • Added AdminUiSettings to the log_all_config_values helper method #647
  • Prettier formatting CI check for frontend code #655
  • Adds default policies #654
  • Added ConnectionConfig connection_type and disabled filters #675
  • Adds Fideslog integration #541
  • Adds endpoint analytics events #622
  • Sample dataset for Salesforce with access configuration #676
  • Sample dataset and access configuration for Zendesk (ticket endpoints) #677
  • Include number of records to be masked in masking endpoint's log message #692
  • Datastore Connection Landing Page #674
  • Added the ability to delete a datastore from the frontend #683
  • Added the ability to disable/enable a datastore from the frontend #693
  • Adds Postgres and Redis health checks to health endpoint #690
  • Adds the ability to revoke a pending privacy request #592
  • Added health checks and better error messages on app startup for both db and cache #686
  • Datastore Connection Filters #691

Changed

  • Refactor auth and enable static file serving #577
  • Bumped mypy to version 0.961 #630
  • Bumped Python to version 3.9.13 in the Dockerfile #630
  • Matched the path to the migrations in the mypy settings with the new location #634
  • Sort ConnectionConfig by name ascending #668
  • Install MSSQL By Default #664
  • [Admin UI] Change "Policy Name" to "Request Type" on SR list page.#546
  • Queue PrivacyRequests into a Celery queue for execution #621
  • Added filtering clearing in datastore connections #701

Developer Experience

  • Add celerybeat-schedule file to gitignore #639
  • Use v2.1.0 of fideslib #705

Docs

  • Subject Request detail documentation for the UI #702

Fixed

  • Fixed error with running mypy on M1 Macs #630
  • Fixed error with mypy on Python versions greater than 3.9.6 #630
  • Bumped fideslib to 2.0.4. This fixes the issue where alembic couldn't find the fidesops.toml file from its new location #643
  • Fixes Postman Collection inconsistencies #704

1.5.3

Changed

  • Database migrations now exist as part of the core fidesops package #620

Removed

  • The [package] config section no longer exists #620

Changed

  • Process privacy requests as Celery tasks and not background processes #621

1.5.2

Added

  • Added OAuth2 authentication strategy for SaaS connectors #555
  • Added FIDESOPS__SECURITY__LOG_LEVEL configuration variable to allow controlling the log level #579
  • Added DEBUG logs at startup to view all configuration values #579
  • Modified filter post-processor to include toggles for exact and case sensitive matching #584
  • Added dataset for Outreach with access configuration #588
  • All directories containing *.py files now also contain __init__.py files #590
  • Pause Erasure Request Execution / Resume on Manual Input in #571

Changed

  • Use the RuleResponse schema within the PrivacyRequestReposnse schema #580
  • Updated the webserver to use PORT config variable from the fidesops.toml file #586
  • Updated black-ci makefile command to also check tests/ directory #594

Developer Experience

  • Adds a script for MSSQL schema exploration #557

1.5.1 - 2022-05-27

Added

  • Added FIDESOPS__DATABASE__ENABLED and FIDESOPS__REDIS__ENABLED configuration variables to allow fidesops to run cleanly in a "stateless" mode without any database or redis cache integration #550
  • Pause Access Request Execution / Resume on Manual Input in #554
  • A [package] section of the fidesops.toml configuration file may specify the path to the fidesops package itself #566

Changed

  • MaskingStrategyFactory and associated MaskingStrategy implementations now use a decorator-based registration system, to improve extensibility #560
  • Added from __future__ import annotations to src/fidesops/util/logger.py to maintain backward compatibility with Python < 3.9 #569

Developer Experience

  • Import ordering is now enforced using isort in CI #533
  • Teardown all Docker infra once it's finished with #498
  • Update PR checklist for CHANGELOG.md file #558
  • Database migrations are included in the published PyPI package #566

Docs

  • Updated documentation for the user management ui #530
  • Added documentation for the privacy center #549

Fixed

  • Fixed type errors for privacy center build #540

1.5.0 - 2022-05-18

Added

  • ESLint configuration changes #514
  • User creation, update and permissions in the Admin UI #511
  • Yaml support for dataset upload #284

Breaking Changes

  • Update masking API to take multiple input values #443

Docs

  • DRP feature documentation #520

1.4.2 - 2022-05-12

Added

  • GET routes for users #405
  • Username based search on GET route #444
  • FIDESOPS__DEV_MODE for Easier SaaS Request Debugging #363
  • Track user privileges across sessions #425
  • Add first_name and last_name fields. Also add them along with created_at to FidesUser response #465
  • Denial reasons for DSR and user AuditLog #463
  • DRP action to Policy #453
  • CHANGELOG.md file#484
  • DRP status endpoint #485
  • DRP exerise endpoint #496
  • Frontend for privacy request denial reaons #480
  • Publish Fidesops to Pypi #491
  • DRP data rights endpoint #526

Changed

  • Converted HTTP Status Codes to Starlette constant values #438
  • SaasConnector.send behavior on ignore_errors now returns raw response #462
  • Seed user permissions in create_superuser.py script #468
  • User API Endpoints (update fields and reset user passwords) #471
  • Format tests with black #466
  • Extract privacy request endpoint logic into separate service for DRP #470
  • Fixing inconsistent SaaS connector integration tests #473
  • Add user data to login response #501

Breaking Changes

  • Update masking API to take multiple input values #443

Docs

  • Added issue template for documentation updates #442
  • Clarify masking updates #464
  • Added dark mode #476

Fixed

  • Removed miradb test warning #436
  • Added missing import #448
  • Removed pypi badge pointing to wrong package #452
  • Audit imports and references #479
  • Switch to using update method on PUT permission endpoint #500

Developer Experience

  • Add script to seed initial Privacy Request #487
  • Add first and last name to createsuperuser script #486