Release v0.8 (#369)

# Changelog

## General

- Framework renamed from DFF to Chatsky (#368)

## Features

- New `has_text` condition for asserting that specific text is contained
inside the last request (#335)
- Validation stage reworked.
`Pipeline.validation_stage`, `Pipeline.verbose` and `Context.validation`
fields are removed.
  Added new type for annotating labels `dff.script.ConstLabel` (#289)
- Functions `cnd.exact_match` and `check_happy_path` now accept both
`Message` and `str`.
The following are now equivalent: `exact_match(Message("text"))` ==
`exact_match("text")`
  (#337)
- CLIMessengerInterface moved to `messengers.console` (#328)
- Attachments reworked (#328):
- `Session`, `Command`, `Link`, `Button`, `Keyboard` and `Attachments`
classes removed, `Message.commands` field removed.
- Added `CallbackQuery`, `Contact`, `Invoice`, `Poll`, `Animation`,
`Sticker`, `VoiceMessage`, `VideoMessage` and `MediaGroup` classes.
  - Added attachment caching feature.
  - Added `has_callback_query` condition.
  - Removed `title` field from Data Attachments. Use `caption` instead.
- Telegram Messenger interface reworked (#328):
  - Now provides methods for attachment byte download.
- Added support for receiving attachment types as members of
`Message.attachments`. Others may be accessed via
`Message.original_message`.
- Added support for sending new attachment types (`Sticker`, `Poll`,
e.t.c.).
- Added support for extra options such as `disable_notification`,
`message_effect_id` or `caption`.
  - Telegram tutorials rewritten to reflect all the changes.
- Added slots that simplify the process of extracting arbitrary data
from user messages and accessing it later (#36):
  - New user guide and tutorials on slots

## Documentation

- Return PRE_RESPONSE_PROCESSING and PRE_TRANSITION_PROCESSING tutorials
- New telegram interface tutorials (#328)
- Slots user guide and tutorials (#36)

## Contrib

- Added `poe quick_test` to run tests that are not slow and do not
require docker (#344)
- `Context.framework_states` renamed to `Context.framework_data` and
made `Pydantic.BaseModel` (#359)
- Added `MessengerInterfaceWithAttachments` class (#328)
- Added tools for enabling json serialization via pickle (#328)
- Telegram tests made independent from telegram servers (#328)
- `wrap_sync_function_in_async` moved to `utils.devel` (#36)

# Checklist

- [x] I have performed a self-review of the changes
- [x] Rename repos (this; template; demo)
- [ ] Add new logo; change favicons in conf.py (this can be done later)
- [x] Create new PyPi project and update pypi token
- [ ] Create a new release
- [x] Make a new release on the
[final_dff_release](https://github.com/deeppavlov/dialog_flow_framework/tree/final_dff_release)
branch

.dockerignore

@@ -1,6 +1,7 @@
 *.DS_Store*
 *.egg-info/
 dist/
+venv/
 build/
 docs/source/apiref
 docs/source/_misc
@@ -21,6 +22,7 @@ GlobalUserTableAccessor*
 memory_debugging*
 opening_database*
 _globals.py
+venv*
 .vscode
 .coverage
 .coverage.*

.github/process_github_events.py

@@ -48,9 +48,6 @@ def post_comment_on_pr(comment: str, pr_number: int):
 - [ ] Update package version
 - [ ] Update `poetry.lock`
 - [ ] Change PR merge option
-- [ ] Test modules without automated testing:
-  - [ ] Requiring telegram `api_id` and `api_hash`
-  - [ ] Requiring `HF_API_KEY`
 - [ ] Search for objects to be deprecated
 """
 

.github/workflows/build_and_publish_docs.yml

@@ -39,8 +39,6 @@ jobs:
       - name: build documentation
         env:
           TG_BOT_TOKEN: ${{ secrets.TG_BOT_TOKEN }}
-          TG_API_ID: ${{ secrets.TG_API_ID }}
-          TG_API_HASH: ${{ secrets.TG_API_HASH }}
           TG_BOT_USERNAME: ${{ secrets.TG_BOT_USERNAME }}
         run: |
           python -m poetry run poe docs

.github/workflows/test_coverage.yml

@@ -38,8 +38,6 @@ jobs:
       - name: run tests
         env:
           TG_BOT_TOKEN: ${{ secrets.TG_BOT_TOKEN }}
-          TG_API_ID: ${{ secrets.TG_API_ID }}
-          TG_API_HASH: ${{ secrets.TG_API_HASH }}
           TG_BOT_USERNAME: ${{ secrets.TG_BOT_USERNAME }}
         run: |
           python -m poetry run poe test_all

.github/workflows/test_full.yml

@@ -40,8 +40,6 @@ jobs:
       - name: run pytest
         env:
           TG_BOT_TOKEN: ${{ secrets.TG_BOT_TOKEN }}
-          TG_API_ID: ${{ secrets.TG_API_ID }}
-          TG_API_HASH: ${{ secrets.TG_API_HASH }}
           TG_BOT_USERNAME: ${{ secrets.TG_BOT_USERNAME }}
         run: |
           python -m poetry run poe test_no_cov
@@ -64,8 +62,6 @@ jobs:
       - name: run pytest
         env:
           TG_BOT_TOKEN: ${{ secrets.TG_BOT_TOKEN }}
-          TG_API_ID: ${{ secrets.TG_API_ID }}
-          TG_API_HASH: ${{ secrets.TG_API_HASH }}
           TG_BOT_USERNAME: ${{ secrets.TG_BOT_USERNAME }}
         run: |
           python -m poetry run poe test_no_deps

.github/workflows/update_dashboard.yml

@@ -5,7 +5,7 @@ on:
     branches:
       - 'master'
     paths:
-      - 'dff/utils/docker/**'
+      - 'chatsky/utils/docker/**'
   workflow_dispatch:
 
 concurrency:
@@ -41,7 +41,7 @@ jobs:
         - name: Build and upload image
           uses: docker/build-push-action@v5
           with:
-            context: dff/utils/docker
-            file: dff/utils/docker/dockerfile_stats
+            context: chatsky/utils/docker
+            file: chatsky/utils/docker/dockerfile_stats
             tags: ${{ steps.meta.outputs.tags }}
             labels: ${{ steps.meta.outputs.labels }}

.gitignore

@@ -1,6 +1,7 @@
 *.DS_Store*
 *.egg-info/
 dist/
+venv/
 build/
 docs/source/apiref
 docs/source/_misc
@@ -21,6 +22,7 @@ GlobalUserTableAccessor*
 memory_debugging*
 opening_database*
 _globals.py
+venv*
 .vscode
 .coverage
 .coverage.*

CONTRIBUTING.md

@@ -1,10 +1,10 @@
 ## Introduction
 We will be glad to receive your pull requests (PRs) and issues for adding new features if you are missing something.
-We always look forward to your contributions to the Dialog Flow Framework (DFF). 
+We always look forward to your contributions to Chatsky. 
 
 ## Rules for submitting a PR
 
-All PRs are reviewed by DFF developers team.
+All PRs are reviewed by Chatsky developers team.
 In order to make the job of reviewing easier and increase the chance that your PR will be accepted,
 please add a short description with information about why this PR is needed and what changes will be made.
 Please use the following rules to write the names of branches and commit messages.
@@ -69,27 +69,9 @@ by activating the virtual environment and then running
 poetry run poe docs
 ```
 
-After that `docs/build` dir will be created and you can open index file `docs/build/index.html` in your browser of choice.
-
-#### Documentation links
-
-In your tutorials, you can use special expanding directives in markdown cells.
-They can help shorten the comments and avoid boilerplate code.
-The documentation links generated by the directives are always relative
-to the local documentation and verified during build.
+Note that you'll need `pandoc` installed on your system in order to build docs.
 
-- `%pip install {args}`
-    This directive generates dependency installation cell, adds a comment and sets up "quiet" flag.
-
-    It should be used in tutorials, like this: `# %pip install dff[...]`.
-- `%doclink({args})`
-    This directive generates a documentation link. It supports 2 or three arguments and the generated link will look like: `ARG1/ARG2#ARG3`.
-
-    The first argument can be either `api` for DFF codebase, `tutorial` for tutorials or `guide` for user guides.
-- `%mddoclink({args})`
-    This directive is a shortcut for `%doclink` that generates a markdown format link instead.
-
-    The generated link will be either `[ARG2](%doclink(ARG1,ARG2))` or `[ARG3](%doclink(ARG1,ARG2,ARG3))`.
+After that `docs/build` dir will be created and you can open index file `docs/build/index.html` in your browser of choice.
 
 #### Documentation links
 
@@ -101,11 +83,11 @@ to the local documentation and verified during build.
 - `%pip install {args}`
     This directive generates dependency installation cell, adds a comment and sets up "quiet" flag.
 
-    It should be used in tutorials, like this: `# %pip install dff[...]`.
+    It should be used in tutorials, like this: `# %pip install chatsky[...]`.
 - `%doclink({args})`
     This directive generates a documentation link. It supports 2 or three arguments and the generated link will look like: `ARG1/ARG2#ARG3`.
 
-    The first argument can be either `api` for DFF codebase, `tutorial` for tutorials or `guide` for user guides.
+    The first argument can be either `api` for Chatsky codebase, `tutorial` for tutorials or `guide` for user guides.
 - `%mddoclink({args})`
     This directive is a shortcut for `%doclink` that generates a markdown format link instead.
 
@@ -132,6 +114,13 @@ poetry run poe test_all
 ```
 for successful execution of this command `Docker` and `docker compose` are required.
 
+To do a quick sanity check without the need to up docker containers or wait for long tests, run
+```bash
+poetry run poe quick_test
+```
+
+_There's also quick_test_coverage for quick htmlcov generation, though it is very likely to be incomplete due to deselection of some tests._
+
 To make sure that the code satisfies only the style requirements, run
 ```bash
 poetry run poe lint
@@ -144,7 +133,7 @@ poetry run poe format
 Tests are configured via [`.env_file`](.env_file).
 
 ### Docker
-DFF uses docker images for two purposes:
+Chatsky uses docker images for two purposes:
 1. Database images for integration testing.
 2. Images for statistics collection.
 

README.md

@@ -1,35 +1,35 @@
-# Dialog Flow Framework
+# Chatsky
 
-[![Documentation Status](https://github.com/deeppavlov/dialog_flow_framework/workflows/build_and_publish_docs/badge.svg?branch=dev)](https://deeppavlov.github.io/dialog_flow_framework)
-[![Codestyle](https://github.com/deeppavlov/dialog_flow_framework/workflows/codestyle/badge.svg?branch=dev)](https://github.com/deeppavlov/dialog_flow_framework/actions/workflows/codestyle.yml)
-[![Tests](https://github.com/deeppavlov/dialog_flow_framework/workflows/test_coverage/badge.svg?branch=dev)](https://github.com/deeppavlov/dialog_flow_framework/actions/workflows/test_coverage.yml)
-[![License Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/deeppavlov/dialog_flow_framework/blob/master/LICENSE)
+[![Documentation Status](https://github.com/deeppavlov/chatsky/workflows/build_and_publish_docs/badge.svg?branch=dev)](https://deeppavlov.github.io/chatsky)
+[![Codestyle](https://github.com/deeppavlov/chatsky/workflows/codestyle/badge.svg?branch=dev)](https://github.com/deeppavlov/chatsky/actions/workflows/codestyle.yml)
+[![Tests](https://github.com/deeppavlov/chatsky/workflows/test_coverage/badge.svg?branch=dev)](https://github.com/deeppavlov/chatsky/actions/workflows/test_coverage.yml)
+[![License Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/deeppavlov/chatsky/blob/master/LICENSE)
 ![Python 3.8, 3.9, 3.10, 3.11](https://img.shields.io/badge/python-3.8%20%7C%203.9%20%7C%203.10%20%7C%203.11-green.svg)
-[![PyPI](https://img.shields.io/pypi/v/dff)](https://pypi.org/project/dff/)
-[![Downloads](https://pepy.tech/badge/dff)](https://pepy.tech/project/dff)
+[![PyPI](https://img.shields.io/pypi/v/chatsky)](https://pypi.org/project/chatsky/)
+[![Downloads](https://pepy.tech/badge/chatsky)](https://pepy.tech/project/chatsky)
 
-The Dialog Flow Framework (DFF) allows you to develop conversational services.
-DFF offers a specialized domain-specific language (DSL) for quickly writing dialogs in pure Python. The service is created by defining a special dialog graph that determines the behavior of the dialog agent. The latter is then leveraged in the DFF pipeline.
+Chatsky allows you to develop conversational services.
+Chatsky offers a specialized domain-specific language (DSL) for quickly writing dialogs in pure Python. The service is created by defining a special dialog graph that determines the behavior of the dialog agent. The latter is then leveraged in the Chatsky pipeline.
 You can use the framework in various services such as social networks, call centers, websites, personal assistants, etc.
 
-DFF, a versatile Python-based conversational service framework, can be deployed across a spectrum of platforms,
+Chatsky, a versatile Python-based conversational service framework, can be deployed across a spectrum of platforms,
 ensuring flexibility for both novice and seasoned developers:
 
-- Cloud platforms like AWS, Azure, and GCP offer scalable environments for DFF,
+- Cloud platforms like AWS, Azure, and GCP offer scalable environments for Chatsky,
   with options such as AWS Lambda and Azure Functions providing serverless execution.
-- For containerized deployment, Docker and Kubernetes streamline the orchestration of DFF applications.
+- For containerized deployment, Docker and Kubernetes streamline the orchestration of Chatsky applications.
 - Furthermore, the framework's adaptability extends to IoT ecosystems,
   making it suitable for integration with edge devices in scenarios like smart homes or industrial automation.
 
 Whether deployed on cloud platforms, containerized environments, or directly on IoT devices,
-DFF's accessibility and customization options make it a robust choice for developing conversational services
+Chatsky's accessibility and customization options make it a robust choice for developing conversational services
 in the evolving landscape of Python applications and IoT connectivity.
 
-## Why choose DFF
+## Why choose Chatsky
 
 * Written in pure Python, the framework is easily accessible for both beginners and experienced developers.
-* For the same reason, all the abstractions used in DFF can be easily customized and extended using regular language synthax.
-* DFF offers easy and straightforward tools for state management which is as easy as setting values of a Python dictionary.
+* For the same reason, all the abstractions used in Chatsky can be easily customized and extended using regular language synthax.
+* Chatsky offers easy and straightforward tools for state management which is as easy as setting values of a Python dictionary.
 * The framework is being actively maintained and thoroughly tested. The team is open to suggestions and quickly reacts to bug reports.
 
 # Quick Start
@@ -44,50 +44,50 @@ in the evolving landscape of Python applications and IoT connectivity.
 
 ## Installation
 
-DFF can be installed via pip:
+Chatsky can be installed via pip:
 
 ```bash
-pip install dff
+pip install chatsky
 ```
 
-The above command will set the minimum dependencies to start working with DFF. 
+The above command will set the minimum dependencies to start working with Chatsky. 
 The installation process allows the user to choose from different packages based on their dependencies, which are:
 ```bash
-pip install dff[json]  # dependencies for using JSON
-pip install dff[pickle] # dependencies for using Pickle
-pip install dff[redis]  # dependencies for using Redis
-pip install dff[mongodb]  # dependencies for using MongoDB
-pip install dff[mysql]  # dependencies for using MySQL
-pip install dff[postgresql]  # dependencies for using PostgreSQL
-pip install dff[sqlite]  # dependencies for using SQLite
-pip install dff[ydb]  # dependencies for using Yandex Database
-pip install dff[telegram]  # dependencies for using Telegram
-pip install dff[benchmark]  # dependencies for benchmarking
+pip install chatsky[json]  # dependencies for using JSON
+pip install chatsky[pickle] # dependencies for using Pickle
+pip install chatsky[redis]  # dependencies for using Redis
+pip install chatsky[mongodb]  # dependencies for using MongoDB
+pip install chatsky[mysql]  # dependencies for using MySQL
+pip install chatsky[postgresql]  # dependencies for using PostgreSQL
+pip install chatsky[sqlite]  # dependencies for using SQLite
+pip install chatsky[ydb]  # dependencies for using Yandex Database
+pip install chatsky[telegram]  # dependencies for using Telegram
+pip install chatsky[benchmark]  # dependencies for benchmarking
 ```
 
 For example, if you are going to use one of the database backends,
 you can specify the corresponding requirements yourself. Multiple dependencies can be installed at once, e.g.
 ```bash
-pip install dff[postgresql,mysql]
+pip install chatsky[postgresql,mysql]
 ```
 
 ## Basic example
 
 The following code snippet builds a simplistic chat bot that replies with messages
 ``Hi!`` and ``OK`` depending on user input, which only takes a few lines of code.
 All the abstractions used in this example are thoroughly explained in the dedicated
-[user guide](https://deeppavlov.github.io/dialog_flow_framework/user_guides/basic_conceptions.html).
+[user guide](https://deeppavlov.github.io/chatsky/user_guides/basic_conceptions.html).
 
 ```python
-from dff.script import GLOBAL, TRANSITIONS, RESPONSE, Message
-from dff.pipeline import Pipeline
-import dff.script.conditions.std_conditions as cnd
+from chatsky.script import GLOBAL, TRANSITIONS, RESPONSE, Message
+from chatsky.pipeline import Pipeline
+import chatsky.script.conditions.std_conditions as cnd
 
 # create a dialog script
 script = {
     GLOBAL: {
         TRANSITIONS: {
-            ("flow", "node_hi"): cnd.exact_match(Message("Hi")),
+            ("flow", "node_hi"): cnd.exact_match("Hi"),
             ("flow", "node_ok"): cnd.true()
         }
     },
@@ -129,19 +129,19 @@ Response:  OK
 ```
 
 More advanced examples are available as a part of documentation:
-[tutorials](https://deeppavlov.github.io/dialog_flow_framework/tutorials.html).
+[tutorials](https://deeppavlov.github.io/chatsky/tutorials.html).
 
 ## Further steps
 
-To further explore the API of the framework, you can make use of the [detailed documentation](https://deeppavlov.github.io/dialog_flow_framework/index.html). 
-Broken down into several sections to highlight all the aspects of development with DFF,
+To further explore the API of the framework, you can make use of the [detailed documentation](https://deeppavlov.github.io/chatsky/index.html). 
+Broken down into several sections to highlight all the aspects of development with Chatsky,
 the documentation for the library is constantly available online.
 
-# Contributing to the Dialog Flow Framework
+# Contributing to Chatsky
 
 We are open to accepting pull requests and bug reports.
-Please refer to [CONTRIBUTING.md](https://github.com/deeppavlov/dialog_flow_framework/blob/master/CONTRIBUTING.md).
+Please refer to [CONTRIBUTING.md](https://github.com/deeppavlov/chatsky/blob/master/CONTRIBUTING.md).
 
 # License
 
-DFF is distributed under the terms of the [Apache License 2.0](https://github.com/deeppavlov/dialog_flow_framework/blob/master/LICENSE).
+Chatsky is distributed under the terms of the [Apache License 2.0](https://github.com/deeppavlov/chatsky/blob/master/LICENSE).

dff/__init__.py → chatsky/__init__.py

@@ -10,7 +10,7 @@
 
 nest_asyncio.apply()
 
-from dff.pipeline import Pipeline
-from dff.script import Context, Script
+from chatsky.pipeline import Pipeline
+from chatsky.script import Context, Script
 
-Script.model_rebuild()
+import chatsky.__rebuild_pydantic_models__

chatsky/__rebuild_pydantic_models__.py

@@ -0,0 +1,9 @@
+# flake8: noqa: F401
+
+from chatsky.pipeline import Pipeline
+from chatsky.pipeline.types import ExtraHandlerRuntimeInfo
+from chatsky.script import Context, Script
+
+Script.model_rebuild()
+Context.model_rebuild()
+ExtraHandlerRuntimeInfo.model_rebuild()

dff/config/README.md → chatsky/config/README.md

@@ -5,6 +5,6 @@
 This directory provides yaml files for Superset dashboard configuration.
 The files inside are not supposed to be edited manually for lest of compatibility breaks.
 Placeholders inside the files will be filled automatically when you use the 
-`dff.stats` CLI command to generate a configuration archive.
+`chatsky.stats` CLI command to generate a configuration archive.
 
-Use `dff.stats -h` for more info.
+Use `chatsky.stats -h` for more info.

dff/config/superset_dashboard/dashboards/DFF_statistics_dashboard_1.yaml → chatsky/config/superset_dashboard/dashboards/chatsky_statistics_dashboard_1.yaml

@@ -1,7 +1,7 @@
-dashboard_title: DFF statistics dashboard
+dashboard_title: Chatsky statistics dashboard
 description: null
 css: ''
-slug: dff-stats
+slug: chatsky-stats
 uuid: 68bce374-99bc-4890-b8c2-cb172409b894
 position:
   CHART-91whs_IaiF:
@@ -297,7 +297,7 @@ position:
   HEADER_ID:
     id: HEADER_ID
     meta:
-      text: DFF statistics dashboard
+      text: Chatsky statistics dashboard
     type: HEADER
   MARKDOWN-8Q9BhcEwva:
     children: []
@@ -378,7 +378,7 @@ position:
 
 
         To make them available in the dashboard, you need to define a custom extractor
-        function for them (see the [Extractor functions](https://deeppavlov.github.io/dialog_flow_framework/tutorials/tutorials.stats.1_extractor_functions.html) ).
+        function for them (see the [Extractor functions](https://deeppavlov.github.io/chatsky/tutorials/tutorials.stats.1_extractor_functions.html) ).
         The output of that function will then be persisted to the `data` column of
         the logs table, while the name of the function will be available in the `data
         key` column. That makes it easy to filter the relevant log entries and use
@@ -621,7 +621,7 @@ position:
         ## Service users
 
 
-        This plot aggregates the count of unique users querying the DFF service at
+        This plot aggregates the count of unique users querying the Chatsky service at
         any given point in time. The time periods to aggregate over can be changed
         using the filter on the left.