Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Telegram interface upgrade #328

Merged
merged 163 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
163 commits
Select commit Hold shift + click to select a range
ae091ab
new telegram message proposal
pseusys Jan 22, 2024
b1c4da9
only required attachments remain
pseusys Jan 22, 2024
86c15c2
keyboard implemented
pseusys Jan 23, 2024
a0a859c
tests reworked
pseusys Jan 29, 2024
89fa2c7
telegram examples debugged
pseusys Jan 31, 2024
3745cff
optional attachment processing added
pseusys Jan 31, 2024
26dc2d6
download all removed, special downloading method created
pseusys Feb 4, 2024
7e65068
callback query added as an attachment
pseusys Feb 7, 2024
efece76
callback query conditions added
pseusys Feb 7, 2024
c947b4e
remove tg tests; move message tests to core tests
RLKRo Feb 22, 2024
25b8ac9
buttons pulled out of attachments list
pseusys Feb 28, 2024
a6c1172
merge
RLKRo Mar 7, 2024
b67b497
some errors fixed
pseusys Mar 12, 2024
68b4a37
review issues fixed
pseusys Mar 18, 2024
efde7c3
Merge remote-tracking branch 'origin/dev' into feat/telegram_interfac…
pseusys Mar 18, 2024
f9a9d79
Merge branch 'dev' into feat/telegram_interface_upgrade
pseusys Mar 18, 2024
6f8de0b
lock file updated
pseusys Mar 18, 2024
771370d
typing fixed
pseusys Mar 18, 2024
1c85052
circular import resolved
pseusys Mar 18, 2024
0cebb97
one more attempt to fix circular import
pseusys Mar 18, 2024
a5f0e02
populate attachment method made implemented
pseusys Mar 19, 2024
01947dd
CLI interface separated
pseusys Mar 19, 2024
5e35935
data message equation fixed
pseusys Mar 19, 2024
513f77a
lint applied
pseusys Mar 19, 2024
48bb29f
move callback query to script.conditions
RLKRo Mar 20, 2024
e2ea371
condition signature fix
pseusys Mar 21, 2024
ea66537
messenger interface parameter added to `get_bytes` method
pseusys Mar 21, 2024
db1de9a
async def removed from `extract_message_from_telegram` function
pseusys Mar 21, 2024
19d00f3
docs fixed
pseusys Mar 21, 2024
11cc1a3
condition import added
pseusys Mar 21, 2024
66f82b1
two attachment tutorials fixed
pseusys Mar 22, 2024
2773c9d
attachment tuples changed to sets
pseusys Mar 22, 2024
80fceb5
lint applied
pseusys Mar 22, 2024
a5fbe60
docs links fixed
pseusys Mar 22, 2024
8e970c9
Merge branch 'dev' into feat/telegram_interface_upgrade
pseusys Mar 27, 2024
3f815e7
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys Mar 27, 2024
f8cecdf
telegram formatted
pseusys Mar 27, 2024
95019ad
lock update
pseusys Mar 27, 2024
8c90763
tuples replaced with sets everywhere
pseusys Mar 27, 2024
30db35a
telegram-specific request attachments added
pseusys Apr 4, 2024
1432893
attachment subclasses moved to extras
pseusys Apr 9, 2024
1bfe524
extras added
pseusys Apr 10, 2024
82e7a3e
lint fixed
pseusys Apr 11, 2024
334cd74
ui replaced with keyboard
pseusys Apr 16, 2024
fb2e90a
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys Apr 16, 2024
bc406f6
"check_if_" removed
pseusys Apr 16, 2024
b08e8f3
undefined message removed
pseusys Apr 16, 2024
c79a732
unnecessary type casting removed
pseusys Apr 16, 2024
7853dbc
args and kwargs removed
pseusys Apr 16, 2024
5b9992f
default poll option voter count added
pseusys Apr 16, 2024
a3beac9
telegram import check added
pseusys Apr 16, 2024
5dd0751
interface attachments sorted
pseusys Apr 16, 2024
b951772
temp attachment dir warning added
pseusys Apr 16, 2024
fe99c4c
attachment message clarified
pseusys Apr 16, 2024
595d55d
single attachment sending proposal + sticker
pseusys Apr 18, 2024
b31ea18
request and response attachments sorted
pseusys Apr 18, 2024
c882abf
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys Apr 18, 2024
dd57886
attachments tutorial added
pseusys Apr 22, 2024
33c72cb
advanced tutorial added
pseusys Apr 23, 2024
17280e3
`cached_filename` field added
pseusys Apr 26, 2024
2a1f01a
Tutorial texts altered.
pseusys Apr 26, 2024
fb58bda
happy path removed, use cache flag added
pseusys Apr 26, 2024
a2e0ca3
literals added + tests started
pseusys May 1, 2024
a07b73b
Attachment links updated
pseusys May 7, 2024
87001c0
new_attachment_tests_added
pseusys May 7, 2024
aabb3f8
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys May 7, 2024
b4dd7fc
Merge branch 'dev' into feat/telegram_interface_upgrade
pseusys May 7, 2024
f478972
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys May 7, 2024
f1a140a
parametrize fixed for telegram testing
pseusys May 7, 2024
3b04c03
Some lint fixed
pseusys May 7, 2024
4db3d6f
file path fixed
pseusys May 7, 2024
74a0402
attachments type fixed
pseusys May 8, 2024
9bc4b3c
supported attachment sets renamed
pseusys May 8, 2024
cb28fe3
final message test fixed
pseusys May 8, 2024
d30358b
dot path used instead of slash
pseusys May 8, 2024
eaad6d2
tutorial tests fixed
pseusys May 8, 2024
c77f41f
fixed getattribute stack overflow exception
pseusys May 8, 2024
28d7f8a
true transition fixed
pseusys May 8, 2024
786b3c7
async function marked
pseusys May 8, 2024
2277af7
cache dir clear and create reversed
pseusys May 8, 2024
6e218b6
pytest warning fixed
pseusys May 8, 2024
e6d1226
tutorial testing done
pseusys May 21, 2024
eb07fae
last test step added
pseusys May 24, 2024
97be6b9
attachments migrated to wiki
pseusys May 24, 2024
33a5c73
pragma removed, inheritance added
pseusys May 27, 2024
140f014
some review parts fixed
pseusys May 27, 2024
47610a3
venv ignore added back
pseusys May 27, 2024
0362fc8
some review comments updated
pseusys Jun 3, 2024
07d887e
erview notes taken into account
pseusys Jun 5, 2024
121b092
additional `Message` serialization test introduced
pseusys Jun 5, 2024
5172f0c
extra field verification added
pseusys Jun 7, 2024
27205bb
github action fixes #1
pseusys Jun 7, 2024
0de3e2b
github action fixes N2
pseusys Jun 7, 2024
83cb8a0
github action fixes N3
pseusys Jun 7, 2024
27678f5
misc typing fixed
pseusys Jun 7, 2024
f88e454
Attachment name changed (for windows)
pseusys Jun 7, 2024
85487c3
typing.Annotated fixed
pseusys Jun 7, 2024
e634436
Naming fixed once again
pseusys Jun 7, 2024
5ca6e20
telegram import in tests fixed
pseusys Jun 7, 2024
209d975
telegram import fix (again)
pseusys Jun 7, 2024
2f67816
PathStep quotes added
pseusys Jun 7, 2024
b61c42d
aiofiles dependency fixed
pseusys Jun 8, 2024
cc5772e
typing fixed
pseusys Jun 9, 2024
ae35fb9
naming error fixed
pseusys Jun 10, 2024
a1df33f
ok, we'll just use any I guess
pseusys Jun 10, 2024
d6fbaaf
telegram skip setup correctly
pseusys Jun 10, 2024
4416801
telegram import guarding fixed
pseusys Jun 10, 2024
b4d9942
Docs for new DFF classes and functions
pseusys Jun 18, 2024
4cc00b7
tests fixed and updated
pseusys Jun 19, 2024
d74a2bc
telegram interfaces renamed
pseusys Jun 19, 2024
de12ca2
tutorial documentation introduced
pseusys Jun 19, 2024
eeb1bf7
lockfile updated
pseusys Jun 19, 2024
3f0b86e
Merge branch 'dev' into feat/telegram_interface_upgrade
pseusys Jun 19, 2024
21d5af2
locked
pseusys Jun 19, 2024
eef4498
dummy type annotation added
pseusys Jun 20, 2024
25ed6b2
field doclink fixed
pseusys Jun 20, 2024
3aa7046
lint fixed partly
pseusys Jun 20, 2024
0c12ad2
formatting fixed
pseusys Jun 21, 2024
652453c
Once again string concatenation fixed
pseusys Jun 21, 2024
a50fabe
once again: reformatting
pseusys Jun 21, 2024
e55d801
one whitespace removed
pseusys Jun 21, 2024
23ee7f2
json dump fixed
pseusys Jun 21, 2024
f82d4ed
import error check added
pseusys Jun 21, 2024
c6cd39c
non-native hashes used
pseusys Jun 22, 2024
e95d236
hashing description fixed
pseusys Jun 22, 2024
d02fa87
log level for tmpdir attachment_dir set to info
RLKRo Jun 24, 2024
680d6b6
add abstract class MessengerInterfaceWithAttachments
RLKRo Jun 24, 2024
d8ad2f1
revert init call changes
RLKRo Jun 24, 2024
386f028
return venv ignores
RLKRo Jun 24, 2024
1bd7fff
fix typo
RLKRo Jun 24, 2024
39ca58c
doc improvements
RLKRo Jun 24, 2024
4415606
json-serialization improvements
RLKRo Jun 25, 2024
fc98008
patch TG_BOT_TOKEN for tutorial tests
RLKRo Jun 25, 2024
4f49b99
remove pytest.mark.telegram
RLKRo Jun 25, 2024
1765a33
lint
RLKRo Jun 25, 2024
85e84a4
standardize main block in tutorials
RLKRo Jun 25, 2024
2412ba8
merge utils as development utils
RLKRo Jun 25, 2024
1602242
fix test data
RLKRo Jun 25, 2024
7e1d77c
fix MessengerInterfaceWithAttachments usage in tests
RLKRo Jun 25, 2024
dfbccf7
cached filename check added
pseusys Jun 26, 2024
037a5d3
telegram API connections updated
pseusys Jun 26, 2024
4313477
media group attachment introduced
pseusys Jun 26, 2024
7b894c7
pickle validator parameter removed
pseusys Jun 26, 2024
64fbef0
Type definition sorted
pseusys Jun 26, 2024
2042186
mediagroup tests added
pseusys Jun 27, 2024
d6dfccd
lint fixed
pseusys Jun 27, 2024
b1131ef
message send order updated
pseusys Jun 27, 2024
2adb069
make attachment class abstract
RLKRo Jun 27, 2024
f20b99c
media group improvements
RLKRo Jun 27, 2024
06f0dcd
rename tutorial tests
RLKRo Jun 27, 2024
98dbd37
add filename to attachments in tutorial
RLKRo Jun 27, 2024
5cad29a
simplify second tutorial with const attachment list
RLKRo Jun 27, 2024
91094a5
minor tutorial changes
RLKRo Jun 27, 2024
2b02c80
improve json-pickle serialization utils
RLKRo Jun 28, 2024
3da516b
allow arbitrary DataAttachments in MediaGroup
RLKRo Jun 28, 2024
ed58bdc
add tool for generating test_happy_paths.json
RLKRo Jun 29, 2024
eb106c1
minor doc improvements
RLKRo Jul 1, 2024
46c742d
replace populate_attachment with get_attachment_bytes
RLKRo Jul 1, 2024
b32492d
test DataAttachment.source type conversions
RLKRo Jul 2, 2024
9adeccd
remove title field
RLKRo Jul 2, 2024
e3d154e
replace cached_filename type annotation to Path
RLKRo Jul 2, 2024
d33e3f3
remove unused features
RLKRo Jul 2, 2024
365a301
patch attachment comparison to support different OSes
RLKRo Jul 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion dff/messengers/common/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# -*- coding: utf-8 -*-

from .interface import MessengerInterface, PollingMessengerInterface, CallbackMessengerInterface, CLIMessengerInterface
from .interface import (
MessengerInterface,
MessengerInterfaceWithAttachments,
PollingMessengerInterface,
CallbackMessengerInterface,
)
from .types import PollingInterfaceLoopFunction
102 changes: 56 additions & 46 deletions dff/messengers/common/interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,15 @@
import abc
import asyncio
import logging
import uuid
from typing import Optional, Any, List, Tuple, TextIO, Hashable, TYPE_CHECKING

from dff.script import Context, Message
from dff.messengers.common.types import PollingInterfaceLoopFunction
from pathlib import Path
from tempfile import gettempdir
from typing import Optional, Any, List, Tuple, Hashable, TYPE_CHECKING, Type

if TYPE_CHECKING:
from dff.script import Context, Message
from dff.pipeline.types import PipelineRunnerFunction
from dff.messengers.common.types import PollingInterfaceLoopFunction
from dff.script.core.message import Attachment

logger = logging.getLogger(__name__)

Expand All @@ -39,6 +40,55 @@ async def connect(self, pipeline_runner: PipelineRunnerFunction):
raise NotImplementedError


class MessengerInterfaceWithAttachments(MessengerInterface, abc.ABC):
"""
MessengerInterface subclass that has methods for attachment handling.

:param attachments_directory: Directory where attachments will be stored.
If not specified, the temporary directory will be used.
"""

supported_request_attachment_types: set[Type[Attachment]] = set()
"""
Types of attachment that this messenger interface can receive.
Attachments not in this list will be neglected.
"""

supported_response_attachment_types: set[Type[Attachment]] = set()
"""
Types of attachment that this messenger interface can send.
Attachments not in this list will be neglected.
"""

def __init__(self, attachments_directory: Optional[Path] = None) -> None:
pseusys marked this conversation as resolved.
Show resolved Hide resolved
tempdir = gettempdir()
if attachments_directory is not None and not str(attachments_directory.absolute()).startswith(tempdir):
self.attachments_directory = attachments_directory
else:
warning_start = f"Attachments directory for {type(self).__name__} messenger interface"
warning_end = "attachment data won't be cached locally!"
if attachments_directory is None:
self.attachments_directory = Path(tempdir) / f"dff-cache-{type(self).__name__}"
logger.info(f"{warning_start} is None, so will be set to tempdir and {warning_end}")
else:
self.attachments_directory = attachments_directory
logger.info(f"{warning_start} is in tempdir, so {warning_end}")
self.attachments_directory.mkdir(parents=True, exist_ok=True)

@abc.abstractmethod
async def get_attachment_bytes(self, source: str) -> bytes:
"""
Get attachment bytes from file source.

E.g. if a file attachment consists of a URL of the file uploaded to the messenger servers,
this method is the right place to call the messenger API for the file downloading.

:param source: Identifying string for the file.
:return: The attachment bytes.
"""
raise NotImplementedError


class PollingMessengerInterface(MessengerInterface):
"""
Polling message interface runs in a loop, constantly asking users for a new input.
Expand Down Expand Up @@ -119,7 +169,7 @@ class CallbackMessengerInterface(MessengerInterface):
Callback message interface is waiting for user input and answers once it gets one.
"""

def __init__(self):
def __init__(self) -> None:
self._pipeline_runner: Optional[PipelineRunnerFunction] = None

async def connect(self, pipeline_runner: PipelineRunnerFunction):
Expand All @@ -142,43 +192,3 @@ def on_request(
This method has the same signature as :py:class:`~dff.pipeline.types.PipelineRunnerFunction`.
"""
return asyncio.run(self.on_request_async(request, ctx_id, update_ctx_misc))


class CLIMessengerInterface(PollingMessengerInterface):
"""
Command line message interface is the default message interface, communicating with user via `STDIN/STDOUT`.
This message interface can maintain dialog with one user at a time only.
"""

def __init__(
self,
intro: Optional[str] = None,
prompt_request: str = "request: ",
prompt_response: str = "response: ",
out_descriptor: Optional[TextIO] = None,
):
super().__init__()
self._ctx_id: Optional[Hashable] = None
self._intro: Optional[str] = intro
self._prompt_request: str = prompt_request
self._prompt_response: str = prompt_response
self._descriptor: Optional[TextIO] = out_descriptor

def _request(self) -> List[Tuple[Message, Any]]:
return [(Message(input(self._prompt_request)), self._ctx_id)]

def _respond(self, responses: List[Context]):
print(f"{self._prompt_response}{responses[0].last_response.text}", file=self._descriptor)

async def connect(self, pipeline_runner: PipelineRunnerFunction, **kwargs):
"""
The CLIProvider generates new dialog id used to user identification on each `connect` call.

:param pipeline_runner: A function that should process user request and return context;
usually it's a :py:meth:`~dff.pipeline.pipeline.pipeline.Pipeline._run_pipeline` function.
:param \\**kwargs: argument, added for compatibility with super class, it shouldn't be used normally.
"""
self._ctx_id = uuid.uuid4()
if self._intro is not None:
print(self._intro)
await super().connect(pipeline_runner, **kwargs)
49 changes: 49 additions & 0 deletions dff/messengers/console.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
from typing import Any, Hashable, List, Optional, TextIO, Tuple
from uuid import uuid4
from dff.messengers.common.interface import PollingMessengerInterface
from dff.pipeline.types import PipelineRunnerFunction
from dff.script.core.context import Context
from dff.script.core.message import Message


class CLIMessengerInterface(PollingMessengerInterface):
"""
Command line message interface is the default message interface, communicating with user via `STDIN/STDOUT`.
This message interface can maintain dialog with one user at a time only.
"""

supported_request_attachment_types = set()
supported_response_attachment_types = set()

def __init__(
self,
intro: Optional[str] = None,
prompt_request: str = "request: ",
prompt_response: str = "response: ",
out_descriptor: Optional[TextIO] = None,
):
super().__init__()
self._ctx_id: Optional[Hashable] = None
self._intro: Optional[str] = intro
self._prompt_request: str = prompt_request
self._prompt_response: str = prompt_response
self._descriptor: Optional[TextIO] = out_descriptor

def _request(self) -> List[Tuple[Message, Any]]:
return [(Message(input(self._prompt_request)), self._ctx_id)]

def _respond(self, responses: List[Context]):
print(f"{self._prompt_response}{responses[0].last_response.text}", file=self._descriptor)

async def connect(self, pipeline_runner: PipelineRunnerFunction, **kwargs):
"""
The CLIProvider generates new dialog id used to user identification on each `connect` call.

:param pipeline_runner: A function that should process user request and return context;
usually it's a :py:meth:`~dff.pipeline.pipeline.pipeline.Pipeline._run_pipeline` function.
:param \\**kwargs: argument, added for compatibility with super class, it shouldn't be used normally.
"""
self._ctx_id = uuid4()
if self._intro is not None:
print(self._intro)
await super().connect(pipeline_runner, **kwargs)
14 changes: 2 additions & 12 deletions dff/messengers/telegram/__init__.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,4 @@
# -*- coding: utf-8 -*-

try:
import telebot
except ImportError:
raise ImportError("telebot is not installed. Run `pip install dff[telegram]`")

from .messenger import TelegramMessenger
from .interface import PollingTelegramInterface, CallbackTelegramInterface
from .message import TelegramUI, TelegramMessage, RemoveKeyboard, ParseMode
from .messenger import (
UpdateType,
telegram_condition,
)
from .abstract import telegram_available
from .interface import LongpollingInterface, WebhookInterface
Loading
Loading