Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Messages with multiple attachments or parts display out of order #3

Open
sabogalc opened this issue May 3, 2021 · 9 comments
Labels
bug Something isn't working

Comments

@sabogalc
Copy link

sabogalc commented May 3, 2021

The message I sent with the picture of Thanos and the text caption were sent from an iOS device.
Thanos AirMessage
Thanos iOS
I have pasted Tagavari's question from my reddit post, along with my answer.

Is that image and text content part of the same message (rather than being sent as 2 separate messages)? When provided with a mixed message, AirMessage will by default put the text content above attachments, though Apple Messages puts it below. This could be changed fairly easily. However, if these are 2 separate messages, then this would be a sorting issue instead that would probably take a bit more work to resolve.

It is the same message/a mixed message.

Is the Protocol 5.2 sorting parameter useful for this issue, or no?
u-Tagavari

This particular photo of Thanos has been an issue that I have reported on quite a lot.
See here, here, here, here, and here.

{
  "id": "D3DBDAD8-5EEE-4D72-ADB7-20A74501706D",
  "fromMe": true,
  "isTypingMessage": false,
  "description": "We'll show up and he'll be like",
  "flags": 1081349,
  "fileTransferIDs": [
    "72D48466-A535-4C93-AA78-6791A8AB53F1"
  ],
  "chatID": "chat526745548997352517",
  "time": 1615577622383.0002,
  "timeRead": 0,
  "isSOS": false,
  "timeDelivered": 0,
  "isAudioMessage": false,
  "isCancelTypingMessage": false,
  "items": [
    {
      "type": "attachment",
      "payload": {
        "metadata": {
          "size": {
            "width": 500,
            "height": 255
          },
          "id": "72D48466-A535-4C93-AA78-6791A8AB53F1",
          "filename": "AE5727AE-29A7-4E84-B537-17AFD35FEB41.gif",
          "uti": "com.compuserve.gif",
          "mime": "image/gif"
        },
        "id": "p:0/D3DBDAD8-5EEE-4D72-ADB7-20A74501706D",
        "chatID": "chat526745548997352517",
        "transferID": "72D48466-A535-4C93-AA78-6791A8AB53F1",
        "fromMe": true,
        "time": 1615577622383.0002
      }
    },
    {
      "type": "text",
      "payload": {
        "chatID": "chat526745548997352517",
        "fromMe": true,
        "parts": [
          {
            "type": "text",
            "string": "We'll show up and he'll be like"
          }
        ],
        "id": "p:1/D3DBDAD8-5EEE-4D72-ADB7-20A74501706D",
        "time": 1615577622383.0002,
        "text": "We'll show up and he'll be like",
        "acknowledgments": [
          {
            "chatID": "chat526745548997352517",
            "sender": "*my friend's email*",
            "associatedID": "p:1/D3DBDAD8-5EEE-4D72-ADB7-20A74501706D",
            "id": "am:F0716C13-6871-4286-A4A6-3CF3FB0B25CD",
            "fromMe": false,
            "time": 1615577636093.2012,
            "acknowledgmentType": 2003
          }
        ]
      }
    }
  ],
  "service": "iMessage",
  "timePlayed": 0,
  "sender": "e:*my email*",
  "isDelivered": false
}
@tagavari tagavari added the bug Something isn't working label May 3, 2021
@sabogalc
Copy link
Author

sabogalc commented May 3, 2021

Here is another example with the same conversation on the 3.1.5 Android app and an iOS 14.5 device
20210503_181141
PNG image

@tagavari
Copy link
Member

tagavari commented May 3, 2021

Do you know of a way to consistently reproduce this bug?

@sabogalc
Copy link
Author

sabogalc commented May 3, 2021

I would do so by sending an attachment together with a text caption directly from an iOS or macOS device. Afterwards, receive a tapback on either the attachment, caption, or both, and it should show up differently in AirMessage than it does on native iMessage.

@tagavari
Copy link
Member

tagavari commented May 3, 2021

Thanks, I'll take a look at this tomorrow

@tagavari
Copy link
Member

tagavari commented May 4, 2021

Okay, so one issue is that particularly on macOS, you're able to insert attachments at any point within the message and have the text split accordingly. However, I'm not sure how this is accounted for in the messages database.

@sabogalc
Copy link
Author

sabogalc commented May 4, 2021

That's a good point. You can do this on iOS too if you paste in the attachments instead of actually, well, attaching them. I hadn't considered that use case.

@sabogalc
Copy link
Author

sabogalc commented May 16, 2021

Okay, so one issue is that particularly on macOS, you're able to insert attachments at any point within the message and have the text split accordingly. However, I'm not sure how this is accounted for in the messages database.

Regarding this comment, I ran a test that mimicked this behavior on iOS. Below is the conversation on iOS
iOS
and in AirMessage
AirMessage
I tested this in all the other iMessage forwarding apps that I know of, and the only one that got it right was MyMessage.
MyMessage
I will make similar comments for the other apps that experience similar bugs, and I will also ask Eric how he sorts messages in these kinds of instances.

UPDATE - Here is some message information that I got from MyMessage

{
  "id": "28935DBA-A57E-485E-9442-10FBAD3A3C95",
  "fromMe": true,
  "isTypingMessage": false,
  "description": "Here is a message that has images intermittently scattered throughout it",
  "flags": 1085445,
  "fileTransferIDs": [
    "49CA2847-DF32-4D66-9857-7163EE1FCE49",
    "9C606BB6-2443-4101-A55A-8E2610D08988",
    "00A6CB41-165B-43B8-881F-59D725515B36"
  ],
  "chatID": "*recipient*",
  "time": 1620776404411,
  "timeRead": 0,
  "isSOS": false,
  "timeDelivered": 1620776405263.5593,
  "isAudioMessage": false,
  "isCancelTypingMessage": false,
  "items": [
    {
      "type": "text",
      "payload": {
        "id": "p:0/28935DBA-A57E-485E-9442-10FBAD3A3C95",
        "chatID": "*recipient*",
        "parts": [
          {
            "type": "text",
            "string": "Here is a message"
          }
        ],
        "fromMe": true,
        "time": 1620776404411,
        "text": "Here is a message"
      }
    },
    {
      "type": "attachment",
      "payload": {
        "metadata": {
          "size": {
            "width": 200,
            "height": 200
          },
          "id": "49CA2847-DF32-4D66-9857-7163EE1FCE49",
          "filename": "IMG_1752.png",
          "uti": "public.png",
          "mime": "image/png"
        },
        "id": "p:1/28935DBA-A57E-485E-9442-10FBAD3A3C95",
        "chatID": "*recipient*",
        "transferID": "49CA2847-DF32-4D66-9857-7163EE1FCE49",
        "fromMe": true,
        "time": 1620776404411
      }
    },
    {
      "type": "text",
      "payload": {
        "id": "p:2/28935DBA-A57E-485E-9442-10FBAD3A3C95",
        "chatID": "*recipient*",
        "parts": [
          {
            "type": "text",
            "string": "that has images"
          }
        ],
        "fromMe": true,
        "time": 1620776404411,
        "text": "that has images"
      }
    },
    {
      "type": "attachment",
      "payload": {
        "metadata": {
          "size": {
            "width": 200,
            "height": 200
          },
          "id": "9C606BB6-2443-4101-A55A-8E2610D08988",
          "filename": "IMG_3897.png",
          "uti": "public.png",
          "mime": "image/png"
        },
        "id": "p:3/28935DBA-A57E-485E-9442-10FBAD3A3C95",
        "chatID": "*recipient*",
        "transferID": "9C606BB6-2443-4101-A55A-8E2610D08988",
        "fromMe": true,
        "time": 1620776404411
      }
    },
    {
      "type": "text",
      "payload": {
        "id": "p:4/28935DBA-A57E-485E-9442-10FBAD3A3C95",
        "chatID": "*recipient*",
        "parts": [
          {
            "type": "text",
            "string": "intermittently scattered"
          }
        ],
        "fromMe": true,
        "time": 1620776404411,
        "text": "intermittently scattered"
      }
    },
    {
      "type": "attachment",
      "payload": {
        "metadata": {
          "size": {
            "width": 200,
            "height": 200
          },
          "id": "00A6CB41-165B-43B8-881F-59D725515B36",
          "filename": "IMG_4390.png",
          "uti": "public.png",
          "mime": "image/png"
        },
        "id": "p:5/28935DBA-A57E-485E-9442-10FBAD3A3C95",
        "chatID": "*recipient*",
        "transferID": "00A6CB41-165B-43B8-881F-59D725515B36",
        "fromMe": true,
        "time": 1620776404411
      }
    },
    {
      "type": "text",
      "payload": {
        "id": "p:6/28935DBA-A57E-485E-9442-10FBAD3A3C95",
        "chatID": "*recipient*",
        "parts": [
          {
            "type": "text",
            "string": "throughout it"
          }
        ],
        "fromMe": true,
        "time": 1620776404411,
        "text": "throughout it"
      }
    }
  ],
  "service": "iMessage",
  "timePlayed": 0,
  "sender": "*me*",
  "isDelivered": true
}

@tneotia
Copy link

tneotia commented Dec 13, 2022

As per @CSab6482 's request, the way to solve this is to deserialize attributedBody (which you've already done) and look at each component inside its runs. You can see our code here: https://github.com/BlueBubblesApp/bluebubbles-app/blob/development/lib/services/ui/message/message_widget_controller.dart#L121-L154

The attributedBody also provides data for mentions within texts.

@tagavari
Copy link
Member

As per @CSab6482 's request, the way to solve this is to deserialize attributedBody (which you've already done) and look at each component inside its runs. You can see our code here: https://github.com/BlueBubblesApp/bluebubbles-app/blob/development/lib/services/ui/message/message_widget_controller.dart#L121-L154

The attributedBody also provides data for mentions within texts.

This is helpful, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants