api: Stop assuming fetched messages' reactions have deprecated `user` field #5773

chrisbobbe · 2023-10-02T20:53:32Z

Our code at the edge for transforming the old format of reactions,
with user, into the new format, with user_id, was depending on
user always being present.

Since user_id is documented as being added in Zulip 3.0:
https://zulip.com/api/get-messages#response
, and since #5747 we've been refusing to connect to servers pre-3.0,
we can assume user_id is always present and stop reading user.

Fixes: #4072

gnprice

Thanks for taking care of this! Comments below.

gnprice · 2023-10-02T21:06:25Z

src/utils/avatar.js

      ensureUnreachable(rawAvatarUrl);
      const msg = 'Unexpected value for `rawAvatarUrl` in `AvatarURL.fromUserOrBotData`';
-      logging.error(msg, { value: rawAvatarUrl });
      throw new Error(msg);


avatar: Remove redundant logging.error when we're already throwing an error This would produce log noise in the Jest tests in an upcoming commit.

Hmm, how do we end up reaching this code in the tests? The ensureUnreachable means the type-checker should verify it can't be reached given our types, and we generally write our tests to be well-typed.

I think the reason the logging.error is there in addition to the throw is that it adds the extra data of what the unexpected rawAvatarUrl was; that's what makes it not redundant with the throw. Happy to remove it if it's getting in the way somehow, though — it looks like we see zero cases of this error in Sentry.

(Answered below: #5773 (comment))

gnprice · 2023-10-02T21:09:40Z

src/api/rawModelTypes.js

-    const { user, ...restReaction } = reaction;
-    return {
-      ...restReaction,
-      user_id: user.id,


Oof. Yeah, what we should have done long ago would be to make this say user.id ?? restReaction.user_id (or perhaps, equivalently, user.id ?? user_id after saying const { user_id, user, ...restReaction } above… or perhaps better yet, write the fallback the other way around with modern first, like user_id ?? user.id).

Then it would have already been forward-compatible with an eventual server version that dropped the old user property, without really being materially more complicated.

gnprice · 2023-10-02T21:16:01Z

src/message/__tests__/fetchActions-test.js

-          // Flow would complain at `faultyReaction` if it
+          // Flow would complain at `avatar_url` if it
          // type-checked `response`, but we should ignore it if that
-          // day comes. It's badly shaped on purpose.
-          messages: [{ ...fetchedMessage1, reactions: [faultyReaction] }],
+          // day comes. It's badly typed on purpose.
+          messages: [{ ...fetchedMessage1, avatar_url: 123 }],


Ah, I see now the answer to my first question (in utils/avatar.js).

gnprice · 2023-10-02T21:26:59Z

src/message/__tests__/fetchActions-test.js

-        // Simulate #4156, a real-life problem that a user at server commit
-        // 0af2f9d838 ran into [1], by having `user` be missing on reactions
-        // on a message:
-        //   https://github.com/zulip/zulip-mobile/issues/4156#issuecomment-655905093
        const store = mockStore<GlobalState, Action>(baseState);

-        // Missing `user` (and `user_id`, for good measure), to
-        // simulate #4156.
-        const faultyReaction = {


I think the comment "Simulate #…, a real-life problem that a user ran into [link]" is helpful — this is otherwise a pretty weird-looking test.

Given that context, and given this latter comment too — which says user_id is missing for good measure — what happens if we leave the test unchanged, with user_id still missing? … Ah, I guess the point is that this is testing what happens if transformFetchedMessage (or anything else within the migrateResponse helper of api.getMessages) throws an exception. Which doesn't happen for just any malformed data, only for malformations in the data it actually has any transformations to perform on.

gnprice · 2023-10-02T21:30:22Z

src/message/__tests__/fetchActions-test.js

-          // day comes. It's badly shaped on purpose.
-          messages: [{ ...fetchedMessage1, reactions: [faultyReaction] }],
+          // day comes. It's badly typed on purpose.
+          messages: [{ ...fetchedMessage1, avatar_url: 123 }],


From reading the avatars code (since that's now the only part of transformFetchedMessages that seems like it could throw an error on bad server data), here's a different fun way to cause an exception:

messages: [{ ...fetchedMessage1, avatar_url: null, sender_email: undefined }],

That causes the email.toLowerCase() here to throw:

`${GravatarURL.ORIGIN}/avatar/${md5(email.toLowerCase())}?d=identicon`,

and doesn't provoke any logging calls.

(The sender_email: undefined is a crude way to get the same effect in the consuming code as if sender_email were omitted altogether. Could also use a line or two more code here and do that properly.)

chrisbobbe · 2023-10-02T21:51:23Z

Thanks for the review! Revision pushed.

Do you still see something to do for #5773 (comment), to make it a less weird-looking test?

gnprice · 2023-10-03T18:51:46Z

Thanks!

Do you still see something to do for #5773 (comment), to make it a less weird-looking test?

I think the comment was helpful for that. Here's an updated comment:

+        // Regression test for #4156.  There was a server bug that caused
+        // message data to be malformed, and though it was only briefly in
+        // main, a user did run into it in real life:
+        //   https://github.com/zulip/zulip-mobile/issues/4156#issuecomment-655905093

Otherwise all looks good — I'll add that comment and merge.

… field Our code at the edge for transforming the old format of reactions, with `user`, into the new format, with `user_id`, was depending on `user` always being present. Since `user_id` is documented as being added in Zulip 3.0: https://zulip.com/api/get-messages#response , and since zulip#5747 we've been refusing to connect to servers pre-3.0, we can assume `user_id` is always present and stop reading `user`. Fixes: zulip#4072

chrisbobbe · 2023-10-04T22:44:14Z

Here's an updated comment:

But this isn't a regression test for #4156, is it? That bug was that we expected a user object field and behaved badly when servers accidentally omitted it: we showed a dreaded infinite loading spinner. The fix was to show an error message instead: #4205.

With this PR, we stop expecting user at all. So the circumstance where we handle its accidental omission badly—in short, #4156—isn't really possible or the kind of thing I'd expect a regression test for.

The test itself doesn't do anything interesting with user/user_id: it simulates a problem we've never actually encountered in the wild, where a different field (sender_email) is missing, tripping up the avatar-URL logic invoked in transformFetchedMessages.

When we added this test in #4205, I wrote in a comment:

        // […] We don't want to care specifically
        // how the data is malformed.

I guess we removed that bit, in 505f78f. I don't remember why, and I think it's not clear from the PR.

chrisbobbe · 2023-10-04T22:47:42Z

On the other hand, I guess it makes sense as a regression test for the kind of issue #4156 was: bad behavior that could be prevented by catching and handling malformed server data in a certain way. If I squint at #4156, blurring over the specifics of how it was about user, then I guess this is in fact a regression test for it.

gnprice · 2023-10-05T20:58:23Z

On the other hand, I guess it makes sense as a regression test for the kind of issue #4156 was: bad behavior that could be prevented by catching and handling malformed server data in a certain way. If I squint at #4156, blurring over the specifics of how it was about user, then I guess this is in fact a regression test for it.

Yeah, exactly. It's a regression test for a somewhat generalized version of #4156, though not for the exact circumstances that bit us then (because with this PR's changes, we happen to no longer be vulnerable to those exact circumstances in the first place).

gnprice · 2023-10-05T21:01:34Z

I guess we removed that bit, in 505f78f. I don't remember why, and I think it's not clear from the PR.

From looking just at that commit, my guess is that the reason we deleted it was mostly about the means the comment was proposing ("See if we can mock migrateMessages"), and not about the end ("We don't want to care specifically …").

chrisbobbe added the api migrations label Oct 2, 2023

chrisbobbe requested a review from gnprice October 2, 2023 20:53

gnprice reviewed Oct 2, 2023

View reviewed changes

gnprice added the P1 high-priority label Oct 2, 2023

chrisbobbe force-pushed the pr-reaction-user-id-migration branch from c63396b to bacded8 Compare October 2, 2023 21:50

gnprice force-pushed the pr-reaction-user-id-migration branch from bacded8 to df1dd79 Compare October 3, 2023 18:52

gnprice merged commit df1dd79 into zulip:main Oct 3, 2023
1 check passed

chrisbobbe deleted the pr-reaction-user-id-migration branch October 4, 2023 22:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

api: Stop assuming fetched messages' reactions have deprecated `user` field #5773

api: Stop assuming fetched messages' reactions have deprecated `user` field #5773

chrisbobbe commented Oct 2, 2023

gnprice left a comment

gnprice Oct 2, 2023

chrisbobbe Oct 2, 2023

gnprice Oct 2, 2023

gnprice Oct 2, 2023

gnprice Oct 2, 2023

gnprice Oct 2, 2023

chrisbobbe commented Oct 2, 2023

gnprice commented Oct 3, 2023

chrisbobbe commented Oct 4, 2023

chrisbobbe commented Oct 4, 2023

gnprice commented Oct 5, 2023

gnprice commented Oct 5, 2023

api: Stop assuming fetched messages' reactions have deprecated user field #5773

api: Stop assuming fetched messages' reactions have deprecated user field #5773

Conversation

chrisbobbe commented Oct 2, 2023

gnprice left a comment

Choose a reason for hiding this comment

gnprice Oct 2, 2023

Choose a reason for hiding this comment

chrisbobbe Oct 2, 2023

Choose a reason for hiding this comment

gnprice Oct 2, 2023

Choose a reason for hiding this comment

gnprice Oct 2, 2023

Choose a reason for hiding this comment

gnprice Oct 2, 2023

Choose a reason for hiding this comment

gnprice Oct 2, 2023

Choose a reason for hiding this comment

chrisbobbe commented Oct 2, 2023

gnprice commented Oct 3, 2023

chrisbobbe commented Oct 4, 2023

chrisbobbe commented Oct 4, 2023

gnprice commented Oct 5, 2023

gnprice commented Oct 5, 2023

api: Stop assuming fetched messages' reactions have deprecated `user` field #5773

api: Stop assuming fetched messages' reactions have deprecated `user` field #5773