[issue] When I reload “VERIFIED” tab from Capture Cam, it keeps loading many many very old Captures (seems like it tried to reload everything instead of only the latest ones)

[sync-by-unito]

User story

As a user of the Capture Cam feature,

I want the "VERIFIED" tab to reload and display only the latest verified captures upon refresh,

So that I can easily access the most recent content without waiting long time for refresh.

Reproduce step

1. Login Capture Cam & recovery the asset
2. Create asset from another side
3. pull down to upload the latest asset instead of reload every asset in the account

Expectation

When I reload “VERIFIED” tab from Capture Cam, it should only reload the latest ones.

Additional information:

https://dt42-numbers.slack.com/archives/C0323488MEJ/p1695983857350369

https://docs.google.com/presentation/d/1jdmYyUynCzaMsfHMdAdzJ23KnF5XhgsJt27FgPjNQiY/edit#slide=id.p

┆Issue is synchronized with this Asana task by Unito
┆Created By: Kenny Hung

[sync-by-unito]

➤ Kenny Hung commented:

Scott Yan (cc Tammy YangBofu Chen)

This task I'll suggest arranging into 1023 sprint.

[sync-by-unito]

➤ Sam commented:

Kenny Hung yes that’s a current implementation when you pull it will refetch all. And of course it make sense to reload only first 10 latest items.

One more thing to mention why old implementation is pulling all the assets might be that

Before registering it check if the capture was uploaded before if yes then it stop uploading to backend. So currently it check for duplicates in ionic side.

[sync-by-unito]

➤ Kenny Hung commented:

Added https://docs.google.com/presentation/d/1jdmYyUynCzaMsfHMdAdzJ23KnF5XhgsJt27FgPjNQiY/edit#slide=id.p ( https://docs.google.com/presentation/d/1jdmYyUynCzaMsfHMdAdzJ23KnF5XhgsJt27FgPjNQiY/edit#slide=id.p ) in the description.

[sync-by-unito]

➤ Kenny Hung commented:

https://dt42-numbers.slack.com/archives/C0323488MEJ/p1697623438724939 ( https://dt42-numbers.slack.com/archives/C0323488MEJ/p1697623438724939 )

[sync-by-unito]

➤ Kenny Hung commented:

According to https://dt42-numbers.slack.com/archives/C0323488MEJ/p1698742854255449 ( https://dt42-numbers.slack.com/archives/C0323488MEJ/p1698742854255449 ) push to 1106 sprint.

[sync-by-unito]

➤ Sam commented:

Kenny Hung, I'm still working on this task. I also found that when I reload it do too many request then it should do which also contributes to slow reloads.

I need more time to work on this task. Today I will focus on this task and later will notify with updates if I have any.

[sync-by-unito]

➤ Kenny Hung commented:

Sam (cc Bofu ChenJames ChienScott YanTammy Yang)

Just confirm, since the goal of this task is to enable the reload function to reload the current latest captures, it's necessary first to address the issue of excessive API calls during assets fetch to complete this task, right?

[sync-by-unito]

➤ Sam commented:

Kenny Hung, I found the root cause (explained in video).

Solution for excessive API calls is to

1. add new field "caption" to Proof object (in capture cam).
2. every time caption is changed => update Proof.caption
3. then in verified tabs check if asset has caption using proof.caption instead of doing API call.
4. also when reload/refresh is triggered we can use prefetch (

   capture-lite/src/app/shared/dia-backend/asset/prefetching/dia-backend-asset-prefetching.service.ts

   Line 34 in 6fb9378

   await this.downloadingService.storeRemoteCapture(

   )method to update local captions (aka Proof.caption)

[sync-by-unito]

➤ Sam commented:

Kenny Hung, regarding your question ( https://app.asana.com/0/0/1205627319931691/1206137299959857/f ).

it's necessary first to address the issue of excessive API calls during assets fetch to complete this task, right?

reload function to reload the current latest capturesis not blocked or depends on issue of excessive API calls during assets fetch.

I will create separate asana card with high priority and make sure we arrange as soon as possible.

[sync-by-unito]

➤ Sam commented:

Kenny Hung, asana card is created [Issue] Excessive API calls during reload/restore captures ( https://app.asana.com/0/1201016280880500/1206137436792270/f )

[sync-by-unito]

➤ Kenny Hung commented:

Sam (cc Bofu ChenJames ChienScott YanTammy Yang)

Thanks, please focus on this task unless other tasks could help to improve or block this task.

[sync-by-unito]

➤ Sam commented:

Kenny Hung, just status update I finished lazy load for VERIFIED tab only here is the demo.

Next I will work on lazy load COLLECTED tab.

After that I will test few more time and prepare firebase release. I plan to submit release reminder for Tue (Dec 12) morning.

[sync-by-unito]

➤ Kenny Hung commented:

It seems that every time user scroll down, the latest 10 assets are added, am I right?

[sync-by-unito]

➤ Sam commented:

Almost.

PUBLIC CAPTURE CAM

when user does pull to refresh it does the following

1. Fetch all assets (only json data)
2. For each asset download thumbnail (this part make user wait too long and make app slow)

When use scroll down

1. Just show 10 more assets (already downloaded)

VERSION IN DEMO (new implementation)

when user does pull to refresh 25 things happen

1. Fetch all assets (only json data)
2. Download first 10 thumbnails

When user scroll down

1. Download 10 more thumbnails (if not downloaded yet)

Of course new implementation can be improved further and further but I think this is good enough improvement for now as you can see from demo video.

[sync-by-unito]

➤ Kenny Hung commented:

Sam (cc Scott YanSherry ChungTammy YangBofu ChenJames Chien)

QA found when user pull down to refresh captures, that only 10 images will be loaded, and Capture Cam don’t seem to be saved. After closing and reopening the app, there are still only 10 images.

The expectation should be

1. When user re-login then press restore captures (restore process finished), the verified items should be display even user is offline. (This part should be the same as previous version.)
2. When user pull down to refresh captures, QA suppose the new lazy loading design is okay, but the items which has loaded, even user is offline, it should display instead of still only 10 items.

[sync-by-unito]

➤ Kenny Hung commented:

Sam Will you re-release tomorrow?

If you have certain idea to fix, maybe we could discuss with Tammy Yang.

I summary the v231207 & public version on miro ( https://miro.com/app/board/o9J_knEiIYo=/?moveToWidget=3458764572931523236&cot=14 ).

My suggestions are: (Needs Tammy Yang's confirmation.)

1. Fix issue Bump protractor from 5.4.4 to 7.0.0 #2
   1. When user pull down to refresh captures, the items which has loaded, even user is offline, it should display instead of still only 10 items.

2. Download asset thumbnail from 10 to 50 when use scroll down. (Precise numbers require Sam’s assistance for experimentation to balance UX.)

3. Remove the restore feature.

Pros & cons

1. Pros
   1. The reload performance is better than public version too much unless user just have a few asset
   2. The exist user doesn’t need to wait for restore asset when they re-login

2. Con
   1. If user is offline, they can’t see older assets. (This part IG is the same.)

Sam, if you have any better ideas, we could discuss them together tomorrow.

[sync-by-unito]

➤ Sam commented:

@kenny Hung sure I can release tomorrow. And we can discuss that tomorrow as well. Just make sure we confirm the changes and I can implement them

[sync-by-unito]

➤ Tammy Yang commented:

SamKenny Hung The requirements of the Verified page (same as the Live version)

1. Content should be displayed even when user is offline
2. All content displayed should be the original file instead of thumbnail only
3. No matter user upload from App or dashboard, the content should be synced (current issue)

The reason is, Capture serves as the "asset wallet" to users, so we should let users hold their own content instead of downloading only the thumbnails. Capture is not IG.

This requirement made the syncing of Capture very challenging since day 1 we built it. However, solving the challenge and finding good ways to achieve it shows the value of our developers.

Please propose a good plan to solve the issue. I strongly suggest Sam create a miro to explain how do we plan to solve the issue, and I want to see the flow by the end of tomorrow. Once approved, we can move on

[sync-by-unito]

➤ Sherry Chung commented:

Sam (cc Kenny Hung) If you need my help for further clarification. I’m available tomorrow 2:00-3:00 pm for huddle

[sync-by-unito]

➤ Sam commented:

Miro related to this task can be found here ( https://miro.com/app/board/o9J_knEiIYo=/?moveToWidget=3458764573039264715&cot=14 ) (Work In Progress). I will ask some question to while working on miro.

Just to remind

Currently in public version when user opens asset details page

1. if online we show bubble iframe to display assets
   1. we show all possible information provided by backend (like uploaded_at etc)
   2. use can see original image or play original video

2. if offline we show ionic ui to display asset
   1. we only show fields provided by proof object (limited information)
   2. we only show thumbnail preview (for video and images) meaning user can not see original image or play original video

2b is caused by changes related to bubble iframe. Original asset video/image suppose to be downloaded (if not locally stored) every time when user enters details page but since we use bubble iframe to show information and preview photo/videos we no longer download it locally.

[sync-by-unito]

➤ Sam commented:

Kenny Hung (cc: Tammy Yang, Sherry Chung)

Since implementing these requirements ( https://app.asana.com/0/0/1205627319931691/1206142917957845/f ) might take some time I would recommend first fix QA feedback ( https://app.asana.com/0/0/1205627319931691/1206147430104127/f ) and release to public.

After that we can focus on implementing these requirements ( https://app.asana.com/0/0/1205627319931691/1206142917957845/f )

[sync-by-unito]

➤ Sherry Chung commented:

Sam (cc Kenny Hung) I would suggest we don't do release for release. We can discard current Ionic release if that didn't really solve the problem. This is the only question we need to solve in this sprint.

Please try to find a solution to meet the requirement. Ping me on Slack if you need Huddle

[sync-by-unito]

➤ Kenny Hung commented:

Samhttps://chat.openai.com/share/6c30bc3e-2cee-4c57-a31a-7549891b6f1d ( https://chat.openai.com/share/6c30bc3e-2cee-4c57-a31a-7549891b6f1d )

I'm not sure if this helps, but I just asked ChatGPT about possible solutions, which might give you some inspiration.

[sync-by-unito]

➤ Sam commented:

Tammy Yang (cc: Sherry Chung, Kenny Hung).

Capture cam public version

1. when user initiate restore (related miro ( https://miro.com/app/board/o9J_knEiIYo=/?moveToWidget=3458764573039426239&cot=14 ))
   1. it fetches all remote assets
   2. for each asset

   1. download asset_thumbnail (if not downloaded and stored locally before)

2. when user initiate pull to refresh (related miro ( https://miro.com/app/board/o9J_knEiIYo=/?moveToWidget=3458764573040552407&cot=14 ))
   1. it fetch all remote assets
   2. for each asset

   1. download asset_thumbnail (if not downloaded and stored before)

Capture cam public version

1. when user enters Details Page (related miro ( https://miro.com/app/board/o9J_knEiIYo=/?moveToWidget=3458764573060701482&cot=10 ))
   1. if online
   1. show asset details using bubble iframe (can view photo/video since bubble work in online mode and have access to remote video/photo)

2. if offline
  1. show asset details using ionic components and proof object (can not view video/photo only thumbnail preview)

Capture cap version before adding bubble page to Details page

1. when user enter Details Page (related miro ( https://miro.com/app/board/o9J_knEiIYo=/?moveToWidget=3458764573060701623&cot=10 ))
   1. regardless of online/offline
   2. try to download asset_file (if not downloaded and stored before)
   3. can preview image/video since it downloaded

Kenny Hung, (cc: Tammy Yang, Sherry Chung)

For now I don't have super cool solution to fix [issue] When I reload “VERIFIED” tab from Capture Cam, it keeps loading many many very old Captures (seems like it tried to reload everything instead of only the latest ones) ( https://app.asana.com/0/1201016280880500/1205627319931691/f ) however I have 1 proposal in mind

When user trigger pull to refresh do the following

1. show alert "Refresh captures? And explain user that it might download remote assets to your local device if not downloaded yet and it might take few minutes and explain that since capture cam is content wallet etc"
2. if user say no => do nothing
3. if user say yes => show blocking UI with progress (just like restore captures does, in fact we just reuse restore captures feature)

This proposal inform user that it will take time and make him aware.

[sync-by-unito]

➤ Tammy Yang commented:

I actually think this is smart, good idea! Let's do this:

When users pull down to refresh, ask user

• Subject: Sync and restore all assets?

• Description: This will download all assets again and making sure your local data is up to date.

If user choose "Confirm", go to your proposal step #3 (show blocking UI with progress)

If user choose "Do it later", do nothing.

BTW, I hope my messages are delivered well to you, Kenny Hung and Sherry Chung

When we face a super challenging issue, we need to work together like this to brainstorm and find a solution.

However, how do we know the issue is super challenging? We need to communicate with each other and make sure we understand the problem well. It is important that we care about each others work and help each other.

Let's go with this plan, it's not something like magic, but good enough UX to move on!

[sync-by-unito]

➤ Sam commented:

Kenny Hung, here is the final result of implemented proposal ( https://app.asana.com/0/0/1205627319931691/1206142917957851/f ).

https://app.claap.io/numbers-protocol/feat-capture-tab-show-confirm-dialog-before-refr-c-O35CsUM4Uy-8WYI1bAHJrhA ( https://app.claap.io/numbers-protocol/feat-capture-tab-show-confirm-dialog-before-refr-c-O35CsUM4Uy-8WYI1bAHJrhA )

[sync-by-unito]

➤ Sam commented:

Here is the PR ( #3171 )

[sync-by-unito]

➤ Sam commented:

Here is the new PR ( #3197 ) because previously we reverted this changes ( https://app.asana.com/0/0/1206261707717918/1206483891493527/f ).