Staging site down post-Valkyrie

[KatharineV]

One of our staging sites is giving an error message when we try to view the homepage. It worked before Valkyrie, so I hope it's just related to that.

I can see the dashboard when I enter a direct link, but the homepage is no good. Unfortunately, this is a staging site that I use to train repository managers in our consortium, so I hope it can be brought back online at some point.

https://testing.s2.adventistdigitallibrary.org/?locale=en

[KatharineV]

I checked all our staging sites, and this seems to be the only one affected.

[ShanaLMoore]

related convo

[ShanaLMoore]

@laritakr I wonder if the solution for this issue is captured here. app/views/hyrax/homepage/_featured_fields.html.erb:6 is part of the error's stack trace

[ShanaLMoore]

From live editing the server:

One or multiple of these works will be the cause of this issue:

[ShanaLMoore]

FeaturedWork.all => 
[#<FeaturedWork:0x00007ff6810adbb8
  id: 1,
  order: 0,
  work_id: "dsgucpbyeqg_graduate_baccalaureate_church_service",
  created_at: Mon, 24 Jul 2023 20:42:22.232882000 UTC +00:00,
  updated_at: Mon, 24 Jul 2023 20:43:12.352370000 UTC +00:00>,
 #<FeaturedWork:0x00007ff68b105340
  id: 4,
  order: 1,
  work_id: "019887_an_explanation_of_the_new_folding_prophetic_chart",
  created_at: Mon, 24 Jul 2023 20:42:50.391977000 UTC +00:00,
  updated_at: Mon, 24 Jul 2023 20:43:12.343324000 UTC +00:00>,
 #<FeaturedWork:0x00007ff68b1052a0
  id: 2,
  order: 2,
  work_id: "nes_b1900_1910_p001_south_lancaster_workers_1901",
  created_at: Mon, 24 Jul 2023 20:42:29.735130000 UTC +00:00,
  updated_at: Mon, 24 Jul 2023 20:43:12.349648000 UTC +00:00>,
 #<FeaturedWork:0x00007ff68b105200
  id: 3,
  order: 3,
  work_id: "10_32597_eve_in_the_image_of_man_feminist_concerns_in_paradise_lost",
  created_at: Mon, 24 Jul 2023 20:42:42.861959000 UTC +00:00,
  updated_at: Mon, 24 Jul 2023 20:43:12.346795000 UTC +00:00>,
 #<FeaturedWork:0x00007ff68b105160
  id: 6,
  order: 6,
  work_id: "auss_32_1_1994_pp107_110_the_adventist_heritage_center_and_a_tribute_to_it

It's concerning that when I go to the works index page it says there are none. If this is a demo-able tenant, where's all the data?

[ShanaLMoore]

I turned off featured works for now and am able to reach the tenant's homepage now.

[ShanaLMoore]

LaRita cleared the FeaturedWorks table. It seems like the works in that table were deleted, and thus caused this error because the table was still referencing them. The tenant home page loads now and the Featured Work feature flipper is turned back on.

The default theme looks kind of strange but that is likely a separate Hyku concern. Switched to the Adventist theme to match the ADL and SDAPI s2 setting's.

ref: https://testing.s2.adventistdigitallibrary.org/?locale=en

[KatharineV]

Folks, there were a lot of works in this tenant. The fact that they were all deleted in the move to Valkyrie and Version 6 gives me some anxiety, since we have multiple tenants in production as well. Can the team speak to why this might have happened to the testing.s2 tenant on staging? Is it reasonable for me to be concerned that it may happen again when we push to prod?

[ShanaLMoore]

@KatharineV I noticed this yesterday as well and found it odd too. The lack of data ultimately was the cause of this bug ticket. I'll need time to meet with Rob and the team to find out what happened to this specific tenant and whether or not it can be restored.

We haven't converted active fedora works into valkryie on a mass scale so I'm at a loss for what might have happened here. I'm also hesitant to blame it on Valkyrie because of that.

I also found it strange that there are collections and apparently some have an item count (170), but when I clicked into them I wasn't able to see anything. I thought maybe the works were still there but that there's an issue with permissions.

In Rancher...

ActiveFedora::Base.count
=> 2732

[KatharineV]

Since the site is back up, and this ticket wasn't about the visibility or invisibility of existing works, I'm moving the ticket to the next column and marking it as done with Client QA.