Deleting Push with Last View yields Error 500

[Viajaz]

🐛 Bug Report

Deleting a password push that is already on it's last view will yield HTTP error 500 on pglombardo/pwpush-public-gateway:stable

🔬 How To Reproduce

Steps to reproduce the behavior:

1. Create Password Push with 5 days and 1 view
2. View Password Push with pglombardo/pwpush-public-gateway:stable
3. Delete the Push

Environment

Where are you running/using Password Pusher?

• pwpush.com
• Docker Image
  • pwpush
  • Custom image
• Heroku
• Digital Ocean
  • App Platform
  • Kubernetes Service
• Microsoft Azure
  • App Service
  • Container Instances (ACI)
  • Kubernetes Service (AKS)
• Google Cloud
  • App Engine
  • Cloud Run
  • Kubernetes Engine
• AWS
  • Elastic Container Service (ECS)
  • Kubernetes Service (AKS)
• Source Code
• Other (please specify)

If applicable, what version of Password Pusher?: 1.45.4

📈 Expected behavior

Probably "We apologize but this secret link has expired."

[pglombardo]

Hey @Viajaz - I've been a bit underwater. I'll take a look at this soon.

[Viajaz]

Let me know if I can provide more information. Sorry it's a bit light on details but, as far as I can tell, it seems to be just that simple to reproduce.

[pglombardo]

Just reproduced the error now:

# Logfile created on 2024-09-18 12:33:43 +0000 by logger.rb/v1.6.1
E, [2024-09-23T12:54:00.273596 #39] ERROR -- : [2b11ce46-9262-4883-927f-e1a7ae248ebe]
[2b11ce46-9262-4883-927f-e1a7ae248ebe] NoMethodError (undefined method `root_url' for an instance of PasswordsController):
[2b11ce46-9262-4883-927f-e1a7ae248ebe]
[2b11ce46-9262-4883-927f-e1a7ae248ebe] app/controllers/passwords_controller.rb:293:in `block (2 levels) in destroy'
[2b11ce46-9262-4883-927f-e1a7ae248ebe] app/controllers/passwords_controller.rb:292:in `destroy'
[2b11ce46-9262-4883-927f-e1a7ae248ebe] app/controllers/concerns/set_locale.rb:9:in `set_locale'

[pglombardo]

Fix released in v1.45.9 which is building now. Thanks @Viajaz!

[Viajaz]

I switched to :latest for both containers and attempted to reproduce, unfortunately, still error 500 for me.

Confirmed in logs that pwpush-public-gateway is running 1.45.9:

Password Pusher: migrating database to latest...
Password Pusher Version: 1.45.9
Password Pusher: precompiling assets for customisations...
Password Pusher Version: 1.45.9

@pglombardo

[pglombardo]

How are you deleting the push? Via API or UI?

[Viajaz]

It was the UI, however, when I went to perform another reproduction, and collect some more HTTP request details for you, the error is now gone. I don't have an explanation, it was the same test instance that I had used 8 hours before this post and I checked to ensure the version was the latest when posting last.

Regardless, I guess problem is fixed?
@pglombardo

[pglombardo]

No I think you are right. There is another code path that can give this error. I need a better fix.

[magaiverpr]

Hello,

Testing here, If I delete a push sended to a Public gateway, the page dont redirect to the page "you link expired", its runs eternal. BUT the link is deleted, if I refresh manually the page (press F5), the link was gone.

[pglombardo]

When a push is deleted/expired, if anonymous, you get redirected back to the "expired" page. If you are logged in and the push owner/creator, you get sent to the audit log.

Audit log doesn't exist on the public gateway. My theory is that the problem is somewhere in this redirect logic.

If you both test in an incognito window, does the problem still occur?

I'll get back on this soon.

[magaiverpr]

When a push is deleted/expired, if anonymous, you get redirected back to the "expired" page. If you are logged in and the push owner/creator, you get sent to the audit log.

Audit log doesn't exist on the public gateway. My theory is that the problem is somewhere in this redirect logic.

If you both test in an incognito window, does the problem still occur?

I'll get back on this soon.

I have 2 enviroments here: DEV and PROD.

Only PROD is using the gateway.

Tested in DEV enviroment without public gateway and incognito mode in another browser:

Testing in PROD mode with public gateway and incognito mode in another browser:

Forget to say, I am not using audit mode nor login

[Viajaz]

There's an argument to be made about the need to even show a Delete button if this was the last view.

[wedowhateverwewant]

I can confirm this issue is still active in version 1.45.10 also using public gateway with pwpusher setup and after the last view , you refresh and the blue error 500 comes up but refresh again and you get the regular this link expired page. in my testing it happens with file links.

[pglombardo]

This should be fixed now in v1.45.11.

1. The "Delete Immediately" button no longer shows if there are no views left.
2. Greatly simplified the push deletion checks.
3. Fixed an issue where root path wasn't defined in the gateway image.
4. On deletion operation, the redirect is now always back to the expired push (simpler)

If some of you could confirm, it would be very much appreciated.

[wedowhateverwewant]

Hi,

the issue is still very much active, after pushing files and accessing the link ( set for 1 view) . restarting the page will still show the blue screen error 500, but here is the thing I found out. If I upload 3 files ( set with 1 view) go to the push link. and after refreshing the page I get the blue screen error, but now I have to refresh 3 times to get to the expired page. and if you upload 5 files you got to refresh the page 5 times to get to the expired page.( tested with chrome, and iphone safari)

on the side note, im not sure if your able to verify this , but with URLs ( set for 1 view) the link is expired before I even access it. tried on multiple devices and browser same issue.

[pglombardo]

That sounds like an issue with the file storage. Could you get me the stack trace of the error from inside the public gateway container? The logs are in /opt/PasswordPusher/log

[wedowhateverwewant]

here are the logs from /opt/PasswordPusher/log/production.log

1343322be5ec:/opt/PasswordPusher/log$ tail -f production.log
[b284718d-9fde-419b-b9a6-f911cf64d048] app/models/file_push.rb:108:in validate!' [b284718d-9fde-419b-b9a6-f911cf64d048] app/controllers/file_pushes_controller.rb:27:in show'
[b284718d-9fde-419b-b9a6-f911cf64d048] app/controllers/concerns/set_locale.rb:9:in set_locale' E, [2024-10-07T18:03:43.771633 #23] ERROR -- : [169f66b8-e387-4a15-9f9c-a0078059eb5f] [169f66b8-e387-4a15-9f9c-a0078059eb5f] Errno::EACCES (Permission denied @ apply2files - /opt/PasswordPusher/storage/cw/03/cw03zcbetrjoim63yk4npomvimxn): [169f66b8-e387-4a15-9f9c-a0078059eb5f] [169f66b8-e387-4a15-9f9c-a0078059eb5f] app/models/file_push.rb:45:in expire'
[169f66b8-e387-4a15-9f9c-a0078059eb5f] app/models/file_push.rb:108:in validate!' [169f66b8-e387-4a15-9f9c-a0078059eb5f] app/controllers/file_pushes_controller.rb:27:in show'
[169f66b8-e387-4a15-9f9c-a0078059eb5f] app/controllers/concerns/set_locale.rb:9:in set_locale' E, [2024-10-07T21:27:27.988338 #23] ERROR -- : [4390e648-ddaf-4941-b822-8cb5af2d9248] [4390e648-ddaf-4941-b822-8cb5af2d9248] Errno::EACCES (Permission denied @ apply2files - /opt/PasswordPusher/storage/mx/l8/mxl8hu4f9ww4lreucncswutydgu9): [4390e648-ddaf-4941-b822-8cb5af2d9248] [4390e648-ddaf-4941-b822-8cb5af2d9248] app/models/file_push.rb:45:in expire'
[4390e648-ddaf-4941-b822-8cb5af2d9248] app/models/file_push.rb:108:in validate!' [4390e648-ddaf-4941-b822-8cb5af2d9248] app/controllers/file_pushes_controller.rb:27:in show'
[4390e648-ddaf-4941-b822-8cb5af2d9248] app/controllers/concerns/set_locale.rb:9:in set_locale' E, [2024-10-07T21:27:33.686777 #23] ERROR -- : [9ef777cd-d675-4fc5-93b3-1d977049bdae] [9ef777cd-d675-4fc5-93b3-1d977049bdae] Errno::EACCES (Permission denied @ apply2files - /opt/PasswordPusher/storage/y7/y1/y7y1g0x5vonc4dxzj3jqx95yb7mz): [9ef777cd-d675-4fc5-93b3-1d977049bdae] [9ef777cd-d675-4fc5-93b3-1d977049bdae] app/models/file_push.rb:45:in expire'
[9ef777cd-d675-4fc5-93b3-1d977049bdae] app/models/file_push.rb:108:in validate!' [9ef777cd-d675-4fc5-93b3-1d977049bdae] app/controllers/file_pushes_controller.rb:27:in show'
[9ef777cd-d675-4fc5-93b3-1d977049bdae] app/controllers/concerns/set_locale.rb:9:in set_locale' E, [2024-10-07T21:27:35.507514 #23] ERROR -- : [0f5d0215-b652-4bcb-a273-701383ebc162] [0f5d0215-b652-4bcb-a273-701383ebc162] Errno::EACCES (Permission denied @ apply2files - /opt/PasswordPusher/storage/8a/ne/8anelujpap5g3zveu2e7j7akkie0): [0f5d0215-b652-4bcb-a273-701383ebc162] [0f5d0215-b652-4bcb-a273-701383ebc162] app/models/file_push.rb:45:in expire'
[0f5d0215-b652-4bcb-a273-701383ebc162] app/models/file_push.rb:108:in validate!' [0f5d0215-b652-4bcb-a273-701383ebc162] app/controllers/file_pushes_controller.rb:27:in show'
[0f5d0215-b652-4bcb-a273-701383ebc162] app/controllers/concerns/set_locale.rb:9:in set_locale' E, [2024-10-07T21:27:36.567631 #20] ERROR -- : [25c4cd52-79da-4d3c-9637-4b360d66f68a] [25c4cd52-79da-4d3c-9637-4b360d66f68a] Errno::EACCES (Permission denied @ apply2files - /opt/PasswordPusher/storage/bu/pb/bupbjo1oywjpumcju1f7knxx700f): [25c4cd52-79da-4d3c-9637-4b360d66f68a] [25c4cd52-79da-4d3c-9637-4b360d66f68a] app/models/file_push.rb:45:in expire'
[25c4cd52-79da-4d3c-9637-4b360d66f68a] app/models/file_push.rb:108:in validate!' [25c4cd52-79da-4d3c-9637-4b360d66f68a] app/controllers/file_pushes_controller.rb:27:in show'
[25c4cd52-79da-4d3c-9637-4b360d66f68a] app/controllers/concerns/set_locale.rb:9:in `set_locale'

[wedowhateverwewant]

any update on this ? let me know if you need more logs.

[pglombardo]

Apologies all - I am just flooded this week.

Errno::EACCES (Permission denied @ apply2files

This definitely shouldn't be there and would definitely cause an error 500. The last view triggers an immediate expiration of the push - and deletes the payload and attached files.

Can the pwpusher user in the container delete the files it lays down?

[wedowhateverwewant]

if I look inside the directory folder I am using for local storage for the pwpusher container I do see a bunch of directory and files, Im assuming once the link is expired these files should be purged but they are not ?

[pglombardo]

Without the permission denied, the files are deleted. You can see the line of code in that stack trace that calls files.purge:

https://github.com/pglombardo/PasswordPusher/blob/master/app/models/file_push.rb#L45

When you are inside the container as user pwpusher, can you touch and delete a file?

[wedowhateverwewant]

yes inside the pwpush container I was able to create and delete a text file. One thing id like to know, in the previous screen shot I shared , that is the storage for Files. are those directory's ( 0i , 73, b2, g4, t6) are they supposed to be auto removed by the container once the file link expires ?

[pglombardo]

I would think so yes but if they are not, it's no big deal as they are just organizational directories based on blob ID. More important is that the files themselves are deleted.

Occasionally if a file upload gets interrupted or something goes wrong (like the error 500) files can be orphaned as the process didn't complete. In that case you can run bin/pwpush active_storage:cleanup. It will clean up any orphaned files. I'll be adding this to the pwpush-worker container soon.

Between the pwpush and pwpush-public-gateway container, they use a shared filesystem mount?

[pglombardo]

Filed #2646 so that I remember to do this soon.

[wedowhateverwewant]

between the pwpush and the gateway container , they both use the same local storage thats on the host.

When you mentioned permissions in your last comment, I added the (user: "0:0") to the gateway container as well, this actually seems to have resolved my error 500 issue. but those files I mentioned before just dont get cleaned up.

inside pwpush console, how do I exactly run the active_storage:cleanup command ? an example would be great

[pglombardo]

I'm away from the computer so let's see if I can type this. Log into pwpush container and run bin/pwpush active_storage:cleanup

/opt/PasswordPusher directory

Back in a bit

[wedowhateverwewant]

its saying unrecognized command

15e41661190e:/opt/PasswordPusher# bin/pwpush active_storage:cleanup
Password Pusher Version: 1.45.11
Unrecognized command "active_storage:cleanup" (Rails::Command::UnrecognizedCommandError)
15e41661190e:/opt/PasswordPusher#

[pglombardo]

My bad - the command has changed in the beta release. The command for the latest pwpush container is:

bin/pwpush active_storage:purge_unattached

[wedowhateverwewant]

well the command worked, but those directory are still there even tho my file pushes are expiring fine now and no error 500.

[pglombardo]

Ok well that's good news then. Let me know if it happens again.

Otherwise I'll close out this issue soon if nothing else remains.

Have a good weekend!

[wedowhateverwewant]

even tho the command you gave me ran successfully, the directory files still remained, is that standard ? or should it be removing those ? or do I need to manually remove these once in a while

[pglombardo]

I would leave them. The blob IDs are randomly generated and the directories are used to organize files.

This is really low level but if it helps, for every file uploaded a base64-encoded blob id is generated. It includes a signature for integrity and also contains other metadata:

eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBBNzhtIiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--d7af67854df5c83c59a2bce64af8db15b312d52f

Also this way you don't have uploaded files with sensitive filenames such as bank_account_numbers_for_ssn_123456789.txt.

so tldr; leave the directories.

[wedowhateverwewant]

alright sounds good , thanks

Good to close the ticket