Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs for daemons #346

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Docs for daemons #346

wants to merge 1 commit into from

Conversation

labkode
Copy link

@labkode labkode commented Jul 15, 2024

Fixes #296

@labkode labkode changed the title draft docs for daemons Docs for daemons Jul 18, 2024
@labkode labkode marked this pull request as ready for review July 18, 2024 09:16
@labkode labkode force-pushed the daemons branch 3 times, most recently from 300d24b to 910beb0 Compare July 18, 2024 09:27
@labkode labkode requested a review from bari12 July 18, 2024 09:28
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
@labkode labkode force-pushed the daemons branch 2 times, most recently from 827df06 to 2810abd Compare July 19, 2024 08:34
Copy link
Author

@labkode labkode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes applied

@labkode labkode requested a review from rdimaio July 19, 2024 08:36
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved

## How is a replica declared bad?
- An operator can declare a replica as bad issuing rucio cli commands. These bad replicas are taken by the necromancer daemon and then deleted if they cannot be recovered from other RSEs.
- the suspicios-replica-recoverer is a daemon that will analyze different counters (transfer errors, download errors, etc ...) to mark replicas as being suspicious to have an issue. After hitting
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- the suspicios-replica-recoverer is a daemon that will analyze different counters (transfer errors, download errors, etc ...) to mark replicas as being suspicious to have an issue. After hitting
- The suspicious-replica-recoverer is a daemon that will analyze different counters (transfer errors, download errors, etc ...) to mark replicas as being suspicious to have an issue. After hitting

Copy link
Contributor

@voetberg voetberg Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still misspelled

docs/started/daemons.md Outdated Show resolved Hide resolved
docs/started/daemons.md Outdated Show resolved Hide resolved
@labkode
Copy link
Author

labkode commented Nov 12, 2024

@rdimaio @voetberg changes applied, thanks for your suggestions.


## Daemons

Name | Domain| Purpose | Details
Copy link
Contributor

@voetberg voetberg Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will be the last nit-pick. Is it is possible to set the links in Details up with the a relative path? Using this method will redirect the user to a different site to look at the executable page if they are looking at the gh docs mirror.

a certain limit (configurable), the replica is marked as bad and eventually consumed by the necromancer.

## What is the purpose of undertaker?
A dataset is "never" deleted, however, when the dataset is known to be bad, there is no point having it in the catalog. The undertaker daemon takes care to remove these datasets.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A dataset is "never" deleted, however, when the dataset is known to be bad, there is no point having it in the catalog. The undertaker daemon takes care to remove these datasets.
A dataset is never "deleted", however, when the dataset is known to be bad, there is no point having it in the catalog. The undertaker daemon takes care to remove these datasets.

"deleted" is the specific term that isn't actually done

Sometimes, replicas can become unhealthy. A dump is created by the dumper daemon. The auditor checks these dumps and declares replicas.

## How is a replica declared bad?
- An operator can declare a replica as bad issuing rucio cli commands. These bad replicas are taken by the necromancer daemon and then deleted if they cannot be recovered from other RSEs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add an example code block? It's useful for operators so they don't have to go diving into document.


### How is data deleted?
When replicas are healthy, the judge-cleaner will set a tombstone on replicas where the lifetime has expired. These replicas are taken by the reaper and they are deleted.
Sometimes, replicas can become unhealthy. A dump is created by the dumper daemon. The auditor checks these dumps and declares replicas.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you define "unhealthy"? It's a vague term here. I took a stab at it

Suggested change
Sometimes, replicas can become unhealthy. A dump is created by the dumper daemon. The auditor checks these dumps and declares replicas.
Sometimes, replicas can become unhealthy (either through corrupted data or incomplete transfers). A dump is created by the dumper daemon. The auditor checks these dumps and declares replicas.

@@ -27,6 +27,7 @@ and more can be found by browsing through the sub-sections of this topic.
- [What is Rucio](started/what_is_rucio.md)
- [Main Components](started/main_components.md)
- [Additional Layers and Resources](started/additional_layers_and_resources.md)
- [Rucio daemons/processes](started/daemons.md)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- [Rucio daemons/processes](started/daemons.md)
- [About Rucio Daemons](started/daemons.md)

@voetberg
Copy link
Contributor

voetberg commented Nov 12, 2024

changes applied, thanks for your suggestions.

Thank you so much for dealing with this long review and making so many changes 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make a table for all daemon and its functionality
4 participants