-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pod creation issue on Task retry with workspace.<name>.volume
settings
#7886
Comments
/assign |
…e>.volume` fix tektoncd#7886 Change the naming method of the workspace volume from completely random to hashed, ensuring that the name generated during a single taskRun lifecycle is consistent each time, and is unique within all current workspaces. This way, we can reuse the logic of retrieving the taskSpec from the status, and also store the content after variable expansion in the taskSpec of the status for easy debugging; it will also not affect the reconstruction of the pod when retrying.
…e>.volume` fix tektoncd#7886 Change the naming method of the workspace volume from completely random to hashed, ensuring that the name generated during a single taskRun lifecycle is consistent each time, and is unique within all current workspaces. This way, we can reuse the logic of retrieving the taskSpec from the status, and also store the content after variable expansion in the taskSpec of the status for easy debugging; it will also not affect the reconstruction of the pod when retrying.
…e>.volume` fix tektoncd#7886 Change the naming method of the workspace volume from completely random to hashed, ensuring that the name generated during a single taskRun lifecycle is consistent each time, and is unique within all current workspaces. This way, we can reuse the logic of retrieving the taskSpec from the status, and also store the content after variable expansion in the taskSpec of the status for easy debugging; it will also not affect the reconstruction of the pod when retrying.
…e>.volume` fix tektoncd#7886 Change the naming method of the workspace volume from completely random to hashed, ensuring that the name generated during a single taskRun lifecycle is consistent each time, and is unique within all current workspaces. This way, we can reuse the logic of retrieving the taskSpec from the status, and also store the content after variable expansion in the taskSpec of the status for easy debugging; it will also not affect the reconstruction of the pod when retrying.
…e>.volume` fix tektoncd#7886 Change the naming method of the workspace volume from completely random to hashed, ensuring that the name generated during a single taskRun lifecycle is consistent each time, and is unique within all current workspaces. This way, we can reuse the logic of retrieving the taskSpec from the status, and also store the content after variable expansion in the taskSpec of the status for easy debugging; it will also not affect the reconstruction of the pod when retrying.
…e>.volume` fix tektoncd#7886 Change the naming method of the workspace volume from completely random to hashed, ensuring that the name generated during a single taskRun lifecycle is consistent each time, and is unique within all current workspaces. This way, we can reuse the logic of retrieving the taskSpec from the status, and also store the content after variable expansion in the taskSpec of the status for easy debugging; it will also not affect the reconstruction of the pod when retrying.
…e>.volume` fix tektoncd#7886 Change the naming method of the workspace volume from completely random to hashed, ensuring that the name generated during a single taskRun lifecycle is consistent each time, and is unique within all current workspaces. This way, we can reuse the logic of retrieving the taskSpec from the status, and also store the content after variable expansion in the taskSpec of the status for easy debugging; it will also not affect the reconstruction of the pod when retrying.
@l-qing I am facing this issue as well: Is there any manual workaround? Will this be backported to older stable versions? |
I didn't find a manual solution. |
Expected Behavior
The pipeline should be able to retry normally.
Actual Behavior
When retrying, the pipeline is unable to create the Pod for the retry.
Steps to Reproduce the Problem
Additional Info
Kubernetes version:
Output of
kubectl version
:Tekton Pipeline version:
Output of
tkn version
orkubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'
Analysis
0. Summary
workspace.<name>.volume
in the taskSpec was replaced with a random value.taskRun.status.taskSpec
.CreateVolumes
, the workspace’s name was generated again.1. reconcile -> CreateVolumes && applyParamsContextsResultsAndWorkspaces && createPod
pipeline/pkg/reconciler/taskrun/taskrun.go
Lines 617 to 635 in f0a1d64
2.1 CreateVolumes -> RestrictLengthWithRandomSuffix
pipeline/pkg/workspace/apply.go
Lines 45 to 82 in f0a1d64
2.2 RestrictLengthWithRandomSuffix -> random string
pipeline/pkg/names/generate.go
Lines 54 to 61 in f0a1d64
3.1 applyParamsContextsResultsAndWorkspaces -> ApplyWorkspaces
pipeline/pkg/reconciler/taskrun/taskrun.go
Lines 905 to 946 in f0a1d64
3.2 ApplyWorkspaces -> ApplyReplacements
pipeline/pkg/reconciler/taskrun/resources/apply.go
Lines 277 to 310 in f0a1d64
3.3
workspaces.%s.volume
pipeline/pkg/reconciler/taskrun/resources/apply.go
Lines 299 to 301 in f0a1d64
4. Set replaced taskSpec to TaskRun Status
pipeline/pkg/reconciler/taskrun/taskrun.go
Line 625 in f0a1d64
5. reconcile -> prepare -> GetTaskFuncFromTaskRun
pipeline/pkg/reconciler/taskrun/resources/taskref.go
Lines 55 to 79 in f0a1d64
if the spec is already in the status, do not try to fetch it again, just use it as source of truth.
Solution
1. The content stored in taskRun.status.taskSpec is the original content of taskSpec, rather than the values after variable substitution.
Advantages: It's very versatile and there is no need to worry about retries being affected even if similar random variables are introduced in the future.
Disadvantages: The information in
taskus.taskSpec
is no longer intuitive. It's impossible to directly see the actual variable values during execution.2. Each time the taskSpec definition is acquired, it is fetched from the remote end, rather than reusing the one in the status.
Disadvantages: It needs to be fetched every time reconciliation occurs, which is not very practical.
3. Change the method of generating volume names
No longer be completely random, but rather to hash the original name. Make sure that the name generated each time is the same and is not duplicated within the current workspaces.
In the current taskSpec, only the value of
workspace.<name>.volume
may be different each time; the values of other variables are fixed and will not change anymore.I personally prefer to fix it using this solution.
4. Remove support for
workspace.<name>.volume
The text was updated successfully, but these errors were encountered: