Deleted hostpath PVs stuck in released state after minikube restart

[debugmiller]

Dynamically provisioned PVs are stuck in the released state if minikube is restarted before deleting the pvc. It seems that the Provisioner Identity changes on reboot which causes the hostpath provisioner to ignore the PV.

The exact command to reproduce the issue:

cat << EOF | kubectl apply -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: mypvc
spec:
  resources:
    requests:
      storage: 10Mi
  accessModes:
    - ReadWriteOnce
EOF

minikube stop
minikube start
kubectl delete pvc mypvc
kubectl get pv

The full output of the command that failed:

$ minikube ssh tail /tmp/storage-provisioner.INFO
...
I0620 17:17:11.727823       1 controller.go:1040] deletion of volume "pvc-8674aa0d-937a-11e9-b2aa-000c29bcb67d" ignored: ignored because identity annotation on PV does not match ours

The output of the minikube logs command:
N/A

The operating system version:
MacOS 10.14.5
minikube v1.1.1
kubernetes 1.14.3

[fenglixa]

Reason of this issue: After minikube restart, the hostPathProvisionerIdentity was changed, that's why report deletion failed.

[fenglixa]

This issue could be reproduced easily with the steps as logged. But I am afraid this can't be fixed by minikube code. After VM restart, the sandbox ID was changed also.

[tstromberg]

AFAIK, this is still an issue with minikube v1.4 as we have not yet swapped out the storage provisioner addon used.

[medyagh]

I believe this is Dupe of #4546

if you don't mind I will close this issue in favor ohttps://github.com//issues/4546 to cenetralize the work on the issue.

[joma74]

Still an issue under 1.6.1. Note that @medyagh dupe reference is a self reference to this very issue.
Any advice/workaround how to kick PV out appreciated.

[fredgate]

Yes still have this probem.
There is no duplicated issue : #4546 is this issue itself.
@medyagh I think that you can reopen this issue

[fredgate]

ping @tstromberg

[medyagh]

Anyone likes to contribute logs to this issue?

minikube logs

[oraclesamdrake]

Still an issue on OSX in Minikube 1.26.0 with the virtualbox driver.

[oraclesamdrake]

minikube.logs.txt

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[maxb]

/reopen

Just because no one has worked on it doesn't mean this confirmed bug has stopped being a problem.

[k8s-ci-robot]

@maxb: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Just because no one has worked on it doesn't mean this confirmed bug has stopped being a problem.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.