-
Notifications
You must be signed in to change notification settings - Fork 196
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move pkg util strings/keymutex from kubernetes kubernetes #55
Merged
thockin
merged 27 commits into
kubernetes:master
from
dims:move-pkg-util-strings-from-kubernetes-kubernetes
Jan 25, 2019
Merged
Changes from all commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
0110028
Fix GCE Cloudprovider waitForOp bug and make GCE attach/detach atomic…
saad-ali d2fe06d
Add strings pkg to hold strings utils
resouer 96c2c2b
fixed spelling typo
markturansky 8620da4
Spelling fixes inspired by github.com/client9/misspell
laushinka 22f7437
Remove "All rights reserved" from all the headers.
david-mcmahon 268d9e1
Increase coverage in pkg/util/strings to 100%
7d203d6
Move linedelimiter to it's own pkg
resouer 0095d8f
Run goimports
luxas 97a3a81
autogenerated
mikedanese 7554c42
Correct the article in generated documents
7e61e68
autoupdate BUILD files
mikedanese 6bb1ef6
autogenerated
mikedanese 034b5d4
snip pkg/util/strings dependency
deads2k 1883524
Enable auto-generating sources rules
ixdy 02cc29c
autogenerated
mikedanese 7557de2
Use buildozer to remove deprecated automanaged tags
ixdy e18510c
Use buildozer to delete licenses() rules except under third_party/
ixdy 2c582b9
Fix godoc comments.
php-coder 280213a
update BUILD files
ixdy 1b4c91e
remove unused function in keymutex_test.go file
WanLinghao 19ea2b7
Autogenerate BUILD files
ixdy 4d41905
Autogenerated: hack/update-bazel.sh
ixdy fc945b6
Using a fixed set of locks, then we don't need to free unused locks
cofyc 58f4cf1
fix golint errors in pkg/util/strings
chrischdi fa5daa7
Move directories to the right spot
dims 41bae72
Drop ForDisk from method names(s), remove IsVowel
dims 2b39f60
Drop (Escape|Unescape)PluginName
dims File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
package(default_visibility = ["//visibility:public"]) | ||
|
||
load( | ||
"@io_bazel_rules_go//go:def.bzl", | ||
"go_library", | ||
"go_test", | ||
) | ||
|
||
go_library( | ||
name = "go_default_library", | ||
srcs = [ | ||
"hashed.go", | ||
"keymutex.go", | ||
], | ||
importpath = "k8s.io/kubernetes/pkg/util/keymutex", | ||
deps = ["//vendor/github.com/golang/glog:go_default_library"], | ||
) | ||
|
||
go_test( | ||
name = "go_default_test", | ||
srcs = ["keymutex_test.go"], | ||
embed = [":go_default_library"], | ||
) | ||
|
||
filegroup( | ||
name = "package-srcs", | ||
srcs = glob(["**"]), | ||
tags = ["automanaged"], | ||
visibility = ["//visibility:private"], | ||
) | ||
|
||
filegroup( | ||
name = "all-srcs", | ||
srcs = [":package-srcs"], | ||
tags = ["automanaged"], | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
/* | ||
Copyright 2018 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package keymutex | ||
|
||
import ( | ||
"hash/fnv" | ||
"runtime" | ||
"sync" | ||
|
||
"github.com/golang/glog" | ||
) | ||
|
||
// NewHashed returns a new instance of KeyMutex which hashes arbitrary keys to | ||
// a fixed set of locks. `n` specifies number of locks, if n <= 0, we use | ||
// number of cpus. | ||
// Note that because it uses fixed set of locks, different keys may share same | ||
// lock, so it's possible to wait on same lock. | ||
func NewHashed(n int) KeyMutex { | ||
if n <= 0 { | ||
n = runtime.NumCPU() | ||
} | ||
return &hashedKeyMutex{ | ||
mutexes: make([]sync.Mutex, n), | ||
} | ||
} | ||
|
||
type hashedKeyMutex struct { | ||
mutexes []sync.Mutex | ||
} | ||
|
||
// Acquires a lock associated with the specified ID. | ||
func (km *hashedKeyMutex) LockKey(id string) { | ||
glog.V(5).Infof("hashedKeyMutex.LockKey(...) called for id %q\r\n", id) | ||
km.mutexes[km.hash(id)%len(km.mutexes)].Lock() | ||
glog.V(5).Infof("hashedKeyMutex.LockKey(...) for id %q completed.\r\n", id) | ||
} | ||
|
||
// Releases the lock associated with the specified ID. | ||
func (km *hashedKeyMutex) UnlockKey(id string) error { | ||
glog.V(5).Infof("hashedKeyMutex.UnlockKey(...) called for id %q\r\n", id) | ||
km.mutexes[km.hash(id)%len(km.mutexes)].Unlock() | ||
glog.V(5).Infof("hashedKeyMutex.UnlockKey(...) for id %q completed.\r\n", id) | ||
return nil | ||
} | ||
|
||
func (km *hashedKeyMutex) hash(id string) int { | ||
h := fnv.New32a() | ||
h.Write([]byte(id)) | ||
return int(h.Sum32()) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
/* | ||
Copyright 2015 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package keymutex | ||
|
||
// KeyMutex is a thread-safe interface for acquiring locks on arbitrary strings. | ||
type KeyMutex interface { | ||
// Acquires a lock associated with the specified ID, creates the lock if one doesn't already exist. | ||
LockKey(id string) | ||
|
||
// Releases the lock associated with the specified ID. | ||
// Returns an error if the specified ID doesn't exist. | ||
UnlockKey(id string) error | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
/* | ||
Copyright 2015 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package keymutex | ||
|
||
import ( | ||
"testing" | ||
"time" | ||
) | ||
|
||
const ( | ||
callbackTimeout = 1 * time.Second | ||
) | ||
|
||
func newKeyMutexes() []KeyMutex { | ||
return []KeyMutex{ | ||
NewHashed(0), | ||
NewHashed(1), | ||
NewHashed(2), | ||
NewHashed(4), | ||
} | ||
} | ||
|
||
func Test_SingleLock_NoUnlock(t *testing.T) { | ||
for _, km := range newKeyMutexes() { | ||
// Arrange | ||
key := "fakeid" | ||
callbackCh := make(chan interface{}) | ||
|
||
// Act | ||
go lockAndCallback(km, key, callbackCh) | ||
|
||
// Assert | ||
verifyCallbackHappens(t, callbackCh) | ||
} | ||
} | ||
|
||
func Test_SingleLock_SingleUnlock(t *testing.T) { | ||
for _, km := range newKeyMutexes() { | ||
// Arrange | ||
key := "fakeid" | ||
callbackCh := make(chan interface{}) | ||
|
||
// Act & Assert | ||
go lockAndCallback(km, key, callbackCh) | ||
verifyCallbackHappens(t, callbackCh) | ||
km.UnlockKey(key) | ||
} | ||
} | ||
|
||
func Test_DoubleLock_DoubleUnlock(t *testing.T) { | ||
for _, km := range newKeyMutexes() { | ||
// Arrange | ||
key := "fakeid" | ||
callbackCh1stLock := make(chan interface{}) | ||
callbackCh2ndLock := make(chan interface{}) | ||
|
||
// Act & Assert | ||
go lockAndCallback(km, key, callbackCh1stLock) | ||
verifyCallbackHappens(t, callbackCh1stLock) | ||
go lockAndCallback(km, key, callbackCh2ndLock) | ||
verifyCallbackDoesntHappens(t, callbackCh2ndLock) | ||
km.UnlockKey(key) | ||
verifyCallbackHappens(t, callbackCh2ndLock) | ||
km.UnlockKey(key) | ||
} | ||
} | ||
|
||
func lockAndCallback(km KeyMutex, id string, callbackCh chan<- interface{}) { | ||
km.LockKey(id) | ||
callbackCh <- true | ||
} | ||
|
||
func verifyCallbackHappens(t *testing.T, callbackCh <-chan interface{}) bool { | ||
select { | ||
case <-callbackCh: | ||
return true | ||
case <-time.After(callbackTimeout): | ||
t.Fatalf("Timed out waiting for callback.") | ||
return false | ||
} | ||
} | ||
|
||
func verifyCallbackDoesntHappens(t *testing.T, callbackCh <-chan interface{}) bool { | ||
select { | ||
case <-callbackCh: | ||
t.Fatalf("Unexpected callback.") | ||
return false | ||
case <-time.After(callbackTimeout): | ||
return true | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
package(default_visibility = ["//visibility:public"]) | ||
|
||
load( | ||
"@io_bazel_rules_go//go:def.bzl", | ||
"go_library", | ||
"go_test", | ||
) | ||
|
||
go_library( | ||
name = "go_default_library", | ||
srcs = [ | ||
"escape.go", | ||
"line_delimiter.go", | ||
"strings.go", | ||
], | ||
importpath = "k8s.io/kubernetes/pkg/util/strings", | ||
) | ||
|
||
go_test( | ||
name = "go_default_test", | ||
srcs = [ | ||
"escape_test.go", | ||
"line_delimiter_test.go", | ||
"strings_test.go", | ||
], | ||
embed = [":go_default_library"], | ||
) | ||
|
||
filegroup( | ||
name = "package-srcs", | ||
srcs = glob(["**"]), | ||
tags = ["automanaged"], | ||
visibility = ["//visibility:private"], | ||
) | ||
|
||
filegroup( | ||
name = "all-srcs", | ||
srcs = [":package-srcs"], | ||
tags = ["automanaged"], | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
/* | ||
Copyright 2014 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package strings | ||
|
||
import ( | ||
"strings" | ||
) | ||
|
||
// EscapeQualifiedName converts a plugin name, which might contain a / into a | ||
// string that is safe to use on-disk. This assumes that the input has already | ||
// been validates as a qualified name. we use "~" rather than ":" here in case | ||
// we ever use a filesystem that doesn't allow ":". | ||
func EscapeQualifiedName(in string) string { | ||
return strings.Replace(in, "/", "~", -1) | ||
} | ||
|
||
// UnescapeQualifiedName converts an escaped plugin name (as per EscapeQualifiedName) | ||
// back to its normal form. This assumes that the input has already been | ||
// validates as a qualified name. | ||
func UnescapeQualifiedName(in string) string { | ||
return strings.Replace(in, "~", "/", -1) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
/* | ||
Copyright 2016 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package strings | ||
|
||
import ( | ||
"testing" | ||
) | ||
|
||
func TestEscapeQualifiedNameForDisk(t *testing.T) { | ||
testCases := []struct { | ||
input string | ||
output string | ||
}{ | ||
{"kubernetes.io/blah", "kubernetes.io~blah"}, | ||
{"blah/blerg/borg", "blah~blerg~borg"}, | ||
{"kubernetes.io", "kubernetes.io"}, | ||
} | ||
for i, tc := range testCases { | ||
escapee := EscapeQualifiedName(tc.input) | ||
if escapee != tc.output { | ||
t.Errorf("case[%d]: expected (%q), got (%q)", i, tc.output, escapee) | ||
} | ||
original := UnescapeQualifiedName(escapee) | ||
if original != tc.input { | ||
t.Errorf("case[%d]: expected (%q), got (%q)", i, tc.input, original) | ||
} | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does golint pass on this package? I thought it complained about comments not starting with function names. New code should be golint clean (he says, realizing now that he forgot to check for his PRs).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
travis seems to test at least gofmt and vet. though not go lint. if we want to do golint, we should start by adding it there there and not start on this PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's fair. I need to do it with my merged code as well so I'll raise a PR with those changes independent of this one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Getting clean lint and vet should be a requirement for packages promoted to util