Skip to content

Commit

Permalink
merge master into pubsub-hp (#1591)
Browse files Browse the repository at this point in the history
* Update version to 0.8.1-SNAPSHOT (#1467)

Also, update versions in README to 0.8.0

* Add link to Maven Central for maven-central badge. (#1468)

Used to link to the image, which wasn't super useful.

* fix more races in pubsub tests

Previously BlockingProcessStreamReader has a terminate() method,
used to tell the Reader to stop reading from the emulator process.

This causes an inter-process race.
If the Reader stops before reading emulator's output,
the emulator process will hang as it tries to write to stdout/stderr
as there's no one to read from the other side of the pipe.

Since there is no way to safely stop the Reader,
this commit deletes the method and its associated test.

Additionally, the timeout for LocalSystemTest is increased to 3 minutes,
since the emulator, somehow, consistently takes just longer than a
minute to shut down.

* Regenerating SPI layer (#1501)

* Converting Error Reporting and Monitoring to use resource name types
* Removing formatX/parseX methods from pubsub, converting usage of the
  same to resource name types
* New methods in Logging and PubSub

* Updating grpc dependency to 1.0.3 (#1504)

* Release 0.8.1 (#1512)

* Fix code snippet (wrong method name) in README.md

Original code snippet in _"Querying data"_ section: 

`..while (!queryResponse.jobComplete()) {..`

This results in a compile error: 

_"Cannot resolve method jobComplete()"_

The correct method is `jobCompleted()`

* Updating version in README files. [ci skip]

* Update version to 0.8.2-alpha-SNAPSHOT

* Allow path in URIs passed to newFileSystem (#1470)

* This makes it easier for users who start with a URI describing a full
path to get a FileSystem that can work with that path, since they no
longer have to needlessly remove the path from the URI.

Note that Oracle's description of newFileSystem [1] puts no restriction
on the passed URI.

[1]
https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map)

* Preventing logging re-entrance at FINE level (#1523)

* Preventing logging re-entrance at FINE level

Also:
* Reducing the scope of synchronized blocks
* Removing logger exclusions except for Http2FrameLogger

* Add a PathMatcher for CloudStorageFileSystem (#1469)

Add a test, as well.
We reuse the default PathMatcher since it does a fine job of globbing files.

* Set timestamp from LogRecord (#1533)

* Initialize the default MonitoredResource from a GAE environment (#1535)

* BigQuery: Add support to FormatOptions for AVRO

googleapis/google-cloud-java#1441

Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.

* Reverting changed commited by mistake before review.

* BigQuery: Add support to FormatOptions for AVRO

googleapis/google-cloud-java#1441

Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.

* use RpcFuture and remove old BundlingSettings (#1572)

* use RpcFuture and remove old BundlingSettings

* Release 0.8.2

Note: This version was accidentally released to Sonatype because of
experimenting with deployment commands, combined with the fact that
autoReleaseAfterClose is set to true. Since releases can't be taken
back, we might as well own up to the release and push the code
forward.

* Updating version in README files. [ci skip]

* Fixing javadoc error in GaeFlexLoggingEnhancer (#1582)

* get tests to compile and pass
  • Loading branch information
pongad authored Feb 7, 2017
1 parent 37e382a commit 2f7f115
Show file tree
Hide file tree
Showing 8 changed files with 46 additions and 17 deletions.
4 changes: 2 additions & 2 deletions google-cloud-nio-examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ To run this example:
4. Run the sample with:

```
java -cp google-cloud-contrib/google-cloud-nio/target/google-cloud-nio-0.7.1-SNAPSHOT-shaded.jar:google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.7.1-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
java -cp google-cloud-contrib/google-cloud-nio/target/google-cloud-nio-0.8.3-alpha-SNAPSHOT-shaded.jar:google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.8.3-alpha-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
```

Notice that it lists Google Cloud Storage, which it wouldn't if you ran it without the NIO jar:
```
java -cp google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.7.1-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
java -cp google-cloud-contrib/google-cloud-nio-examples/target/google-cloud-nio-examples-0.8.3-alpha-SNAPSHOT.jar com.google.cloud.nio.examples.ListFilesystems
```

The sample doesn't have anything about Google Cloud Storage in it. It gets that ability from the NIO
Expand Down
2 changes: 1 addition & 1 deletion google-cloud-nio-examples/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<parent>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-contrib</artifactId>
<version>0.8.1-SNAPSHOT</version>
<version>0.8.2-alpha</version>
</parent>
<properties>
<site.installationModule>google-cloud-nio-examples</site.installationModule>
Expand Down
6 changes: 3 additions & 3 deletions google-cloud-nio/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,16 @@ If you are using Maven, add this to your pom.xml file
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-nio</artifactId>
<version>0.8.0</version>
<version>0.8.2-alpha</version>
</dependency>
```
If you are using Gradle, add this to your dependencies
```Groovy
compile 'com.google.cloud:google-cloud-nio:0.8.0'
compile 'com.google.cloud:google-cloud-nio:0.8.2-alpha'
```
If you are using SBT, add this to your dependencies
```Scala
libraryDependencies += "com.google.cloud" % "google-cloud-nio" % "0.8.0"
libraryDependencies += "com.google.cloud" % "google-cloud-nio" % "0.8.2-alpha"
```

Example Applications
Expand Down
2 changes: 1 addition & 1 deletion google-cloud-nio/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<parent>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-contrib</artifactId>
<version>0.8.1-SNAPSHOT</version>
<version>0.8.2-alpha</version>
</parent>
<properties>
<site.installationModule>google-cloud-nio</site.installationModule>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import java.net.URISyntaxException;
import java.nio.file.FileStore;
import java.nio.file.FileSystem;
import java.nio.file.FileSystems;
import java.nio.file.Path;
import java.nio.file.PathMatcher;
import java.nio.file.WatchService;
Expand Down Expand Up @@ -210,13 +211,9 @@ public Set<String> supportedFileAttributeViews() {
return SUPPORTED_VIEWS;
}

/**
* Throws {@link UnsupportedOperationException} because this feature hasn't been implemented yet.
*/
@Override
public PathMatcher getPathMatcher(String syntaxAndPattern) {
// TODO(#813): Implement me.
throw new UnsupportedOperationException();
return FileSystems.getDefault().getPathMatcher(syntaxAndPattern);
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,14 +170,15 @@ public CloudStorageFileSystem getFileSystem(URI uri) {
}

/**
* Returns Cloud Storage file system, provided a URI with no path, e.g. {@code gs://bucket}.
* Returns Cloud Storage file system, provided a URI, e.g. {@code gs://bucket}.
* The URI can include a path component (that will be ignored).
*
* @param uri bucket and current working directory, e.g. {@code gs://bucket}
* @param env map of configuration options, whose keys correspond to the method names of
* {@link CloudStorageConfiguration.Builder}. However you are not allowed to set the working
* directory, as that should be provided in the {@code uri}
* @throws IllegalArgumentException if {@code uri} specifies a user, query, fragment, or scheme is
* not {@value CloudStorageFileSystem#URI_SCHEME}
* @throws IllegalArgumentException if {@code uri} specifies a port, user, query, or fragment, or
* if scheme is not {@value CloudStorageFileSystem#URI_SCHEME}
*/
@Override
public CloudStorageFileSystem newFileSystem(URI uri, Map<String, ?> env) {
Expand All @@ -191,11 +192,10 @@ public CloudStorageFileSystem newFileSystem(URI uri, Map<String, ?> env) {
CloudStorageFileSystem.URI_SCHEME, uri);
checkArgument(
uri.getPort() == -1
&& isNullOrEmpty(uri.getPath())
&& isNullOrEmpty(uri.getQuery())
&& isNullOrEmpty(uri.getFragment())
&& isNullOrEmpty(uri.getUserInfo()),
"GCS FileSystem URIs mustn't have: port, userinfo, path, query, or fragment: %s",
"GCS FileSystem URIs mustn't have: port, userinfo, query, or fragment: %s",
uri);
CloudStorageUtil.checkBucket(uri.getHost());
initStorage();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@
import java.nio.file.OpenOption;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/**
* Unit tests for {@link CloudStorageFileSystemProvider}.
Expand Down Expand Up @@ -644,6 +646,12 @@ public void testProviderEquals() {
assertThat(path1.getFileSystem().provider()).isNotEqualTo(path3.getFileSystem().provider());
}

@Test
public void testNewFileSystem() throws IOException {
Map<String,String> env = new HashMap<>();
FileSystems.newFileSystem(URI.create("gs://bucket/path/to/file"), env);
}

private static CloudStorageConfiguration permitEmptyPathComponents(boolean value) {
return CloudStorageConfiguration.builder().permitEmptyPathComponents(value).build();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.PathMatcher;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
Expand Down Expand Up @@ -159,4 +160,27 @@ public void testListFiles() throws IOException {
assertThat(got).containsExactlyElementsIn(goodPaths);
}
}

@Test
public void testMatcher() throws IOException {
try (FileSystem fs = CloudStorageFileSystem.forBucket("bucket")) {
String pattern1 = "glob:*.java";
PathMatcher javaFileMatcher = fs.getPathMatcher(pattern1);
assertMatches(fs, javaFileMatcher, "a.java", true);
assertMatches(fs, javaFileMatcher, "a.text", false);
assertMatches(fs, javaFileMatcher, "folder/c.java", true);
assertMatches(fs, javaFileMatcher, "d", false);

String pattern2 = "glob:*.{java,text}";
PathMatcher javaAndTextFileMatcher = fs.getPathMatcher(pattern2);
assertMatches(fs, javaAndTextFileMatcher, "a.java", true);
assertMatches(fs, javaAndTextFileMatcher, "a.text", true);
assertMatches(fs, javaAndTextFileMatcher, "folder/c.java", true);
assertMatches(fs, javaAndTextFileMatcher, "d", false);
}
}

private void assertMatches(FileSystem fs, PathMatcher matcher, String toMatch, boolean expected) {
assertThat(matcher.matches(fs.getPath(toMatch).getFileName())).isEqualTo(expected);
}
}

0 comments on commit 2f7f115

Please sign in to comment.