BigQuery: Add support to FormatOptions for AVRO #1441
Assignees
Labels
api: bigquery
Issues related to the BigQuery API.
Comments
tswast
added
api: bigquery
|
This is additive - we will work on it after beta is declared. |
garrettjonesgoogle
removed
type: bug
vam-google
added a commit
that referenced
this issue
Jan 27, 2017
#1441 Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
vam-google
added a commit
to vam-google/google-cloud-java
that referenced
this issue
Jan 27, 2017
googleapis#1441 Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
vam-google
added a commit
that referenced
this issue
Jan 28, 2017
BigQuery: Add support to FormatOptions for AVRO #1441 Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
|
Resolved. |
pongad
added a commit
that referenced
this issue
Feb 7, 2017
* Update version to 0.8.1-SNAPSHOT (#1467) Also, update versions in README to 0.8.0 * Add link to Maven Central for maven-central badge. (#1468) Used to link to the image, which wasn't super useful. * fix more races in pubsub tests Previously BlockingProcessStreamReader has a terminate() method, used to tell the Reader to stop reading from the emulator process. This causes an inter-process race. If the Reader stops before reading emulator's output, the emulator process will hang as it tries to write to stdout/stderr as there's no one to read from the other side of the pipe. Since there is no way to safely stop the Reader, this commit deletes the method and its associated test. Additionally, the timeout for LocalSystemTest is increased to 3 minutes, since the emulator, somehow, consistently takes just longer than a minute to shut down. * Regenerating SPI layer (#1501) * Converting Error Reporting and Monitoring to use resource name types * Removing formatX/parseX methods from pubsub, converting usage of the same to resource name types * New methods in Logging and PubSub * Updating grpc dependency to 1.0.3 (#1504) * Release 0.8.1 (#1512) * Fix code snippet (wrong method name) in README.md Original code snippet in _"Querying data"_ section: `..while (!queryResponse.jobComplete()) {..` This results in a compile error: _"Cannot resolve method jobComplete()"_ The correct method is `jobCompleted()` * Updating version in README files. [ci skip] * Update version to 0.8.2-alpha-SNAPSHOT * Allow path in URIs passed to newFileSystem (#1470) * This makes it easier for users who start with a URI describing a full path to get a FileSystem that can work with that path, since they no longer have to needlessly remove the path from the URI. Note that Oracle's description of newFileSystem [1] puts no restriction on the passed URI. [1] https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map) * Preventing logging re-entrance at FINE level (#1523) * Preventing logging re-entrance at FINE level Also: * Reducing the scope of synchronized blocks * Removing logger exclusions except for Http2FrameLogger * Add a PathMatcher for CloudStorageFileSystem (#1469) Add a test, as well. We reuse the default PathMatcher since it does a fine job of globbing files. * Set timestamp from LogRecord (#1533) * Initialize the default MonitoredResource from a GAE environment (#1535) * BigQuery: Add support to FormatOptions for AVRO #1441 Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required. * Reverting changed commited by mistake before review. * BigQuery: Add support to FormatOptions for AVRO #1441 Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required. * use RpcFuture and remove old BundlingSettings (#1572) * use RpcFuture and remove old BundlingSettings * Release 0.8.2 Note: This version was accidentally released to Sonatype because of experimenting with deployment commands, combined with the fact that autoReleaseAfterClose is set to true. Since releases can't be taken back, we might as well own up to the release and push the code forward. * Updating version in README files. [ci skip] * Fixing javadoc error in GaeFlexLoggingEnhancer (#1582) * get tests to compile and pass
chingor13
pushed a commit
to googleapis/java-core
that referenced
this issue
Aug 27, 2019
* Update version to 0.8.1-SNAPSHOT (#1467)
Also, update versions in README to 0.8.0
* Add link to Maven Central for maven-central badge. (#1468)
Used to link to the image, which wasn't super useful.
* fix more races in pubsub tests
Previously BlockingProcessStreamReader has a terminate() method,
used to tell the Reader to stop reading from the emulator process.
This causes an inter-process race.
If the Reader stops before reading emulator's output,
the emulator process will hang as it tries to write to stdout/stderr
as there's no one to read from the other side of the pipe.
Since there is no way to safely stop the Reader,
this commit deletes the method and its associated test.
Additionally, the timeout for LocalSystemTest is increased to 3 minutes,
since the emulator, somehow, consistently takes just longer than a
minute to shut down.
* Regenerating SPI layer (#1501)
* Converting Error Reporting and Monitoring to use resource name types
* Removing formatX/parseX methods from pubsub, converting usage of the
same to resource name types
* New methods in Logging and PubSub
* Updating grpc dependency to 1.0.3 (#1504)
* Release 0.8.1 (#1512)
* Fix code snippet (wrong method name) in README.md
Original code snippet in _"Querying data"_ section:
`..while (!queryResponse.jobComplete()) {..`
This results in a compile error:
_"Cannot resolve method jobComplete()"_
The correct method is `jobCompleted()`
* Updating version in README files. [ci skip]
* Update version to 0.8.2-alpha-SNAPSHOT
* Allow path in URIs passed to newFileSystem (#1470)
* This makes it easier for users who start with a URI describing a full
path to get a FileSystem that can work with that path, since they no
longer have to needlessly remove the path from the URI.
Note that Oracle's description of newFileSystem [1] puts no restriction
on the passed URI.
[1]
https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map)
* Preventing logging re-entrance at FINE level (#1523)
* Preventing logging re-entrance at FINE level
Also:
* Reducing the scope of synchronized blocks
* Removing logger exclusions except for Http2FrameLogger
* Add a PathMatcher for CloudStorageFileSystem (#1469)
Add a test, as well.
We reuse the default PathMatcher since it does a fine job of globbing files.
* Set timestamp from LogRecord (#1533)
* Initialize the default MonitoredResource from a GAE environment (#1535)
* BigQuery: Add support to FormatOptions for AVRO
googleapis/google-cloud-java#1441
Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
* Reverting changed commited by mistake before review.
* BigQuery: Add support to FormatOptions for AVRO
googleapis/google-cloud-java#1441
Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
* use RpcFuture and remove old BundlingSettings (#1572)
* use RpcFuture and remove old BundlingSettings
* Release 0.8.2
Note: This version was accidentally released to Sonatype because of
experimenting with deployment commands, combined with the fact that
autoReleaseAfterClose is set to true. Since releases can't be taken
back, we might as well own up to the release and push the code
forward.
* Updating version in README files. [ci skip]
* Fixing javadoc error in GaeFlexLoggingEnhancer (#1582)
* get tests to compile and pass
chingor13
pushed a commit
to googleapis/java-core
that referenced
this issue
Sep 6, 2019
* Update version to 0.8.1-SNAPSHOT (#1467)
Also, update versions in README to 0.8.0
* Add link to Maven Central for maven-central badge. (#1468)
Used to link to the image, which wasn't super useful.
* fix more races in pubsub tests
Previously BlockingProcessStreamReader has a terminate() method,
used to tell the Reader to stop reading from the emulator process.
This causes an inter-process race.
If the Reader stops before reading emulator's output,
the emulator process will hang as it tries to write to stdout/stderr
as there's no one to read from the other side of the pipe.
Since there is no way to safely stop the Reader,
this commit deletes the method and its associated test.
Additionally, the timeout for LocalSystemTest is increased to 3 minutes,
since the emulator, somehow, consistently takes just longer than a
minute to shut down.
* Regenerating SPI layer (#1501)
* Converting Error Reporting and Monitoring to use resource name types
* Removing formatX/parseX methods from pubsub, converting usage of the
same to resource name types
* New methods in Logging and PubSub
* Updating grpc dependency to 1.0.3 (#1504)
* Release 0.8.1 (#1512)
* Fix code snippet (wrong method name) in README.md
Original code snippet in _"Querying data"_ section:
`..while (!queryResponse.jobComplete()) {..`
This results in a compile error:
_"Cannot resolve method jobComplete()"_
The correct method is `jobCompleted()`
* Updating version in README files. [ci skip]
* Update version to 0.8.2-alpha-SNAPSHOT
* Allow path in URIs passed to newFileSystem (#1470)
* This makes it easier for users who start with a URI describing a full
path to get a FileSystem that can work with that path, since they no
longer have to needlessly remove the path from the URI.
Note that Oracle's description of newFileSystem [1] puts no restriction
on the passed URI.
[1]
https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map)
* Preventing logging re-entrance at FINE level (#1523)
* Preventing logging re-entrance at FINE level
Also:
* Reducing the scope of synchronized blocks
* Removing logger exclusions except for Http2FrameLogger
* Add a PathMatcher for CloudStorageFileSystem (#1469)
Add a test, as well.
We reuse the default PathMatcher since it does a fine job of globbing files.
* Set timestamp from LogRecord (#1533)
* Initialize the default MonitoredResource from a GAE environment (#1535)
* BigQuery: Add support to FormatOptions for AVRO
googleapis/google-cloud-java#1441
Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
* Reverting changed commited by mistake before review.
* BigQuery: Add support to FormatOptions for AVRO
googleapis/google-cloud-java#1441
Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
* use RpcFuture and remove old BundlingSettings (#1572)
* use RpcFuture and remove old BundlingSettings
* Release 0.8.2
Note: This version was accidentally released to Sonatype because of
experimenting with deployment commands, combined with the fact that
autoReleaseAfterClose is set to true. Since releases can't be taken
back, we might as well own up to the release and push the code
forward.
* Updating version in README files. [ci skip]
* Fixing javadoc error in GaeFlexLoggingEnhancer (#1582)
* get tests to compile and pass
chingor13
pushed a commit
to googleapis/java-trace
that referenced
this issue
Sep 6, 2019
* Update version to 0.8.1-SNAPSHOT (#1467)
Also, update versions in README to 0.8.0
* Add link to Maven Central for maven-central badge. (#1468)
Used to link to the image, which wasn't super useful.
* fix more races in pubsub tests
Previously BlockingProcessStreamReader has a terminate() method,
used to tell the Reader to stop reading from the emulator process.
This causes an inter-process race.
If the Reader stops before reading emulator's output,
the emulator process will hang as it tries to write to stdout/stderr
as there's no one to read from the other side of the pipe.
Since there is no way to safely stop the Reader,
this commit deletes the method and its associated test.
Additionally, the timeout for LocalSystemTest is increased to 3 minutes,
since the emulator, somehow, consistently takes just longer than a
minute to shut down.
* Regenerating SPI layer (#1501)
* Converting Error Reporting and Monitoring to use resource name types
* Removing formatX/parseX methods from pubsub, converting usage of the
same to resource name types
* New methods in Logging and PubSub
* Updating grpc dependency to 1.0.3 (#1504)
* Release 0.8.1 (#1512)
* Fix code snippet (wrong method name) in README.md
Original code snippet in _"Querying data"_ section:
`..while (!queryResponse.jobComplete()) {..`
This results in a compile error:
_"Cannot resolve method jobComplete()"_
The correct method is `jobCompleted()`
* Updating version in README files. [ci skip]
* Update version to 0.8.2-alpha-SNAPSHOT
* Allow path in URIs passed to newFileSystem (#1470)
* This makes it easier for users who start with a URI describing a full
path to get a FileSystem that can work with that path, since they no
longer have to needlessly remove the path from the URI.
Note that Oracle's description of newFileSystem [1] puts no restriction
on the passed URI.
[1]
https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map)
* Preventing logging re-entrance at FINE level (#1523)
* Preventing logging re-entrance at FINE level
Also:
* Reducing the scope of synchronized blocks
* Removing logger exclusions except for Http2FrameLogger
* Add a PathMatcher for CloudStorageFileSystem (#1469)
Add a test, as well.
We reuse the default PathMatcher since it does a fine job of globbing files.
* Set timestamp from LogRecord (#1533)
* Initialize the default MonitoredResource from a GAE environment (#1535)
* BigQuery: Add support to FormatOptions for AVRO
googleapis/google-cloud-java#1441
Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
* Reverting changed commited by mistake before review.
* BigQuery: Add support to FormatOptions for AVRO
googleapis/google-cloud-java#1441
Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
* use RpcFuture and remove old BundlingSettings (#1572)
* use RpcFuture and remove old BundlingSettings
* Release 0.8.2
Note: This version was accidentally released to Sonatype because of
experimenting with deployment commands, combined with the fact that
autoReleaseAfterClose is set to true. Since releases can't be taken
back, we might as well own up to the release and push the code
forward.
* Updating version in README files. [ci skip]
* Fixing javadoc error in GaeFlexLoggingEnhancer (#1582)
* get tests to compile and pass
chingor13
pushed a commit
to googleapis/java-speech
that referenced
this issue
Sep 9, 2019
* Update version to 0.8.1-SNAPSHOT (#1467)
Also, update versions in README to 0.8.0
* Add link to Maven Central for maven-central badge. (#1468)
Used to link to the image, which wasn't super useful.
* fix more races in pubsub tests
Previously BlockingProcessStreamReader has a terminate() method,
used to tell the Reader to stop reading from the emulator process.
This causes an inter-process race.
If the Reader stops before reading emulator's output,
the emulator process will hang as it tries to write to stdout/stderr
as there's no one to read from the other side of the pipe.
Since there is no way to safely stop the Reader,
this commit deletes the method and its associated test.
Additionally, the timeout for LocalSystemTest is increased to 3 minutes,
since the emulator, somehow, consistently takes just longer than a
minute to shut down.
* Regenerating SPI layer (#1501)
* Converting Error Reporting and Monitoring to use resource name types
* Removing formatX/parseX methods from pubsub, converting usage of the
same to resource name types
* New methods in Logging and PubSub
* Updating grpc dependency to 1.0.3 (#1504)
* Release 0.8.1 (#1512)
* Fix code snippet (wrong method name) in README.md
Original code snippet in _"Querying data"_ section:
`..while (!queryResponse.jobComplete()) {..`
This results in a compile error:
_"Cannot resolve method jobComplete()"_
The correct method is `jobCompleted()`
* Updating version in README files. [ci skip]
* Update version to 0.8.2-alpha-SNAPSHOT
* Allow path in URIs passed to newFileSystem (#1470)
* This makes it easier for users who start with a URI describing a full
path to get a FileSystem that can work with that path, since they no
longer have to needlessly remove the path from the URI.
Note that Oracle's description of newFileSystem [1] puts no restriction
on the passed URI.
[1]
https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map)
* Preventing logging re-entrance at FINE level (#1523)
* Preventing logging re-entrance at FINE level
Also:
* Reducing the scope of synchronized blocks
* Removing logger exclusions except for Http2FrameLogger
* Add a PathMatcher for CloudStorageFileSystem (#1469)
Add a test, as well.
We reuse the default PathMatcher since it does a fine job of globbing files.
* Set timestamp from LogRecord (#1533)
* Initialize the default MonitoredResource from a GAE environment (#1535)
* BigQuery: Add support to FormatOptions for AVRO
googleapis/google-cloud-java#1441
Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
* Reverting changed commited by mistake before review.
* BigQuery: Add support to FormatOptions for AVRO
googleapis/google-cloud-java#1441
Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required.
* use RpcFuture and remove old BundlingSettings (#1572)
* use RpcFuture and remove old BundlingSettings
* Release 0.8.2
Note: This version was accidentally released to Sonatype because of
experimenting with deployment commands, combined with the fact that
autoReleaseAfterClose is set to true. Since releases can't be taken
back, we might as well own up to the release and push the code
forward.
* Updating version in README files. [ci skip]
* Fixing javadoc error in GaeFlexLoggingEnhancer (#1582)
* get tests to compile and pass
chingor13
pushed a commit
to googleapis/java-storage-nio
that referenced
this issue
Nov 26, 2019
* Update version to 0.8.1-SNAPSHOT (#1467) Also, update versions in README to 0.8.0 * Add link to Maven Central for maven-central badge. (#1468) Used to link to the image, which wasn't super useful. * fix more races in pubsub tests Previously BlockingProcessStreamReader has a terminate() method, used to tell the Reader to stop reading from the emulator process. This causes an inter-process race. If the Reader stops before reading emulator's output, the emulator process will hang as it tries to write to stdout/stderr as there's no one to read from the other side of the pipe. Since there is no way to safely stop the Reader, this commit deletes the method and its associated test. Additionally, the timeout for LocalSystemTest is increased to 3 minutes, since the emulator, somehow, consistently takes just longer than a minute to shut down. * Regenerating SPI layer (#1501) * Converting Error Reporting and Monitoring to use resource name types * Removing formatX/parseX methods from pubsub, converting usage of the same to resource name types * New methods in Logging and PubSub * Updating grpc dependency to 1.0.3 (#1504) * Release 0.8.1 (#1512) * Fix code snippet (wrong method name) in README.md Original code snippet in _"Querying data"_ section: `..while (!queryResponse.jobComplete()) {..` This results in a compile error: _"Cannot resolve method jobComplete()"_ The correct method is `jobCompleted()` * Updating version in README files. [ci skip] * Update version to 0.8.2-alpha-SNAPSHOT * Allow path in URIs passed to newFileSystem (#1470) * This makes it easier for users who start with a URI describing a full path to get a FileSystem that can work with that path, since they no longer have to needlessly remove the path from the URI. Note that Oracle's description of newFileSystem [1] puts no restriction on the passed URI. [1] https://docs.oracle.com/javase/7/docs/api/java/nio/file/FileSystems.html#newFileSystem(java.net.URI,%20java.util.Map) * Preventing logging re-entrance at FINE level (#1523) * Preventing logging re-entrance at FINE level Also: * Reducing the scope of synchronized blocks * Removing logger exclusions except for Http2FrameLogger * Add a PathMatcher for CloudStorageFileSystem (#1469) Add a test, as well. We reuse the default PathMatcher since it does a fine job of globbing files. * Set timestamp from LogRecord (#1533) * Initialize the default MonitoredResource from a GAE environment (#1535) * BigQuery: Add support to FormatOptions for AVRO googleapis/google-cloud-java#1441 Added new constant in FormatOptions, and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required. * Reverting changed commited by mistake before review. * BigQuery: Add support to FormatOptions for AVRO googleapis/google-cloud-java#1441 Added new constant in FormatOptions and a corresponding factory method. Updated test cases. Confirmed that AVRO does not require special treatment (like CSV does), so no additional changes are required. * use RpcFuture and remove old BundlingSettings (#1572) * use RpcFuture and remove old BundlingSettings * Release 0.8.2 Note: This version was accidentally released to Sonatype because of experimenting with deployment commands, combined with the fact that autoReleaseAfterClose is set to true. Since releases can't be taken back, we might as well own up to the release and push the code forward. * Updating version in README files. [ci skip] * Fixing javadoc error in GaeFlexLoggingEnhancer (#1582) * get tests to compile and pass
github-actions bot
pushed a commit
that referenced
this issue
Aug 16, 2022
…1.0 (#1441) [](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [com.google.cloud:libraries-bom](https://cloud.google.com/java/docs/bom) ([source](https://togithub.com/googleapis/java-cloud-bom)) | `26.0.0` -> `26.1.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Renovate will not automatically rebase this PR, because other commits have been found. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox. ⚠ **Warning**: custom changes will be lost. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-bigquerydatatransfer). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzMi4xNTguMCIsInVwZGF0ZWRJblZlciI6IjMyLjE1OC4wIn0=-->
](https://renovatebot.com){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
configuration.load.sourceFormatacceptsCSV,DATASTORE_BACKUP,NEWLINE_DELIMITED_JSON, andAVRO. See the API reference docs.FormatOptions is missing an option for Avro.
The text was updated successfully, but these errors were encountered: