feature request: better api for arbitrary keyword argument pass to query ?

[c0b]

from this example https://googlecloudplatform.github.io/google-cloud-python/stable/bigquery-usage.html#querying-data-asynchronous

>>> dataset = client.dataset('dataset_name')
>>> table = dataset.table(name='person_ages')
>>> job = client.run_async_query('fullname-age-query-job', query)
>>> job.destination = table
>>> job.write_disposition= 'WRITE_TRUNCATE'
>>> job.name
'fullname-age-query-job'
>>> job.job_type
'query'
>>> job.created
None
>>> job.state
None

how about a better api by allowing arbitrary keyword list to be appended on the insert job POST request
https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs the keyword destinationTable and writeDisposition here are from rest api reference, this way can keep google-cloud-python library to be a thinner layer, just gives user the best flexibility

>>> job = client.run_async_query(query,
          destinationTable={'datasetId': 'dataset_name', 'tableId': 'person_ages'},
          writeDisposition='WRITE_TRUNCATE',
          )

in python it can be implemented as dict as last argument:

def run_async_query(query, **kwargs):
    # populate from kwargs to be on the `configuration.query` in the post request body,
    # it could optionally have jobId

Btw, from #3209 I am not puting a job_name / jobid field because I don't think that's necessary, the run_async_query shouldn't require user to provide a jobid, in the rest api insert job request, if jobid is absent, server side would generate a unique id

[lukesneeringer]

Hi @c0b, thanks for reporting. We will evaluate this.

[tswast]

👍 on a way to provide job config directly.

Closing this since I'm tracking this internally for GA redesign.

[tswast]

Actually, reopening since this might not make it into the first iteration of the GA library.

I think the most appropriate place for this while preserving existing behavior will be to add an argument to the .begin() method of the various job classes. Example:

query_job = client.query(query_string)
query_job.begin(config_overrides={
        'destinationTable': {
            'projectId': 'some-project',
            'datasetId': 'some-dataset',
            'tableId': 'some-table'
        }
})

Any properties that appear in the config_overrides dictionary will be merged into the API resource representation before calling the jobs insert HTTP method.

[lukesneeringer]

@tswast Go ahead and track it in your doc in lieu of here, and just have a section in that doc for "things that might be post-GA".