Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow empty indexed arrays. #3751

Closed
wants to merge 11 commits into from
Closed

Allow empty indexed arrays. #3751

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
efa90d8
Non-functional datastore fix for #3152.
Mar 16, 2017
b96eaca
Make CONTRIBUTING.rst be up to date. (#3750)
lukesneeringer Aug 8, 2017
fe757be
session.run_in_transaction returns the callback's return value. (#3753)
lukesneeringer Aug 8, 2017
abfec70
Add google.api.core.exceptions (#3738)
Aug 8, 2017
90e6fe7
Bump GAX dependency to 0.15.14. (#3752)
lukesneeringer Aug 8, 2017
aaaaf7d
Reference valid input formats in API docs. (#3758)
lukesneeringer Aug 8, 2017
1ff7708
Make exclude_from_indexes a set, and public API. (#3756)
lukesneeringer Aug 8, 2017
cdadf4c
Move google.cloud.future to google.api.core (#3764)
Aug 8, 2017
18478f8
Non-functional datastore fix for #3152.
Mar 16, 2017
8d36c47
Merge branch 'datastore-issue-3152' of github.com:lukesneeringer/goog…
Aug 8, 2017
ef4ed3a
Fix the unit test.
Aug 8, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
229 changes: 60 additions & 169 deletions CONTRIBUTING.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ In order to add a feature to ``google-cloud-python``:
documentation (in ``docs/``).

- The feature must work fully on the following CPython versions: 2.7,
3.4, and 3.5 on both UNIX and Windows.
3.4, 3.5, and 3.6 on both UNIX and Windows.

- The feature must not add unnecessary dependencies (where
"unnecessary" is of course subjective, but new dependencies should
Expand Down Expand Up @@ -57,7 +57,7 @@ You'll have to create a development environment to hack on
Now your local repo is set up such that you will push changes to your GitHub
repo, from which you can submit a pull request.

To work on the codebase and run the tests, we recommend using ``tox``,
To work on the codebase and run the tests, we recommend using ``nox``,
but you can also use a ``virtualenv`` of your own creation.

.. _repo: https://github.com/GoogleCloudPlatform/google-cloud-python
Expand All @@ -68,11 +68,15 @@ Using a custom ``virtualenv``
- To create a virtualenv in which to install ``google-cloud-python``::

$ cd ${HOME}/hack-on-google-cloud-python
$ virtualenv --python python2.7 ${ENV_NAME}
$ virtualenv --python python3.6 ${ENV_NAME}

You can choose which Python version you want to use by passing a ``--python``
flag to ``virtualenv``. For example, ``virtualenv --python python2.7``
chooses the Python 2.7 interpreter to be installed.
flag to ``virtualenv``. For example, ``virtualenv --python python3.6``
chooses the Python 3.6 interpreter to be installed.

.. note::
We recommend developing in Python 3, and using the test suite to
ensure compatibility with Python 2.

- From here on in within these instructions, the
``${HOME}/hack-on-google-cloud-python/${ENV_NAME}`` virtual environment you
Expand All @@ -91,43 +95,32 @@ Using a custom ``virtualenv``
Unfortunately using ``setup.py develop`` is not possible with this
project, because it uses `namespace packages`_.

Using ``tox``
Using ``nox``
=============

- To test your changes, run unit tests with ``tox``::

$ tox -e py27
$ tox -e py34
$ ...

- If you'd like to poke around your code in an interpreter, let
``tox`` install the environment of your choice::

$ # Install only; without running tests
$ tox -e ${ENV} --recreate --notest
We use `nox`_ to instrument our tests.

After doing this, you can activate the virtual environment and
use the interpreter from that environment::
- To test your changes, run unit tests with ``nox``::

$ source .tox/${ENV}/bin/activate
(ENV) $ .tox/${ENV}/bin/python
$ nox -f datastore/nox.py -s "unit_tests(python_version='2.7')"
$ nox -f datastore/nox.py -s "unit_tests(python_version='3.4')"
$ ...

Unfortunately, your changes to the source tree won't be picked up
by the ``tox`` environment, so if you make changes, you'll need
to again ``--recreate`` the environment.
.. note::

- To run unit tests on a restricted set of packages::
The unit tests and system tests are contained in the individual
``nox.py`` files in each directory; substitute ``datastore`` in the
example above with the package of your choice.

$ tox -e py27 -- core datastore

Alternatively, you can just navigate directly to the package you are
currently developing and run tests there::

$ export GIT_ROOT=$(pwd)
$ cd ${GIT_ROOT}/core/
$ tox -e py27
$ cd ${GIT_ROOT}/datastore/
$ tox -e py27
$ nox -s "unit_tests(python_version='3.6')"

.. nox: https://pypi.org/project/nox-automation/

Note on Editable Installs / Develop Mode
========================================
Expand Down Expand Up @@ -162,13 +155,13 @@ On Debian/Ubuntu::
Coding Style
************

- PEP8 compliance, with exceptions defined in ``tox.ini``.
If you have ``tox`` installed, you can test that you have not introduced
- PEP8 compliance, with exceptions defined in the linter configuration.
If you have ``nox`` installed, you can test that you have not introduced
any non-compliant code via::

$ tox -e lint
$ nox -s lint

- In order to make ``tox -e lint`` run faster, you can set some environment
- In order to make ``nox -s lint`` run faster, you can set some environment
variables::

export GOOGLE_CLOUD_TESTING_REMOTE="upstream"
Expand All @@ -185,49 +178,20 @@ Exceptions to PEP8:
"Function-Under-Test"), which is PEP8-incompliant, but more readable.
Some also use a local variable, ``MUT`` (short for "Module-Under-Test").

*************
Running Tests
*************

- To run all tests for ``google-cloud-python`` on a single Python version, run
``py.test`` from your development virtualenv (See
`Using a Development Checkout`_ above).

.. _Using a Development Checkout: #using-a-development-checkout

- To run the full set of ``google-cloud-python`` tests on all platforms, install
``tox`` (https://tox.readthedocs.io/en/latest/) into a system Python. The
``tox`` console script will be installed into the scripts location for that
Python. While ``cd``'-ed to the ``google-cloud-python`` checkout root
directory (it contains ``tox.ini``), invoke the ``tox`` console script.
This will read the ``tox.ini`` file and execute the tests on multiple
Python versions and platforms; while it runs, it creates a ``virtualenv`` for
each version/platform combination. For example::

$ sudo --set-home /usr/bin/pip install tox
$ cd ${HOME}/hack-on-google-cloud-python/
$ /usr/bin/tox

.. _Using a Development Checkout: #using-a-development-checkout

********************
Running System Tests
********************

- To run system tests you can execute::

$ tox -e system-tests
$ tox -e system-tests3

or run only system tests for a particular package via::
- To run system tests for a given package, you can execute::

$ python system_tests/run_system_test.py --package {package}
$ python3 system_tests/run_system_test.py --package {package}
$ nox -f datastore/nox.py -s "system_tests(python_version='3.6')"
$ nox -f datastore/nox.py -s "system_tests(python_version='2.7')"

To run a subset of the system tests::
.. note::

$ tox -e system-tests -- datastore storage
$ python system_tests/attempt_system_tests.py datastore storage
System tests are only configured to run under Python 2.7 and
Python 3.6. For expediency, we do not run them in older versions
of Python 3.

This alone will not run the tests. You'll need to change some local
auth settings and change some configuration in your project to
Expand Down Expand Up @@ -270,90 +234,21 @@ Running System Tests
- For datastore query tests, you'll need stored data in your dataset.
To populate this data, run::

$ python system_tests/populate_datastore.py
$ python datastore/tests/system/utils/populate_datastore.py

- If you make a mistake during development (i.e. a failing test that
prevents clean-up) you can clear all system test data from your
datastore instance via::

$ python system_tests/clear_datastore.py

System Test Emulators
=====================

- System tests can also be run against local `emulators`_ that mock
the production services. To run the system tests with the
``datastore`` emulator::

$ tox -e datastore-emulator
$ GOOGLE_CLOUD_DISABLE_GRPC=true tox -e datastore-emulator

This also requires that the ``gcloud`` command line tool is
installed. If you'd like to run them directly (outside of a
``tox`` environment), first start the emulator and
take note of the process ID::

$ gcloud beta emulators datastore start --no-legacy 2>&1 > log.txt &
[1] 33333
$ python datastore/tests/system/utils/clear_datastore.py

then determine the environment variables needed to interact with
the emulator::

$ gcloud beta emulators datastore env-init
export DATASTORE_LOCAL_HOST=localhost:8417
export DATASTORE_HOST=http://localhost:8417
export DATASTORE_DATASET=google-cloud-settings-app-id
export DATASTORE_PROJECT_ID=google-cloud-settings-app-id

using these environment variables run the emulator::

$ DATASTORE_HOST=http://localhost:8471 \
> DATASTORE_DATASET=google-cloud-settings-app-id \
> GOOGLE_CLOUD_NO_PRINT=true \
> python system_tests/run_system_test.py \
> --package=datastore --ignore-requirements

and after completion stop the emulator and any child
processes it spawned::

$ kill -- -33333

.. _emulators: https://cloud.google.com/sdk/gcloud/reference/beta/emulators/

- To run the system tests with the ``pubsub`` emulator::

$ tox -e pubsub-emulator
$ GOOGLE_CLOUD_DISABLE_GRPC=true tox -e pubsub-emulator

If you'd like to run them directly (outside of a ``tox`` environment), first
start the emulator and take note of the process ID::

$ gcloud beta emulators pubsub start 2>&1 > log.txt &
[1] 44444

then determine the environment variables needed to interact with
the emulator::

$ gcloud beta emulators pubsub env-init
export PUBSUB_EMULATOR_HOST=localhost:8897

using these environment variables run the emulator::

$ PUBSUB_EMULATOR_HOST=localhost:8897 \
> python system_tests/run_system_test.py \
> --package=pubsub

and after completion stop the emulator and any child
processes it spawned::

$ kill -- -44444

*************
Test Coverage
*************

- The codebase *must* have 100% test statement coverage after each commit.
You can test coverage via ``tox -e cover``.
You can test coverage via ``nox -s cover``.

******************************************************
Documentation Coverage and Building HTML Documentation
Expand Down Expand Up @@ -386,10 +281,10 @@ using to develop ``google-cloud-python``):
#. Open the ``docs/_build/html/index.html`` file to see the resulting HTML
rendering.

As an alternative to 1. and 2. above, if you have ``tox`` installed, you
As an alternative to 1. and 2. above, if you have ``nox`` installed, you
can build the docs via::

$ tox -e docs
$ nox -s docs

********************************************
Note About ``README`` as it pertains to PyPI
Expand All @@ -404,27 +299,15 @@ may cause problems creating links or rendering the description.

.. _description on PyPI: https://pypi.org/project/google-cloud/

********************************************
Travis Configuration and Build Optimizations
********************************************
**********************
CircleCI Configuration
**********************

All build scripts in the ``.travis.yml`` configuration file which have
Python dependencies are specified in the ``tox.ini`` configuration.
They are executed in the Travis build via ``tox -e ${ENV}`` where
All build scripts in the ``.circleci/config.yml`` configuration file which have
Python dependencies are specified in the ``nox.py`` configuration.
They are executed in the Travis build via ``nox -s ${ENV}`` where
``${ENV}`` is the environment being tested.

If new ``tox`` environments are added to be run in a Travis build, they
should be listed in ``[tox].envlist`` as a default environment.

We speed up builds by using the Travis `caching feature`_.

.. _caching feature: https://docs.travis-ci.com/user/caching/#pip-cache

We intentionally **do not** cache the ``.tox/`` directory. Instead, we
allow the ``tox`` environments to be re-built for every build. This
way, we'll always get the latest versions of our dependencies and any
caching or wheel optimization to be done will be handled automatically
by ``pip``.

*************************
Supported Python Versions
Expand All @@ -435,14 +318,16 @@ We support:
- `Python 2.7`_
- `Python 3.4`_
- `Python 3.5`_
- `Python 3.6`_

.. _Python 2.7: https://docs.python.org/2.7/
.. _Python 3.4: https://docs.python.org/3.4/
.. _Python 3.5: https://docs.python.org/3.5/
.. _Python 3.6: https://docs.python.org/3.6/

Supported versions can be found in our ``tox.ini`` `config`_.
Supported versions can be found in our ``nox.py`` `config`_.

.. _config: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/tox.ini
.. _config: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/nox.py

We explicitly decided not to support `Python 2.5`_ due to `decreased usage`_
and lack of continuous integration `support`_.
Expand Down Expand Up @@ -475,17 +360,23 @@ This library follows `Semantic Versioning`_.

.. _Semantic Versioning: http://semver.org/

It is currently in major version zero (``0.y.z``), which means that anything
may change at any time and the public API should not be considered
Some packages are currently in major version zero (``0.y.z``), which means that
anything may change at any time and the public API should not be considered
stable.

******************************
Contributor License Agreements
******************************

Before we can accept your pull requests you'll need to sign a Contributor License Agreement (CLA):
Before we can accept your pull requests you'll need to sign a Contributor
License Agreement (CLA):

- **If you are an individual writing original source code** and **you own the intellectual property**, then you'll need to sign an `individual CLA <https://developers.google.com/open-source/cla/individual>`__.
- **If you work for a company that wants to allow you to contribute your work**, then you'll need to sign a `corporate CLA <https://developers.google.com/open-source/cla/corporate>`__.
- **If you are an individual writing original source code** and **you own the
intellectual property**, then you'll need to sign an
`individual CLA <https://developers.google.com/open-source/cla/individual>`__.
- **If you work for a company that wants to allow you to contribute your work**,
then you'll need to sign a
`corporate CLA <https://developers.google.com/open-source/cla/corporate>`__.

You can sign these electronically (just scroll to the bottom). After that, we'll be able to accept your pull requests.
You can sign these electronically (just scroll to the bottom). After that,
we'll be able to accept your pull requests.
6 changes: 3 additions & 3 deletions bigquery/google/cloud/bigquery/job.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
import six
from six.moves import http_client

import google.api.core.future.polling
from google.cloud import exceptions
from google.cloud.exceptions import NotFound
from google.cloud._helpers import _datetime_from_microseconds
Expand All @@ -31,7 +32,6 @@
from google.cloud.bigquery._helpers import UDFResourcesProperty
from google.cloud.bigquery._helpers import _EnumProperty
from google.cloud.bigquery._helpers import _TypedProperty
import google.cloud.future.polling

_DONE_STATE = 'DONE'
_STOPPED_REASON = 'stopped'
Expand Down Expand Up @@ -140,7 +140,7 @@ class WriteDisposition(_EnumProperty):
WRITE_EMPTY = 'WRITE_EMPTY'


class _AsyncJob(google.cloud.future.polling.PollingFuture):
class _AsyncJob(google.api.core.future.polling.PollingFuture):
"""Base class for asynchronous jobs.

:type name: str
Expand Down Expand Up @@ -496,7 +496,7 @@ def cancelled(self):

This always returns False. It's not possible to check if a job was
cancelled in the API. This method is here to satisfy the interface
for :class:`google.cloud.future.Future`.
for :class:`google.api.core.future.Future`.

:rtype: bool
:returns: False
Expand Down
Loading