Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding script to enable clearing datastore. #369

Merged
merged 2 commits into from
Nov 12, 2014
Merged

Adding script to enable clearing datastore. #369

merged 2 commits into from
Nov 12, 2014

Conversation

dhermes
Copy link
Contributor

@dhermes dhermes commented Nov 11, 2014

I peeled this off of an ideas branch of been working with:
https://github.com/dhermes/gcloud-python/tree/better-auth-story

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 6a14935 on dhermes:clear-datastore-func into 95e80e1 on GoogleCloudPlatform:master.

tseaver
tseaver reviewed Nov 11, 2014
View reviewed changes
CONTRIBUTING.rst
datastore instance via::

$ python regression//clear_datastore.py

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@coveralls
Copy link

Coverage Status

Changes Unknown when pulling 454f82b on dhermes:clear-datastore-func into * on GoogleCloudPlatform:master*.

@dhermes
Adding script to enable clearing datastore.
25179c9
@dhermes
Refactoring remove_kind to make while loop clearer.
5ad0798 
Also stumbled upon other datastore gotchas (non-ancestor
deletes in a transaction) and added printing to indicate
what is being deleted.
@dhermes
Copy link
Contributor Author

dhermes commented Nov 12, 2014

@tseaver I rebased and 5ad0798 is the while loop fix.

However, while fixing I found the other 90% of the iceberg. I tested by running regression/datastore.py twice and simply turning off TestDatastore.tearDown. The delete actually failed in a transaction due to the Post entities living outside of an entity group and exceeding 5 in number.

ISTM at this point some of these things like

  • clear_datastore.delete_entities
  • clear_datastore.fetch_keys

should go in our codebase. This relates to #336.

WDYT?

/cc @silvolu as an FYI for gcloud-node

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 5ad0798 on dhermes:clear-datastore-func into cf0fca0 on GoogleCloudPlatform:master.

tseaver
tseaver reviewed Nov 12, 2014
View reviewed changes
regression/clear_datastore.py
if query._cursor is not None:
query = query.with_cursor(query.cursor())

return query, query.fetch()

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@tseaver
Copy link
Contributor

tseaver commented Nov 12, 2014

I don't know if "delete all entities returned by a query" is a frequent usecase. I do think having a way for a query to return just keys would be good, but projection queries don't seem to support a "key-only" return type.

@tseaver
Copy link
Contributor

tseaver commented Nov 12, 2014

LGTM. I'd say merge now and consider how to fold useful bits into #336.

dhermes added a commit that referenced this pull request Nov 12, 2014
@dhermes
Merge pull request #369 from dhermes/clear-datastore-func
53db759 
Adding script to enable clearing datastore.
@dhermes dhermes merged commit 53db759 into googleapis:master Nov 12, 2014
@dhermes
Copy link
Contributor Author

dhermes commented Nov 12, 2014

@tseaver What do you mean about projecting onto a key that isn't covered by using the __key__ projection? Do you mean a Key pb returned rather than an Entity pb with only the key set?

@tseaver
Copy link
Contributor

tseaver commented Nov 12, 2014

I misspelled the __key__ projection when I tried it (as __name__). It is the right way to avoid fetching any properties. The protobuf API doesn't support returning keys rather than entities from run_query, but we might extend Query to accomodate that if needed (maybe a fetch_keys() which handled setting the projection and then returned only the key?)

@dhermes
Copy link
Contributor Author

dhermes commented Nov 12, 2014

Added remark to #336 with the idea. Thanks.

@dhermes dhermes deleted the clear-datastore-func branch November 12, 2014 22:42
@dhermes dhermes added testing api: datastore Issues related to the Datastore API. labels Dec 31, 2015
atulep pushed a commit that referenced this pull request Apr 3, 2023
@gcf-owl-bot
chore: [autoapprove] update readme_gen.py to include autoescape True (#…
6d54439 
…369)

Source-Link: googleapis/synthtool@6b4d5a6
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
atulep pushed a commit that referenced this pull request Apr 6, 2023
@gcf-owl-bot
chore: [autoapprove] update readme_gen.py to include autoescape True (#…
dc08b8b 
…369)

Source-Link: googleapis/synthtool@6b4d5a6
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
atulep pushed a commit that referenced this pull request Apr 6, 2023
@gcf-owl-bot
chore: [autoapprove] update readme_gen.py to include autoescape True (#…
2729e3b 
…369)

Source-Link: googleapis/synthtool@6b4d5a6
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
atulep pushed a commit that referenced this pull request Apr 18, 2023
@gcf-owl-bot
chore: [autoapprove] update readme_gen.py to include autoescape True (#…
17dfcb0 
…369)

Source-Link: googleapis/synthtool@6b4d5a6
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this pull request Jun 4, 2023
@gcf-owl-bot
ci(python): fix path to requirements.txt in release script (#369)
bd33f64 
Source-Link: googleapis/synthtool@fdba3ed
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:1f0dbd02745fb7cf255563dab5968345989308544e52b7f460deadd5e78e63b0
parthea added a commit that referenced this pull request Jun 4, 2023
@renovate-bot @parthea
chore(deps): update dependency google-cloud-securitycenter to v1.14.0 (…
fddd690 
…#369)

Co-authored-by: Anthonios Partheniou <[email protected]>
parthea pushed a commit that referenced this pull request Jun 4, 2023
@gcf-owl-bot
chore(python): update dependencies in .kokoro/requirements.txt [autoa…
b001735 
…pprove] (#369)

Source-Link: https://togithub.com/googleapis/synthtool/commit/e3a1277ac35fc88c09db1930533e24292b132ced
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:452901c74a22f9b9a3bd02bce780b8e8805c97270d424684bff809ce5be8c2a2
parthea pushed a commit that referenced this pull request Jul 6, 2023
@gcf-owl-bot
chore: use gapic-generator-python 1.3.1 (#369)
1a1207c 
- [ ] Regenerate this pull request now.

PiperOrigin-RevId: 472772457

Source-Link: googleapis/googleapis@855b74d

Source-Link: googleapis/googleapis-gen@b64b1e7
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYjY0YjFlN2RhM2UxMzhmMTVjYTM2MTU1MmVmMDU0NWU1NDg5MWI0ZiJ9
parthea pushed a commit that referenced this pull request Sep 20, 2023
@gcf-owl-bot
feat: support async write mode for WriteUserEvent API (#369)
d95d2d1 
* feat: support async write mode for WriteUserEvent API
feat: support collect and import GA4 event format with prebuilt whistle rule
feat: support exact searchable and retrievable in catalog attribute config
feat: deprecate retrievable_fields in product attribute
feat: support diversity type in serving config
feat: allow set feed id in merchant center link
feat: support batch remove catalog attribute config
feat: support attribute suggestion in autocomplete
feat: support data output to GCS
feat: expose the local inventory data in product data retrieval
feat: return personal product labels in search response
docs: keep the API doc up-to-date with recent changes

PiperOrigin-RevId: 495450955

Source-Link: googleapis/googleapis@4e4e58c

Source-Link: googleapis/googleapis-gen@c8c05bf
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzhjMDViZmZmZjg4OWE3NzYxNDUyNmU0ZDNiOTBkNDFjNzVjMzFiYyJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this pull request Sep 22, 2023
@gcf-owl-bot
chore: resolve issue with prerelease presubmit [autoapprove] (#369)
55eafcb 
Source-Link: googleapis/synthtool@1b9ad76
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:9db98b055a7f8bd82351238ccaacfd3cda58cdf73012ab58b8da146368330021
parthea pushed a commit that referenced this pull request Sep 22, 2023
@gcf-owl-bot
build(deps): [autoapprove] bump pygments from 2.13.0 to 2.15.0 (#369)
a5efa9a 
Source-Link: https://togithub.com/googleapis/synthtool/commit/eaef28efd179e6eeb9f4e9bf697530d074a6f3b9
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f8ca7655fa8a449cadcabcbce4054f593dcbae7aeeab34aa3fcc8b5cf7a93c9e
parthea added a commit that referenced this pull request Sep 22, 2023
@gcf-owl-bot @parthea
feat: add SoleTenantConfig API (#369)
434c954 
* docs: Operation.Type is now documented in detail
docs: Operation.self_link and Operation.target_link given examples
feat: Cluster resizes will now have their own operation type (RESIZE_CLUSTER) instead of reusing REPAIR_CLUSTER; they will start using this in the near future

PiperOrigin-RevId: 526970355

Source-Link: googleapis/googleapis@83fd1f9

Source-Link: googleapis/googleapis-gen@5a191c3
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNWExOTFjM2FhMmM1MjZhNTllZWE1OTdjN2RkZGE3OTQ4MWI4ZTg1OCJ9

docs: Operation.Type is now documented in detail
docs: Operation.self_link and Operation.target_link given examples
feat: Cluster resizes will now have their own operation type (RESIZE_CLUSTER) instead of reusing REPAIR_CLUSTER; they will start using this in the near future

PiperOrigin-RevId: 526025502

Source-Link: googleapis/googleapis@4273180

Source-Link: googleapis/googleapis-gen@772e2cd
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNzcyZTJjZDQ4ZjU2MWUxZjBhNGQxNzJlOWY5Mjg1YWQ5ZTFjMTIxMCJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: support fleet registration via cluster update

PiperOrigin-RevId: 529131876

Source-Link: googleapis/googleapis@d175293

Source-Link: googleapis/googleapis-gen@b680d48
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYjY4MGQ0OGEwZjY3NjJkOTRlMGUxNGViZmZiNzg2OGU2YzU5MzM1YiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: add a new feature for fleet registration via cluster update in v1alpha1, v1beta1, v1

PiperOrigin-RevId: 529516002

Source-Link: googleapis/googleapis@5ce0665

Source-Link: googleapis/googleapis-gen@5122c36
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNTEyMmMzNjMwMTQ4MzJkZjIxMTNmNDc0ZjU2MTM3YjRiYzBiM2NhMyJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* docs: clarified release channel defaulting behavior for create cluster requests when release channel is unspecified

If release channel is left unspecified on cluster creation and a version is specified, the cluster is now enrolled in the most mature release channel where the version is available (first checking STABLE, then REGULAR, and finally RAPID) instead of being enrolled in STATIC. This only affects clusters with node auto upgrades enabled and no explicit node version specified. To ensure new clusters are enrolled in static, explicitly specify that the cluster should not be enrolled in a channel.

PiperOrigin-RevId: 532773421

Source-Link: googleapis/googleapis@633d3c1

Source-Link: googleapis/googleapis-gen@f779f56
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZjc3OWY1NjI0NzYxOTlkMWYwMjE1NmFjMjhmOTdlYzllMzcwNjdlYiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* docs: clarified release channel defaulting behavior for create cluster requests when release channel is unspecified

If release channel is left unspecified on cluster creation and a version is specified, the cluster is now enrolled in the most mature release channel where the version is available (first checking STABLE, then REGULAR, and finally RAPID) instead of being enrolled in STATIC. This only affects clusters with node auto upgrades enabled and no explicit node version specified. To ensure new clusters are enrolled in static, explicitly specify that the cluster should not be enrolled in a channel.

PiperOrigin-RevId: 532773462

Source-Link: googleapis/googleapis@aeae5ea

Source-Link: googleapis/googleapis-gen@d4d433d
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZDRkNDMzZDRlYWE4YjRkNzgwZjgzMDYyYmYxYTgxZDA0NjQ2ZWE0MSJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: add SoleTenantConfig API

PiperOrigin-RevId: 536516473

Source-Link: googleapis/googleapis@0fb1181

Source-Link: googleapis/googleapis-gen@3d69310
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiM2Q2OTMxMDk5NDI0MmQ1YmRiOWQzNDVmNGQ4OTRkNDViNzI2MmY5ZCJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: add SoleTenantConfig API

PiperOrigin-RevId: 536569254

Source-Link: googleapis/googleapis@3cc45f0

Source-Link: googleapis/googleapis-gen@ed8d754
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZWQ4ZDc1NDZjZDE3NzA1M2Q1MzEyZDBlNDBmN2Y1MDYzNTJiYTgwNSJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <[email protected]>
parthea pushed a commit that referenced this pull request Sep 22, 2023
@renovate-bot
chore(deps): update all dependencies (#369)
b9ee585
parthea pushed a commit that referenced this pull request Sep 22, 2023
@renovate-bot
chore(deps): update dependency google-cloud-pubsub to v2.12.0 (#369)
d4f2af7
parthea pushed a commit that referenced this pull request Sep 22, 2023
@gcf-owl-bot
docs: Minor formatting (#369)
f083864 
* docs: Minor formatting
chore: Update gapic-generator-python to v1.11.5
build: Update rules_python to 0.24.0

PiperOrigin-RevId: 563436317

Source-Link: googleapis/googleapis@42fd37b

Source-Link: googleapis/googleapis-gen@280264c
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMjgwMjY0Y2EwMmZiOTMxNmI0MjM3YTk2ZDBhZjFhMjM0M2E4MWE1NiJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this pull request Sep 22, 2023
@gcf-owl-bot
chore: detect samples tests in nested directories (#369)
89bb6aa 
Source-Link: googleapis/synthtool@50db768
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:e09366bdf0fd9c8976592988390b24d53583dd9f002d476934da43725adbb978

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
parthea added a commit that referenced this pull request Oct 21, 2023
@gcf-owl-bot @parthea
chore: use gapic-generator-python 0.63.4 (#369)
ff43a4a 
* chore: use gapic-generator-python 0.63.4

chore: fix snippet region tag format
chore: fix docstring code block formatting
PiperOrigin-RevId: 430730865

Source-Link: googleapis/googleapis@ea58002

Source-Link: googleapis/googleapis-gen@ca893ff
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiY2E4OTNmZjhhZjI1ZmM3ZmUwMDFkZTE0MDVhNTE3ZDgwNDQ2ZWNjYSJ9

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* remove redundant samples

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <[email protected]>
parthea pushed a commit that referenced this pull request Oct 21, 2023
@gcf-owl-bot
chore(python): drop flake8-import-order in samples noxfile (#369)
2108b5c 
Source-Link: googleapis/synthtool@6ed3a83
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:3abfa0f1886adaf0b83f07cb117b24a639ea1cb9cffe56d43280b977033563eb

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this pull request Oct 21, 2023
@release-please
chore(main): release 2.14.0 (#369)
be14aa0 
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
parthea pushed a commit that referenced this pull request Oct 21, 2023
@gcf-owl-bot
chore: [autoapprove] Pin flake8 version (#369)
66604a9 
Source-Link: https://togithub.com/googleapis/synthtool/commit/0ddbff8012e47cde4462fe3f9feab01fbc4cdfd6
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:bced5ca77c4dda0fd2f5d845d4035fc3c5d3d6b81f245246a36aee114970082b
parthea pushed a commit that referenced this pull request Oct 21, 2023
@renovate-bot
chore(deps): update dependency google-analytics-data to v0.17.0 (#369)
5155123
parthea pushed a commit that referenced this pull request Oct 21, 2023
@rosiezou @dandhlee
chore: updated README with links to Vertex AI migration guide (#369)
476ddb4 
Co-authored-by: Dan Lee <[email protected]>

Co-authored-by: Dan Lee <[email protected]>
parthea pushed a commit that referenced this pull request Oct 21, 2023
@renovate-bot
chore(deps): update dependency google-cloud-build to v3.11.0 (#369)
800dba2
parthea pushed a commit that referenced this pull request Oct 22, 2023
@gcf-owl-bot
chore: [autoapprove] update readme_gen.py to include autoescape True (#…
9e2ee72 
…369)

Source-Link: googleapis/synthtool@6b4d5a6
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:f792ee1320e03eda2d13a5281a2989f7ed8a9e50b73ef6da97fac7e1e850b149

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
api: datastore Issues related to the Datastore API. testing
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@dhermes @coveralls @tseaver