bucket.list_blobs(max_results=n) has no effect

[jlowin]

passing max_results=5 to bucket.list_blobs() appears to have no effect; all blobs are returned in the BlobIterator

[dhermes]

Trying to reproduce right now.

[dhermes]

Confirmed:

>>> list(b.list_blobs(max_results=5))
[<Blob: new1455562849552, a>,
 <Blob: new1455562849552, b>,
 <Blob: new1455562849552, c>,
 <Blob: new1455562849552, d>,
 <Blob: new1455562849552, e>,
 <Blob: new1455562849552, f>]

I'm hoping it's just a query string bug. Digging deeper to ascertain.

[jlowin]

@dhermes thanks!

[dhermes]

Luckily, it's our fault (not the storage backend API). We iterate past the maxResults even when we don't mean to.

A short term fix would be for you to use the iterator manually:

response = blob_iter.get_next_page_response()
results = list(blob_iter.get_items_from_response(response))

Thanks a lot for bringing it to our attention!

[dhermes]

#1787 should be a reference point for a fix here

[dhermes]

Another example of unexpected failure:

bucket = Bucket(client, 'my-bucket-name')
blobs = bucket.list_blob(
    max_results=10, page_token=None, fields="items/name,nextPageToken")

for b in blobs:
   print(b.name)

This prints many more than 10 blobs when the iterator is exhausted.

[tseaver]

FTR, #2490 is the first pass at fixing this issue: the solution there may eventually be obsoleted by the general approach sketched recently in #895.