Abstracted batch methods

[interstateone]

I just want to run through this before I submit a PR.

• get_object_count(class_name)
  abstracts a count query
• get_all_objects(class_name)
  hits a max of 11 000
• delete_all_objects(class_name or array of Parse::Objects)
  hits a max of 11 000 with class_name, but can run more than once to get around that

I've got them in util.rb right now, but that doesn't seem quite perfect. They're batch/query operations but wouldn't be instance methods of those objects. Thoughts?

Any other methods that come to mind?

[adelevie]

I'll take another gander at your code to see what else can be brought over here. I'm just trying to think of a good place within parse-ruby-client.

Maybe within Parse::Util?

[ericcj]

we have similar helpers in a util class we haven't contributed back. the query ones actually would be kind of nice on Query itself i think, particularly as lazy enumerators. here they are for reference:

  def each_value_in_page(query, field, values)
    values.uniq.each_slice(MAX_VALUE_IN) do |slice|
      query.value_in(field, slice)
      each_page(query) do |results|
        yield results
      end
    end
  end

  DEFAULT_PAGE_OPTIONS = {:page_size => MAX_QUERY_LIMIT, :use_skip => true}.freeze

  def fetch_all_pages(query, options = {})
    options = DEFAULT_PAGE_OPTIONS.dup.merge(options)
    results = []
    each_page(query, options) do |page|
      results += page
    end
    results
  end

  def each_result(query, options = {})
    each_page(query, options) do |page|
      page.each do |result|
        yield result
      end
    end
  end

  def each_page(query, options = {})
    options = DEFAULT_PAGE_OPTIONS.dup.merge(options)
    query.limit = options[:page_size]

    if options[:use_order]
      query.order_by = options[:use_order]
    elsif options[:use_skip]
      query.skip = 0
    end

    begin
      results = if options[:retry_forever]
        retry_forever{query.get}
      else
        query.get
      end
      num_results = results.size

      if options[:use_order]
        if results.present?
          if options[:use_order] =~ /^-(.+)/
            query.less_than($1, results.last[$1])
          else
            query.greater_than(options[:use_order], results.last[options[:use_order]])
          end
        end
      elsif options[:use_skip]
        query.skip += num_results
      end

      yield results
    end while num_results >= options[:page_size]
  end

  def retry_forever(&block)
    yield
  rescue StandardError => e
    raise unless block
    log_exception(e, :ignore_notification => true)
    sleep 1
    retry
  end

  def update_each(query)
    each_page(query) do |results|
      batch_updates(results) do |batch, object|
        yield object
        batch.update_object(object)
      end
    end
  end

  def batch_updates(objects)
    objects.each_slice(MAX_BATCH_SIZE) do |slice|
      batch = Parse::Batch.new
      slice.each do |object|
        yield batch, object
      end
      batch.run! if batch.requests.present?
    end
  end

[adelevie]

So something like:

query = Parse::Query.new("Widgets").eq("foo", "bar")
query.each_value_in_page do |page|
  puts page
end

I probably misread exactly how that method works, but the main point is that these methods that take a Parse::Query as an argument could easily be converted to class methods of Parse::Query?

[ericcj]

yes

On Apr 10, 2013, at 1:18 PM, Alan deLevie [email protected] wrote:

So something like:

query = Parse::Query.new("Widgets").eq("foo", "bar")
query.each_value_in_page do |page|
puts page
end
I probably misread exactly how that method works, but the main point is that these methods that take a Parse::Query as an argument could easily be converted to class methods of Parse::Query?

—
Reply to this email directly or view it on GitHub.