Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(db): reduce batch size for chain history indexer #105

Merged
merged 2 commits into from
Oct 14, 2020

Conversation

iand
Copy link
Contributor

@iand iand commented Oct 14, 2020

Makes batch size configurable. The high batch size was causing enormous insert queries
to be sent to postgres which were always taking > 10s to process, holding locks in the
process.

Makes batch size configurable. The high batch size was causing enormous insert queries
to be sent to postgres which were always taking > 10s to process, holding locks in the
process.
@iand iand self-assigned this Oct 14, 2020
commands/run.go Outdated Show resolved Hide resolved
@iand iand merged commit 60c41e5 into master Oct 14, 2020
@iand iand deleted the iand/batch-large-inserts branch October 14, 2020 15:02
placer14 added a commit that referenced this pull request Oct 14, 2020
* origin/master:
  feat: Add historical indexer metrics (#92)
  fix(lens): Include dependencies needed for Repo Lens (#90)
  Move migration to schema version 10
  fix(migrations): migrations require version 0
  feat: add message gas economy processing
  perf(db): reduce batch size for chain history indexer (#105)
  fix: use hash index type for visor_processing_actors_code_idx (#106)
  feat: set application name in postgres connection (#104)
  feat: add visor processing stats table (#96)
  feat(task): add chain economics processing (#94)
  get actor name for both versions of specs-actors (#101)
placer14 added a commit that referenced this pull request Oct 14, 2020
…-view

* origin/master:
  feat: Add miner_sector_posts tracking of window posts (#74)
  feat: Add historical indexer metrics (#92)
  fix(lens): Include dependencies needed for Repo Lens (#90)
  Move migration to schema version 10
  fix(migrations): migrations require version 0
  feat: add message gas economy processing
  perf(db): reduce batch size for chain history indexer (#105)
  fix: use hash index type for visor_processing_actors_code_idx (#106)
  feat: set application name in postgres connection (#104)
  feat: add visor processing stats table (#96)
  feat(task): add chain economics processing (#94)
  get actor name for both versions of specs-actors (#101)
  fix: remove errgrp from UnindexedBlockData persist
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants