After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 774ee0f.

Benchmarks with performance regressions

There were 8 possible performance regressions, according to the lookback z-score method.

Benchmarks with regressions:

• Commit Run on test-mac-arm at 2024-05-29 18:14:19Z

  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-15, scale_factor=1
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-18, scale_factor=1

• Commit Run on arm64-t4g-linux-compute at 2024-05-29 18:14:38Z

  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-18, scale_factor=1
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-15, scale_factor=1

• Commit Run on arm64-m6g-linux-compute at 2024-05-29 18:14:07Z

  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-18, scale_factor=1
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-18, scale_factor=10
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-15, scale_factor=1
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-15, scale_factor=10

All benchmark runs analyzed:

• Commit Run on ursa-i9-9960x at 2024-06-06 11:15:26Z
• Commit Run on test-mac-arm at 2024-05-29 18:14:19Z
• Commit Run on ursa-thinkcentre-m75q at 2024-05-29 16:42:25Z
• Commit Run on arm64-t4g-linux-compute at 2024-05-29 18:14:38Z
• Commit Run on ec2-m5-4xlarge-us-east-2 at 2024-05-29 18:13:54Z
• Commit Run on arm64-m6g-linux-compute at 2024-05-29 18:14:07Z
• Commit Run on ec2-t3-xlarge-us-east-2 at 2024-05-30 00:24:41Z

Unstable benchmarks with performance regressions

The following benchmark results indicate a possible performance regression, but are known to sometimes produce false positives when applying the lookback z-score analysis.

• Commit Run on test-mac-arm at 2024-05-29 18:14:19Z

  • ArrayScalarKernel (C++) with params=<Subtract, Int16Type>/size:4194304/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmark
  • BitBlockCounterSumWithOffset (C++) with params=4096, source=cpp-micro, suite=arrow-bit-block-counter-benchmark

• Commit Run on ursa-thinkcentre-m75q at 2024-05-29 16:42:25Z

  • Constants (C++) with source=cpp-micro, suite=arrow-decimal-benchmark
  • MoveEmptyVector (C++) with params=<STD_VECTOR(std::shared_ptr)>, source=cpp-micro, suite=arrow-small-vector-benchmark
  • ReferenceCompression (C++) with params=Compression::GZIP, source=cpp-micro, suite=arrow-compression-benchmark
  • BM_DeltaBitPackingEncode_Int32_Wide (C++) with params=32768, source=cpp-micro, suite=parquet-encoding-benchmark
  • MoveEmptyVector (C++) with params=<STATIC_VECTOR(std::shared_ptr)>, source=cpp-micro, suite=arrow-small-vector-benchmark
  • BM_PlainEncodingSpacedBoolean (C++) with params=32768/5000, source=cpp-micro, suite=parquet-encoding-benchmark

• Commit Run on arm64-t4g-linux-compute at 2024-05-29 18:14:38Z

  • TouchArea (C++) with params=size:16777216/real_time/threads:1, source=cpp-micro, suite=arrow-memory-pool-benchmark