- #1875: Switch to JDK 21
- #1884: Add Jinja filters for creating dynamic date ranges based on now()
- #1871: [ES-8436] feat: Change Dockerfile base images to Wolfi
- #1865: Add an option to filter by challange in list command
- #1859: Modify rally to allow multiple cars with complex configuration
- #1858: Bump urllib3 from 1.26.18 to 1.26.19
- #1882: fixes #1881 - composite operation failing when track_total_hits is false
- #1879: Pull non-default branch commits selectively
- #1877: Docs: fix level of (warmup-)interations
- #1876: Cast batch size as int
- #1863: Avoid logging ES credentials from running Rally processes
- #1861: Restore time measurement on request end
- #1848: Pin
requestslibrary until we fixdocker-py
- #1839: Add support for Python 3.12
- #1851: Assume no operator if security not enabled in serverless
- #1846: Update create track
- #1845: Allow rally to run ESQL operations on serverless for benchmarks
- #1831: Add client ID to metrics
- #1830: Add looped mode to bulk operation
- #1827: Track dataset size
- #1819: Override serverless cluster version in telemetry
- #1818: Override serverless cluster version
- #1816: Add FS metrics to node-stats telemetry device
- #1835: Avoid immense term error in metrics metadata fields
- #1825: Make Jit telelmetry compatible with JDK 9+
- #1823: Measure time until last response chunk
- #1805: Handle raw response exceptions
- #1797: Document Rally use with Elastic Serverless
- #1791: Add ESQL operator
- #1789: Add serverless-aware runners
- #1787 (Breaking): Inject serverless_operator in track templates
- #1786: Add support for zstd-compression
- #1785: Fix serverless public use in trivial tracks
- #1790: Retrieve cluster info in serverless public mode
- #1784: Fix crash on too many consecutive errors
- #1793: Bump urllib3 from 1.26.9 to 1.26.18
- #1778: Support API key authentication
- #1770: Exclude telemetry devices based on serverless status
- #1768: Detect serverless operator status automatically
- #1772: Fix "Could not checkout. Do you have uncommitted changes?" error
- #1760: Exclude tasks based on serverless status
- #1750 (Breaking): Inject build_flavor in track templates
- #1756: Retrieve serverless build hash from nodes info API
- #1755: Add blob-store-stats telemetry device and tests
- #1743: Bulk action refresh param
- #1763: Fix running tracks across multiple load driver machines
- #1747: Fix rally-annotations index creation
- #1748: Document actor system
- #1759: Ensure serverless.mode/operator are always set
- #1727: Allow configuring logging to use local time
- #1707: Evenly distribute client connections across hosts
- #1706: Allow specifying branch together with timestamp in revision
- #1702: Include 'refresh' and 'flush' stats by default when using 'node-stats-include-indices'
- #1663: Add operating system cgroup stats to node-stats telemetry
- #1722: Increase 'mapping.total_fields.limit' to '2000'
- #1719: Bump up minimum Python version
- #1710: Actually use client returned from '.options()'
- #1705: Refactor some runners to set transport level options
- #1703: Improve exception handling for metrics store errors
- #1698: Update progress after GCS download finished
- #1692: Allow to delete indices related to index templates
- #1691: Ignore HTTP 404 consistently in runners
- #1690: Use correct params for create component template
- #1687: Propagate request start/end lazily
- #1725: Make async context manager synchronous
- #1674: Allow reading metrics password from environment
- #1624: Skip building plugins moved to modules
- #1654: Add role metadata to node-stats telemetry device
- #1649: Allow specifying --installation-id in esrally install
- #1643: Skip response parsing for raw requests
- #1630: Add delay and duration parameters to jfr telmetry device
- #1619: Add shard data to search operation's detailed-results meta-data
- #1618: Support meta-data for composite tasks
- #1610: Support specifying user tags as json
- #1645: Fix --to-date option for list subcommand
- #1640: Pass plugin-params also for modules
- #1638: Don't use --short when retrieving revisions from git
- #1666: Document bulk behavior with multiple clients/documents
- #1648: Fix broken links on the command line reference docs
- #1641: Mention jq is required for development
- #1613: Document Track Processor Interface
- #1605: Update list of available tracks
- #1659: Simplify create_readers
- #1615: Print errors when --quiet is used
- #1611: Use newer ssl parameter in aiohttp client API
- #1607: Cut down on the amount of INFO-level logging
- #1574: Include a new downsampling operation
- #1531: Support data streams in create-track
- #1517: Remove the check for internet connection
- #1599: add
--user-tagsas alias for--user-tag - #1597: Add additional filter options to rally list races
- #1591: Support checking out remote branches
- #1588: Fail benchmark if assertion could not run
- #1576: Add build subcommand
- #1573: Add knn vectors to disk-usage-stats
- #1551: Switch from setuptools to hatchling
- #1542: Add new wait-for-current-snapshots-create operation
- #1541: Update opts.py to allow for 'None' timeout-value in client_options
- #1586: Fix supplier tests for aarch64
- #1580: Remove buggy indices.exists() calls from code
- #1578: Correctly retry HTTP downloads after incomplete reads
- #1563: Fix multi-cluster attribute check for wrapped runners
- #1540: Don't load libs path if not using it
- #1600: Add docs for create-track from data streams
- #1572: Mention that create/restore-snapshot is an administrative operation
- #1568: Encourage users to read tracks documentation
- #1557: Remove duplicate property descriptions from corpora docs
- #1554: Link to the 7.14.0 ES Python client docs.
- #1552: Fix broken links on README.md
- #1579: Use narrower exception to avoid pylint ignore
- #1571: Stop trying to test Elasticsearch 6.8.0 on ARM
- #1565: Fix
make linton Python 3.10 - #1564: Introduce rally-tracks compatibility testing
- #1555: Add an integration test for static responses.
- #1553: Run unit and integration tests in separate tox environments
- #1548: Move most packaging metadata to pyproject.toml
- #1546: Run lint with pre-commit
- #1539: Correctly log how long it took to close transports
- #1498: Bump upper tested version to 8.4.0
- #1497: Remove Elasticsearch 6.3 logic
- #1532: Use main branch of Elasticsearch for source builds
- #1520: Create and use a unique ES API key for each simulated client
- #1535: Use new logo on GitHub, PyPI and Read the Docs
- #1530: Add OS mem stats to node-stats telemetry device
- #1526: Add runner for paging through composite aggregations
- #1522: Fix pyenv install on Apple Silicons Macs
- #1516: Create an ES client per simulated client instead of per worker.
- #1503: Explicit offline mode
- #1502: Display ZGC collector stats in report
- #1499: Flexible support of GC collectors in telemetry.
- #1492: Support wrapped templates produced by elastic-package dump
- #1478: Allow configuring Elasticsearch cluster name
- #1475: Add raw-request op type for latency charts
- #1473: Show both median and mean ML processing time in charts
- #1521: Retry incomplete HTTP downloads
- #1518: Relax target-hosts check for multi clusters
- #1500: When running rally with no options, print help and exit.
- #1489: Support composable template without top-level
template - #1486: Remove template key requirement from composable template param source
- #1482: Allow compare to continue for missing disk usage stats
- #1472: Ensure telemetry path exists for heapdump
- #1529: Make git.clone remote kwarg-only
- #1527: Make esrally.utils.git branch agnostic
- #1512: Prepare switch to Elasticsearch Python client 8.2.0
- #1435: Update Elasticsearch support policy after 8.0 release
- #1469: Set enable_cleanup_closed by default
- #1455: Pick up data streams in disk-usage-stats telemetry
- #1440: Fix build by downgrading markupsafe
- #1447: Stop leading users away from finding errors in docs
- #1433: minor change to docs
- #1422: Rename 'indices stats API' to 'index stats API'
- #1403: Document exporting certificates from PKCS#12 keystores
- #1355: Support Python 3.10
- #1362: Add timeout parameter to bulk operation
- #1418: Derive correct artifact name for ARM architecture
- #1368: Call the correct functions for
scroll-searchandpaginated-searchoperations
- #1358: Document challenge's user-info property
- #1357: Simplify python-caches-clean Makefile command
- #1356: Stop mentioning removed "configure" command
- #1387: Switch to native pytest in tests/utils
- #1343: Re-introduce fix for listing nested tracks
- #1336: Check benchmark stop at least every second in SamplerThread
- #1333: Add percentage diff column to compare subcommand
- #1330: Add master node telemetry device
- #1319: Add support for listing nested tracks
- #1313: Use bridge network in container mode
- #1312: Update rally-* index template settings to default to Elasticsearch defaults
- #1311: Show race id in console
- #1310: Configure persistent datastore index settings
- #1305: Allow parallel tasks to exit on any completion
- #1303: Add create and delete ILM policy runners
- #1296: Add data streams telemetry device
- #1292: Bump Python 3.8 version for dev and CI to 3.8.10
- #1291 (Breaking): Drop relative-time-ms metric
- #1290: update elasticsearch client to 7.13.2
- #1288: add transform-stats operation type
- #1285: Ensure exit code 130 is returned from SIGINT
- #1318: Fix num of docs in -1k generated corpus
- #1294: Fix composable templates runner
- #1293: fix download from s3/gs by moving http query parameter downstream
- #1337: Clarify definition of processing time
- #1332: Improve docs in various ways
- #1327: Allow more copy/pasting in cluster management docs
- #1326: Improve documentation in various ways
- #1325: Document how to use stat on GNU/Linux too
- #1299: Add Makefile target to run all CI checks
- #1304: Run integration tests with the default distro
- #1302: Upgrade Python client to 7.14.0
- #1298: Use explicit config files for black and isort
- #1281: Format code with black and isort
- #1276: Switch to pytest asserts
- #1273: Re-introduce relative-time property
- #1268: Fix error for shard-stats
- #1266: Support gradual ramp-up
- #1258: Record shard allocation
- #1254: assert developers have block storage dependencies installed
- #1252: Make S3 support optional
- #1243: Ensure destructive actions work regardless of wildcard input
- #1287: Do not treat increase in transform processing/indexing/search time as improvement
- #1284: Ensure max_connections value is logged correctly
- #1270: Include requests when installing google-resumable-media
- #1268: Fix error for shard-stats
- #1263: Add pyenv shims to PATH
- #1274: Add example for raw-request
- #1267: Consider new API calls in static response example
- #1265: Consolidate and improve track/team repo revision logic docs
- #1283: Set daemon attribute directly instead of using deprecated setter
- #1245: Strictly check for supported version
- #1238 (Breaking): Disallow to specify cluster settings in the track
- #1236: add support for numbers alignment in table output (#1202)
- #1235: Allow to selectively ignore response errors
- #1234: Allow to specify static responses
- #1231 (Breaking): Drop relative-time metric
- #1220: Store duration time of task in rally-results metrics record
- #1250: Add note about ES version support
- #1247: Update telemetry device docs
- #1233: Document proper usage of the Docker image
- #1230: Update community resources videos
- #1190: Add support for search_after and point-in-time
- #1223: Fix Issue #1222: esrally CLI should always return 130 when cancelled
- #1221: Store relative-time in milliseconds
- #1216: Upgrade template engine
- #1214: Collect metrics on composite subtasks
- #1213: Support chart_name filters in chart annotations
- #1211: Add support for merge_count and merge_time graphs
- #1207: Expose an API to handle file offset tables
- #1203: Log results of cluster health check
- #1200: Add optional name in assertion message
- #1199: Throttle requests from the beginning
- #1193: Add IT tests to test installation according to docs
- #1178: Always rely on source artifact caching
- #1175 (Breaking): Always require a subcommand
- #1172: Improve isolation in race control tests
- #1171: Add request-id to search operator
- #1165: Fallback to closest minor branch for track/team repositories
- #1093: Modify chart generator to support Kibana 7.x dashboards
- #1217: Fix auto build of docs
- #1209: Ensure stable mappings for metrics
- #1192: Fix compare subcommand
- #1177: Use operation name consistently in metrics samples
- #1185: Fix installation docs after #1151
- #1163: Make some document properties extensible
- #1162: Log trace of uncaught exception in load generator
- #1159: Rally improve artifact download headers
- #1157: Add support for assertions
- #1155: Deprecate invoking Rally without a subcommand
- #1154: Allow to filter tasks by tag
- #1153: Only consider race id when loading a race
- #1146: Show mean throughput in command line report
- #1143: Upgrade psutil dependency
- #1160: Include empty mean value in summary stats
- #1147: Don't require a name to register a track processor
- #1169: Document meta-data returned by operations
- #1167: Clarify num of docs in corpora when action and metadata is used
- #1148: Make docs build stricter
- #1140: Improve responsiveness of track preparation
- #1136: Support non default repositories in chart generator
- #1135: Allow to customize track preparation
- #1129: Support async search API
- #1118: Allow index pattern for source-index in shrink-index operation
- #1115: Error wrapper script on unclean local state
- #1109: Throttle tasks based on presence of a scheduler
- #1108: Add support for FOSSA
- #1099: Composable + Component template support
- #1096: Add CI for Python 3.9.0
- #1091: Check registered parameter sources more strictly
- #1070: Add support for custom headers and request-timeout
- #1065: Allow to specify corpus meta-data
- #1132: Allow for filtering of administrative tasks
- #1126: Respect specified order in composite task
- #1125: Wait until cluster health is green
- #1124: Treat some zero target-intervals as unthrottled
- #1122: Remove implicit request timeout from force merge operation in polling
- #1121: Ensure that sleep tasks are timed
- #1119: Support trailing slashes for all URL schemes
- #1117: Don't change scheduler on failed requests
- #1116: Ensure admin tasks are always executed
- #1111: Allow trailing / in base-url
- #1103: Accept some differing units for throttling
- #1145: Fix punctuation
- #1137: Add a configuration file reference
- #1133: Clarify latency-vs-service-time FAQ
- #1102: Clarify task execution order in docs with respect to --include-tasks
- #1097: Upgrade google-auth to latest
- #1094: Add support for Google Cloud Storage buckets
- #1092: Add Support for Datastreams
- #1064: Fail fast if leading
/is missing frompathin raw-request - #1061: Pass JAVA15_HOME in build
- #1055: Check if total_transform_processing_times exist for compare reports
- #1051: Force Merge Runner Improvements - Polling
- #1047: Add runner to wait until snapshot has been created
- #1045: Show processing time also in comparison reports
- #1035: Warn on request errors
- #1031: Upgrade Elasticsearch client library to 7.8
- #1088: Use the new pip dependency resolver
- #1075: Ensure that request error meta-data is string
- #1071: Fix parallel completed-by task identification
- #1055: Check if total_transform_processing_times exist for compare reports
- #1062: Document how to benchmark an Elastic Cloud cluster
- #1049: Docs - hints for handling errors and identifying queries and responses
- #1041: Add indexing_pressure to nodes stats
- #1039: Check for pointless statements
- #1038: Check for trailing comma tuple
- #1029: Add integration test for proxy
- #1028: Fetch artifact revision lazily on build
- #1026: [WIP] Migrate first test_docker_dev_image
- #1025: Upgrade to Thespian 3.10.1
- #1024: Allow runners to determine throughput
- #1023: Migrate configure test to new IT infrastructure
- #1022: add transform support
- #1019: Add timestamps as meta-info in index recovery
- #1016: Allow to forcefully close SSL connections
- #1015: Scale connection pool automatically
- #1013: Allow to measure create/restore snapshot speed
- #1012: Make exists_set_param macro available to tracks
- #1011: add support for transform API's
- #1001: Allow to downsample request metrics
- #998: Cache source artifacts by default
- #997: Allow to abort on fatal errors
- #994: Report number of GCs
- #992: Cache source artifacts
- #1044: Improve support for benchmarks with many clients
- #1027: Resolve JAVA_HOME for source builds lazily
- #1018: Check for recovery details when finished
- #1008: Allow to specify a runtime-jdk during installation
- #1007: Wait until indices are green in generated track
- #996: Ensure artifact cache directory exists
- #987: Validate if car allows for using the bundled JDK
- #1043: Installation and offline dist improvements
- #1037: Correct Rally forum URL
- #1006: Add info about querying challenges in the docs
- #1003: Add network troubleshooting tips
- #993: Update the release process
- #984: Add developer info about intellij
- #970: Add tooling to create a track from existing data
- #944: Allow to use significantly more clients
- #935: Add an asyncio-based load generator
- #875: Add 'Tracker' track-generation tool
- #988: Add experimental segment stats telemetry device
- #986: Allow to override GC log parameters
- #977: Migrate track param check to new IT infrastructure
- #972: Pass actor system env vars in integration test
- #971: Add thespian logs to ci
- #969: Add an it method for rally race
- #968: Add integration tests with eventdata
- #967: Raise an error when attempting to bulk-index without any corpora
- #966: Migrate more integration tests
- #963: Reduce mapped memory with many bulk clients
- #962: Increase bulk size to metrics store
- #959: Clarify the impact on refreshes for node-stats telemetry
- #958: Avoid duplicate URL parsing for async connections
- #954: Add flag to handle running processes automatically
- #953: Pass JAVA14_HOME in build
- #952: Allow to disable HTTP compression for all queries
- #951: Upgrade Thespian to 3.10.0
- #950: Add support for a RALLY_HOME env var everywhere
- #949: Upgrade Elasticsearch client to 7.6.0
- #947: Use pbzip2/pigz to decompress corpora if available
- #945: Add an override for the rally homedir
- #941: Improve response processing
- #938: Store units in results
- #934 (Breaking): Bump minimum Python version to 3.8
- #932: Improve layout of generated dashboards
- #931: Store Rally revision in metrics metadata
- #930: Update psutil to latest version
- #929: Allow index-stats to compare non-existing path
- #928: Don't invoke setup.py directly
- #925: Allow to specify an indices stats condition
- #923: Assume a TTY for interactive commands
- #919: Run integration tests with pytest
- #914: Improve release docs and IT prechecks
- #913: Normalize handling of --preserve-install
- #983: Assign worker-id globally
- #978: Remove os environment variables from env prep
- #965: Always shutdown worker thread pool
- #961: Determine most recent sample per client
- #960: Disable docstring linter check
- #956: Fix fallback implementation of ML-related runners
- #943: Fix request timeout handling
- #942: Honor response-compression-enabled parameter
- #927: Add unit for all query charts
- #918: Gather system metrics on Docker stop if possible
- #980: Change next Rally version in migration docs
- #955: Update docs to advise use of pbzip2 for new track corpus
- #948: Align track reference structure with track
- #939: Correct JMC download link
- #937: Fix port number in recipes docs
- #890: Speed up client-side bulk-handling
- #912: Honor only specified Python versions in prereq
- #909: Add custom probing URL
- #908: Validate docker-compose exists in it tests
- #907: Specify useful variables for Rally CI
- #906: Specify useful variables for Rally CI
- #904: Eliminate deprecation warnings in Python 3.7+
- #903: Show more user-friendly track loader errors
- #899 (Breaking): Use zeroes instead of whitespaces as padding bytes
- #895: Log race id on startup
- #887: Allow to override Python binary for make
- #886: Fix race condition with old ES processes in IT tests
- #884: Detach ES using Python standard mechanism
- #880: Show recent log output on integration test failure
- #879: Don't read from stdout/stderr on ES startup
- #873: Prevent installation with unsupported Python
- #872: Extend error msg to specify operation name for missing index
- #865: Updating force merge operation to force merge only track indices by default
- #901: Fix race in wait_for_pidfile.
- #894: Only set distribution-version if known
- #893: Only clear metrics store on benchmark end
- #891: Align exception handling in disk IO telemetry
- #885: Fix issue with fetching test-mode -1k document corpora.
- #883: Stop all locally running nodes
- #882: Make REST API check stricter
- #881: Wait for REST layer before version check
- #877: Don't write metrics to a closed metrics store
- #874: Store system metrics if race metadata are present
- #871: Skip setup phase in IT tests during docker release
- #868: Fix W0631
- #860: More resilient node shutdown
- #902: Add example how to determine actor system status
- #853: Allow to use the bundled JDK in Elasticsearch
- #830: Manage Elasticsearch nodes with dedicated subcommands
- #815: Add Python 3.8 in Rally tests
- #863: Allow piped stdin in run_subprocess_with_logging
- #862: Add support for excluded tasks in chart_generator
- #850: Allow to show track details
- #844: Add task exclude filter
- #840: Add make target to serve docs locally
- #836: Upgrade boto3
- #832: Only keep the most recent build log
- #829: Reduce usage of platform-specific code
- #818 (Breaking): Store build.log in log directory
- #816: Harmonize local pyenv versions with CI ones
- #805: Add lint + precommit make targets.
- #803: Calculate system metrics per node
- #802: Whitelist py.test in tox tests
- #798: Allow definition of body in restore-snapshot operation
- #793: Add ability to restore from a snapshot
- #789: Let the runner determine progress
- #788: Manage dev dependencies in setup.py
- #785: Don't attach telemetry devices for Docker
- #783: Upgrade pytest to 5.2.0
- #779 (Breaking): Gather cluster-level metrics in driver
- #778 (Breaking): Expose race-id as command line parameter
- #776: Add heapdump telemetry device
- #774: Raise an error if race is not found by id
- #773: Store race files always as race.json
- #771: Store track-related meta-data in results
- #767: Remove merge times from command line report
- #766: Ensure tox environment is clean
- #765: Start release process creating NOTICE
- #727: Allow function-level invocation of integration-test.sh.
- #861: Pass plugin params for all plugins
- #859: Detach Elasticsearch on startup
- #858: Use the venv pylint in the makefile
- #841: Store Disk I/O metrics if available
- #825: Change remaining it tests to port 19200
- #820: Upgrade psutil to work with newer linux kernels
- #814: Fix error handling for automatically derived version
- #804: Allow multiple nodes per IP
- #801: Initialize schedule lazily
- #800: Properly wait for recovery to finish
- #799: Don't set retries for restore snapshot
- #784: Attach telemetry device on Docker launch
- #781: Add support for OSNAME and ARCH variables in dist repo URLs.
- #768: Honor ingest-percentage for bulks
- #763 (Breaking): Run a task completely even without time-periods
- #834: Clarify uses of distribution-version
- #826: Add note about mustache template
- #821: Move logrotate comments to separate lines
- #817: Fix mistake in wait-for-recovery docs
- #810: Fix duplicated IP in distributed_load diagram
- #782: Update Elasticsearch version of targets and metricstore in CCR recipe
- #764 (Breaking): Remove MergeParts internal telemetry device
- #762: Check that origin points to main repo for release
- #761: Allow to retry until success
- #760: Improve logging of schedules
- #759: Show track and team revision when listing races
- #758: Improve error message on SSL errors
- #755: Add pull request template
- #748: Consistently render links to the documentation
- #746: Fixup ProcessLauncherTests issues
- #744: Change DiskIO callbacks to use attach/detach
- #739 (Breaking): Remove lap feature and all references.
- #737: Allow to attach telemetry devices without reprovisioning
- #735: Assume string type for params by default
- #731: Update DiskIo telemetry device to persist the counters
- #730: Be resilient upon startup
- #729 (Breaking): Drop 1.x support for cluster metadata
- #728: Allow to set distribution version as parameter
- #726: Don't check complete list of parameters in integration test
- #725: Capture team and track revisions in metrics metadata
- #723: Always exit on OOME
- #721: Update DiskIo telemetry device to persist the counters
- #720: Change filestore to be indexed by unique ID
- #719: ES as a Daemon (again)
- #716 (Breaking): Drop support for Elasticsearch 1.x
- #711: Change telemetry devices to rely on jvm.config instead of ES_JAVA_OPTS
- #701: Implement ES daemon-mode in process launcher
- #751: Option track-revision should work with track-repository
- #750: Retrieve timestamped commit hash separately
- #747: Log git output
- #702: Add Rally Docker image to release process
- #688: Fail Rally early if there are unused variables in track-params
- #713: Check tags in track and team repos
- #709 (Breaking): Drop support for Python 3.4
- #708: Align default request param extraction for queries
- #707: Upgrade pip for virtualenv
- #705: Provide default for datastore.secure in all cases
- #704: Add download subcommand
- #700: Allow stdout print when running in Docker
- #694: Measure execution time of bulk ingest pipeline
- #691: Remove node_count_per_host
- #710 (Breaking): Don't pass request_cache by default
- #706: Register pyenv Python versions
- #699: Fix home directory paths in provisioner_test.
- #698: Enter degraded mode on bootstrap failure
- #693: Assume UTC timezone if not specified
- #683: Store mean for response-related metrics
- #681: Use single node discovery type if suitable
- #679: Skip Python install if already existing
- #678: Upgrade Jinja to 2.10.1
- #674: [Experimental] Capture peak usage of JVM mem pools
- #671: Add ability to download from S3 buckets
- #670: Pass JAVA12_HOME in integration tests
- #669: Allow to override request timeout for force-merge
- #668: Chart generator segment memory and new track combinations
- #667: Add sleep operation
- #666: Introduce new command line parameter --track-revision
- #662: Add distribution flavor to metrics metadata
- #660: Add user tags when comparing races
- #659: Sort the track parameters / user tags when listing races
- #654: Allow to use ES 7.x as metrics store
- #649 (Breaking): Pass request-params as is in supported operations
- #648: Updates to support 7.x APIs
- #687: Fix release charts after #684
- #686: Honor track-params in chart-generator
- #684: Use license instead of dist flavor in charts
- #682: Provide a platform-independent Rally binary
- #675: Fix chart generator segment memory bug
- #673: Honor runtime JDK in provisioner
- #661: Small change in the venv-create
- #655: Add compatibility layer for existing metrics store
- #652: Properly authenticate at proxy server
- #650: Use --prune for all git fetch operations
- #647: Make types optional
- #646: Add node_name in node-stats docs for ...
- #644: Allow collection of jvm gc section in node-stats telemetry device
- #642: Allow passing any request parameter to the create index API
- #641: Temporarily disable Python 3.4 in build
- #639: Add recovery-stats telemetry device
- #637: Ignore JSON logs for merge parts analysis
- #633: Ensure Makefile install target includes all release dependencies
- #638: Correct recorder-based sampling interval
- #630: Improve error message on missing repo directory
- #627: Warn about skewed results when using node-stats telemetry device
- #625: Allow to specify a team revision
- #622: Include NOTICE.txt in release
- #620: Add license headers
- #617: Fix conflicting pipelines and distribution version
- #615: Add refresh/merge/flush totals in summary
- #612: Extract hits either as number or structured object
- #606: Improve release.sh script and prerequisites
- #604: Change logging level for esrally command line to DEBUG
- #613: Error can be a string, and shards not always present in response.
- #632: Update Release documentation for Rally
- #623: Update Developing Rally docs sections
- #621: Minor typo
- #610: Command fails with "=", ":" is needed.
- #599: Simplify development setup
- #594: Add make install as default target
- #587: Fix typos and inconsistencies in help documentation
- #576: Improved formatting of document count in list tracks output
- #574: Rename benchmark data directory to root directory
- #572: More fine-grained ML metrics
- #571: Reduce logging when loading tracks
- #570: Add ML API runners
- #563: Make Rally compatible with Python 3.7
- #586: Don't fail git clone is tty is not attached
- #579: Switch to official approach for HTTP compression
- #575: Always use remote branch for updates
- #598: Document --limit
- #569: Add shrink index runner
- #559: Write rendered track to a temporary file
- #558: Avoid throughput throttling in test mode
- #557: Adjust flight recorder flags for JDK 11
- #556: Upgrade to Thespian 3.9.3
- #555: Load only required corpora
- #549: Exponential back-off for retrying transport errors to metrics store
- #546: Update ccr stats endpoint
- #545: Use non-standard port for metrics store in tests
- #540: Simplify filtering node-stats-related documents
- #538: Retry metric store for more transport errors
- #565: Improve compatibility when loading invalid JSON
- #562: Defer startup of mechanic actor
- #560: Disable automatic log rotation
- #551: Fix HTTP TLS client certificate verification
- #544: Fix parsing of boolean values datastore.secure
- #542: Warn when cluster settings cannot be applied
- #537: make --test-mode work with uncompressed data #536
- #533: State clearly that only Unix is supported
- #529: Allow to control recency of ids in conflicts
- #523: Use a single shard for metrics indices
- #522: Fine-tune logging
- #521: Remove deprecated usage of parameter source
- #518: Derive JDK version at runtime
- #516: Add multi-cluster support for NodeStats telemetry device
- #515: Fold all stats per shard in the same doc for CCR
- #514: Report indexing times per shard
- #513: Allow to store custom metric document format
- #512: Make challenges optional
- #509: Remove post_launch bootstrap phase
- #508: Add node-stats-include-mem option to record jvm heap stats
- #507: Set retry-on-timeout=true for ES clients used by Telemetry devices
- #503: Use file-based logging configuration
- #497: Add record_process_stats() for process.* stats
- #526: Resolve artefact name based on download URL
- #524: Disable interpolation for config file
- #511: Don't fail if conflict-probability is zero
- #510: Allow conflict-probability value of 0
- #500: Convert exception to string in driver
- #493: Sample more node stats
- #490: Record "took" for bulk indexing
- #489: Get distribution download URL from car config
- #488: Add support for multiple clusters in custom runners
- #487: Add new Makefile target to run it tests inside Docker
- #485: Allow to benchmark Elasticsearch with and without x-pack
- #481: Add post_launch phase for Elasticsearch plugins
- #480: Add generic node-stats telemetry device
- #477: Improve simulation of bulk-indexing conflicts
- #475: Rename telemetry device that gathers GC stats
- #473: Don't pass type implicitly for queries
- #476: Create unique ids even for random conflicts
- #471: Enable actor message handlers to fail immediately
- #470: Allow to use track parameters in index definition
- #469: Align operation param names with runners
- #468 (Breaking): Require at least Rally 0.7.3 for config migration
- #466: Don't require git
- #464: esrally must not log clear text user passwords
- #462: Upgrade Elasticsearch client to 6.2.0
- #461 (Breaking): Remove 'index-settings' property
- #460: Report store and translog size
- #458: Add parameter support to telemetry devices
- #456 (Breaking): Remove automatic index management
- #454: Allow to ingest a subset of a document corpus
- #453 (Breaking): Remove operation type "index"
- #441: Allow to pass parameters via a file
- #440: Use curl retry features to workaround transient network problems
- #467: Return "pages" as unit for scrolls
- #465: Default to upgrading packages with setuptools
- #449: Fix venv detection with venv and Rally execution on non master
- #446: Use more specific name for mandatory plugin check selectively
- #437: Bump pytest* versions and beautify make test output
- #434: Remove Python faulthandler
- #431: Add a unique race id
- #429: Allow to use track parameters also in index / template definitions
- #426: Allow simpler configuration of cluster configuration
- #377: Index-append operation only indexing bulk-size * clients documents
- #423: esrally fails after initial config run if JDK9 is not installed
- #424: Clarify use of virtualenv in developing doc
- #420: Cannot benchmark multi-node cluster with benchmark-only pipeline
- #418: Measure Elasticsearch startup time
- #417: Set heap dump path
- #416: Improve logging when gathering index time related metrics
- #413: Cannot report to ES cluster with self-signed certificate or in-house certificate authority
- #411: Store Rally version in results meta-data
- #410: Allow parameter sources to indicate progress
- #409: Allow to override car variables
- #407: Upgrade to thespian 3.9.2
- #405: Improve internal message handling
- #404: Add a raw request runner
- #402: Enforce UTF-8 encoding for file I/O
- #392: Formatting for user-tags on rally-results
- #399: 0.9.0 - Unable to run race due to missing JDK9
- #398: Allow to override plugin variables
- #387: Require JDK 9 for source builds
- #384: Provide a specific error message if data file is present but wrong size
- #383: Abort current benchmark in case of attempted duplicate starts
- #376: Record indexing throttle time
- #366: Separate document corpus definition from indices
- #361: Don't measure every operation
- #359: Allow to define index settings per index
- #348: Prefer default data directory when --track-path is given
- #293: Allow users to explicitly define index management operations
- #396: Rally requires a team repo even if used as a load generator
- #394: Error in track preparation can hang benchmarks
- #397: Add a gentle reminder to update the Rally kernel with the important elasticsearch system configurations
- #389: Syntax error in elasticsearch_plugins documentation
- #388: Update rally daemon port requirements
- #355: Run In Kubernetes
- #375: Add refresh API
- #310: Introduce track parameters and allow to override them on the command line
- #371: Introduce put pipeline API
- #369: Prepare rename from operation-type "index" to "bulk"
- #363: Allow to retry operations (internally)
- #362: Make plugin remote URL optional
- #360: Show ES distribution version upon race start
- #358: Rally tables should always output the same rows
- #354: Be more lenient when custom parameter source does not provide a parameter
- #353: Make a few track properties optional
- #352: Allow to use a challenge element if there is only one challenge in a track
- #351 (Breaking): Ensure task names are unique
- #345: Allow to use an ES distribution for plugins that are built separately
- #326: Allow to define operations inline
- #356 (Breaking): Set a sane default socket timeout
- #350 (Breaking): Number of iterations should be treated per client, not "global"
- #364: Deprecate --cluster-health
- #333: Provide a clear error message for non-existing releases
- #331: Don't demand a compressed representation of the document corpus
- #226: Provide a Rally package with all dependencies for offline install
- #217: Add ability to continuously stream metrics
- #344: Client assignment can miss a (small) number of docs
- #338: Documents are not found if track is referenced via a file
- #337: Only map numeric HTTP status codes to request meta-data
- #334: Data partition algorithm failling with parallel indexing tasks
- #343: If python is required then python-dev should be as well
- #231: Simplify usage of Rally for offline-only use
- #332: Provide more information about errors
- #330: Provide better error message when Rally reads an incompatible track
- #329: Allow to skip cluster health check
- #323: Rally should not demand a local Java install
- #309: Allow to benchmark plugins from sources
- #292: Add a "simple track" mode
- #259: Prepare Rally for Java 9
- #328: Challenge-specific index settings are not applied to index template definitions
- #322: Add error-type to request meta-data
- #321: Don't log body when creating a new index
- #319: Upgrade to thespian 3.8.0
- #315: Simplify multi-node configuration
- #313: Allow car mixins
- #265: Have Rally detect and use more Java versions
- #318: Run tasks indefinitely
- #257: Distribute load-test driver
- #316: KeyError: 'vm_vendor' when running benchmark against single host cluster with 0.7.0
- #314: Allow to filter executed tasks
- #312 (Breaking): Move action-and-meta-data to (indexing) type definition
- #308: Unify implementation of track and team repositories
- #307: Check Python version on startup and fail if it does not meet requirements
- #304: Make distribution repositories configurable
- #296: Verify whether the number of documents in the track is correct
- #278: relative-time should be reset per task
- #228: Increased flexibility for termination of parallel tasks
- #299: Leave the host parsing on elasticsearch-py
- #227: Enhance flexibility of user tagging
- #196 (Breaking): Externalize car configuration
- #298: Rally froze at the end of a race, did not produce results
- #295: Provide number of query hits as metrics metadata
- #291: Show track size metrics when listing tracks
- #290: Allow to pass arbitrary request parameters for ES queries
- #286: Additional metrics for bulk requests
- #282 (Breaking): Remove list facts subcommand
- #258: Make 'race' self-contained
- #284 (Breaking): Disallow previously deprecated usage of track properties in meta block
- #283: Store race results in a format optimized for reporting
- #279 (Breaking): Separate race and metrics indices
- #276: Add affected index to meta-data returned by bulk index runner
- #275: Allow to define per-challenge cluster-settings
- #267: Provide a progress-indication for downloads
- #266: Make the gc telemetry device Java 9 compatible
- #246: Rally should print a warning if there are no measurement samples
- #223: Allow unlimited number of pages for scroll queries
- #222: Report number of hits, not just pages, for scroll queries
- #220 (Breaking): Allow monthly indices for metrics
- #138 (Breaking): Improve support Elasticsearch 5 as metrics store
- #285: Rally is stuck for benchmarks with a very large number of requests
- #280: The file-reader parameter source does not seem to pass additional parameters
- #288: Clarify usage of remote track repositories in docs
- #287: Document throughput semantics for each operation
- #274: Explain summary report output
- #271: Document how to implement custom runner classes
- #264: Documentation on settings and warning message when using a documents archive file
- #251: Support for non-deterministic distribution of operations
- #260: Have bulk parameter source provide the bulk size
- #249: Saving tournament report
- #155: Improve document count handling
- #263: Race condition when rolling log files
- #261: Unable to determine valid external socket address
- #253: Cannot determine CPU info for POWER8 chips
- #242: Rally does not detect the distribution version correctly for externally provisioned clusters
- #235: Allow Python files in a track directory that are unrelated to the track plugin
- #256: Document additional decompression options
- #247: Description of available operation types in the docs
- #241: Clearly document which Elasticsearch versions are supported
- #244: Show a nice error message when user tries to run a non-existing challenge on an existing track
- #245: Connection behind a proxy fails. has_internet_connection() returns False
- #234: Add request error rate to summary report
- #238: Simplify gathering of facts for benchmark results sharing
- #216: Add possibility to profile load driver internals
- #184: Allow to benchmark a single-machine cluster remotely
- #207: Improve response checks in bulk API runner
- #205: Make track configuration more modular
- #200: Allow root.dir to be set during configure
- #199: Let track authors decide on the default challenge
- #191: Support target-interval for operations
- #225: Final score report blank when metrics store specified
- #221: Scrolls fail against Elasticsearch master
- #203: Index time metrics are not aware of laps
- #202: Bulk index source reads data for all indices
- #224: Issue(s) with --user-tags
- #214: Improve documentation of "parallel"
- #213: Document how to support --test-mode in a track
- #208: Add FAQ item to explain latency, service time and their relation to "took"
The track format has changed a bit due a more flexible approach in how benchmarks are executed:
- Operations are defined in the
operationssection, execution details like number of warmup iterations, warmup time etc. are defined as part of theschedule. - Each query needs to be defined as a separate operation and referenced in the
schedule - You can (and in fact should) specify a
warmup-time-period(defined in sections) for bulk index operations. The warmup time period is specified in seconds.
For details please refer to the updated JSON schema for Rally tracks.
Hint: This is just relevant for you, if you have defined your own tracks. We already took care of updating the official Rally tracks.
We have separated the previously known "track setup" into two parts:
- Challenges: Which describe what happens during a benchmark (whether to index or search and with which parameters)
- Cars: Which describe the benchmark candidate settings (e.g. heap size, logging configuration etc.)
This influences the command line interface in a couple of ways:
- To list all known cars, we have added a new command
esrally list cars. To select a challenge, use now--challengeinstead of--track-setupand also specify a car now with--car. - Tournaments created by older versions of Rally are incompatible
- Rally must now be invoked with only one challenge and only one car (previously it was possible to specify multiple track setups)
We have also moved tracks to a dedicated repository. This allows you to support tracks for multiple versions of Elasticsearch but also requires that all users have git installed.
We have spent a lot of time to simplify first time setup of Rally. For starters, you are not required to setup your own metrics store if you don't need it.
However, you are then just able to run individual benchmarks but you cannot compare results or visualize anything in Kibana. If you don't need this, it is recommended that you
remove the configuration directory and run esrally configure. Rally will notify you on its first start of this change and guide you through the process.
Please raise a ticket in case of problems.
- Add a tournament mode. More information in the user docs
- External benchmarks can now specify target hosts and ports
- Ability to add a user-defined tag as metric meta-data
- Support .gzipped benchmark data contributed by @monk-ee. Thanks!
- Support for perf profiler
- Add a fulltext benchmark
Major changes:
- Rally can now benchmark a binary Elasticsearch distribution (starting with Elasticsearch 5.0.0-alpha1).
- Reporting improvements for query latency and indexing throughput on the command line.
- We store benchmark environment data alongside metrics.
- A new percolator track](#74) contributed by Martijn van Groningen. Thanks!
Major changes:
- Added a JIT profiler. This allows to check warmup times but also in-depth inspection which optimizations were performed by the JIT compiler. If the HotSpot disassembler library is available, the logs will also contain the disassembled JIT compiler output which can be used for low-level analysis. We recommend to use JITWatch for analysis.
- Added pipeline support. Pipelines allow to define more flexibly which steps Rally executes during a benchmark. One of the use-cases for this is to run a benchmark based on a released build of Elasticsearch rather than building it ourselves.
Major changes:
-
Migrated the metrics data store from file-based to a dedicated Elasticsearch instance. Graphical reports can be created with Kibana (optional but recommended). It is necessary to setup an Elasticsearch cluster to store metrics data (a single node is sufficient). The cluster will be configured automatically by Rally. For details please see the documentation.
Related issues: #8, #21, #46,