Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[air/preprocessors] Concatenator doesn't work for pandas==1.2.1 #27503

Closed
richardliaw opened this issue Aug 4, 2022 · 1 comment · Fixed by #27531
Closed

[air/preprocessors] Concatenator doesn't work for pandas==1.2.1 #27503

richardliaw opened this issue Aug 4, 2022 · 1 comment · Fixed by #27531
Assignees
Labels
bug Something that is supposed to be working; but isn't P0 Issues that should be fixed in short order release-blocker P0 Issue that blocks the release

Comments

@richardliaw
Copy link
Contributor

What happened + What you expected to happen

Also... this is a terrible error message.

(TorchTrainer pid=88526) 2022-08-04 10:42:13,240        ERROR function_trainable.py:298 -- Runner Thread raised error.
(TorchTrainer pid=88526) Traceback (most recent call last):
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 289, in run
(TorchTrainer pid=88526)     self._entrypoint()
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 365, in entrypoint
(TorchTrainer pid=88526)     self._status_reporter.get_checkpoint(),
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/util/tracing/tracing_helper.py", line 466, in _resume_span
(TorchTrainer pid=88526)     return method(self, *_args, **_kwargs)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/train/base_trainer.py", line 458, in _trainable_func
(TorchTrainer pid=88526)     super()._trainable_func(self._merged_config, reporter, checkpoint_dir)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 684, in _trainable_func
(TorchTrainer pid=88526)     output = fn()
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/train/base_trainer.py", line 372, in train_func
(TorchTrainer pid=88526)     trainer.preprocess_datasets()
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/train/data_parallel_trainer.py", line 300, in preprocess_datasets
(TorchTrainer pid=88526)     self.preprocessor, self.datasets
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/train/_internal/dataset_spec.py", line 153, in preprocess_datasets
(TorchTrainer pid=88526)     new_datasets[key] = prep.transform(dataset)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/preprocessor.py", line 143, in transform
(TorchTrainer pid=88526)     transformed_ds = self._transform(dataset)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/preprocessors/chain.py", line 64, in _transform
(TorchTrainer pid=88526)     ds = preprocessor.transform(ds)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/preprocessor.py", line 143, in transform
(TorchTrainer pid=88526)     transformed_ds = self._transform(dataset)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/preprocessor.py", line 234, in _transform
(TorchTrainer pid=88526)     return dataset.map_batches(self._transform_pandas, batch_format="pandas")
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/dataset.py", line 495, in map_batches
(TorchTrainer pid=88526)     return Dataset(plan, self._epoch, self._lazy)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/dataset.py", line 201, in __init__
(TorchTrainer pid=88526)     self._plan.execute(allow_clear_input_blocks=False)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/_internal/plan.py", line 309, in execute
(TorchTrainer pid=88526)     blocks, clear_input_blocks, self._run_by_consumer
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/_internal/plan.py", line 672, in __call__
(TorchTrainer pid=88526)     fn_constructor_kwargs=self.fn_constructor_kwargs,
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 126, in _apply
(TorchTrainer pid=88526)     raise e from None
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 113, in _apply
(TorchTrainer pid=88526)     results = map_bar.fetch_until_complete(refs)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/_internal/progress_bar.py", line 75, in fetch_until_complete
(TorchTrainer pid=88526)     for ref, result in zip(done, ray.get(done)):
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/_private/client_mode_hook.py", line 105, in wrapper
(TorchTrainer pid=88526)     return func(*args, **kwargs)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/_private/worker.py", line 2245, in get
(TorchTrainer pid=88526)     raise value.as_instanceof_cause()
(TorchTrainer pid=88526) ray.exceptions.RayTaskError(ValueError): ray::_map_block_nosplit() (pid=88535, ip=127.0.0.1)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 449, in _map_block_nosplit
(TorchTrainer pid=88526)     for new_block in block_fn(block, *fn_args, **fn_kwargs):
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/dataset.py", line 457, in transform
(TorchTrainer pid=88526)     applied = batch_fn(view, *fn_args, **fn_kwargs)
(TorchTrainer pid=88526)   File "/Users/rliaw/ray/python/ray/data/preprocessors/concatenator.py", line 94, in _transform_pandas
(TorchTrainer pid=88526)     df.loc[:, self.output_column_name] = list(concatenated)
(TorchTrainer pid=88526)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 692, in __setitem__
(TorchTrainer pid=88526)     iloc._setitem_with_indexer(indexer, value, self.name)
(TorchTrainer pid=88526)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1604, in _setitem_with_indexer
(TorchTrainer pid=88526)     self._setitem_with_indexer(new_indexer, value, name)
(TorchTrainer pid=88526)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1635, in _setitem_with_indexer
(TorchTrainer pid=88526)     self._setitem_with_indexer_split_path(indexer, value, name)
(TorchTrainer pid=88526)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1672, in _setitem_with_indexer_split_path
(TorchTrainer pid=88526)     self._setitem_with_indexer_2d_value(indexer, value)
(TorchTrainer pid=88526)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1733, in _setitem_with_indexer_2d_value
(TorchTrainer pid=88526)     "Must have equal len keys and value when setting with an ndarray"
(TorchTrainer pid=88526) ValueError: Must have equal len keys and value when setting with an ndarray
(_map_block_nosplit pid=88535) TOTAL LIST LEN 398
(_map_block_nosplit pid=88535) DF (398, 1)
(_map_block_nosplit pid=88535) 2022-08-04 10:42:13,238  INFO worker.py:754 -- Task failed with retryable exception: TaskID(c93105c99631c023ffffffffffffffffffffffff01000000).
(_map_block_nosplit pid=88535) Traceback (most recent call last):
(_map_block_nosplit pid=88535)   File "python/ray/_raylet.pyx", line 662, in ray._raylet.execute_task
(_map_block_nosplit pid=88535)   File "python/ray/_raylet.pyx", line 666, in ray._raylet.execute_task
(_map_block_nosplit pid=88535)   File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 449, in _map_block_nosplit
(_map_block_nosplit pid=88535)     for new_block in block_fn(block, *fn_args, **fn_kwargs):
(_map_block_nosplit pid=88535)   File "/Users/rliaw/ray/python/ray/data/dataset.py", line 457, in transform
(_map_block_nosplit pid=88535)     applied = batch_fn(view, *fn_args, **fn_kwargs)
(_map_block_nosplit pid=88535)   File "/Users/rliaw/ray/python/ray/data/preprocessors/concatenator.py", line 94, in _transform_pandas
(_map_block_nosplit pid=88535)     df.loc[:, self.output_column_name] = list(concatenated)
(_map_block_nosplit pid=88535)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 692, in __setitem__
(_map_block_nosplit pid=88535)     iloc._setitem_with_indexer(indexer, value, self.name)
(_map_block_nosplit pid=88535)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1604, in _setitem_with_indexer
(_map_block_nosplit pid=88535)     self._setitem_with_indexer(new_indexer, value, name)
(_map_block_nosplit pid=88535)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1635, in _setitem_with_indexer
(_map_block_nosplit pid=88535)     self._setitem_with_indexer_split_path(indexer, value, name)
(_map_block_nosplit pid=88535)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1672, in _setitem_with_indexer_split_path
(_map_block_nosplit pid=88535)     self._setitem_with_indexer_2d_value(indexer, value)
(_map_block_nosplit pid=88535)   File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1733, in _setitem_with_indexer_2d_value
(_map_block_nosplit pid=88535)     "Must have equal len keys and value when setting with an ndarray"
(_map_block_nosplit pid=88535) ValueError: Must have equal len keys and value when setting with an ndarray
2022-08-04 10:42:13,450 ERROR trial_runner.py:980 -- Trial TorchTrainer_c1456_00000: Error processing event.
ray.exceptions.RayTaskError(ValueError): ray::TrainTrainable.train() (pid=88526, ip=127.0.0.1, repr=TorchTrainer)
  File "/Users/rliaw/ray/python/ray/tune/trainable/trainable.py", line 347, in train
    result = self.step()
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 417, in step
    self._report_thread_runner_error(block=True)
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 589, in _report_thread_runner_error
    raise e
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 289, in run
    self._entrypoint()
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 365, in entrypoint
    self._status_reporter.get_checkpoint(),
  File "/Users/rliaw/ray/python/ray/train/base_trainer.py", line 458, in _trainable_func
    super()._trainable_func(self._merged_config, reporter, checkpoint_dir)
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 684, in _trainable_func
    output = fn()
  File "/Users/rliaw/ray/python/ray/train/base_trainer.py", line 372, in train_func
    trainer.preprocess_datasets()
  File "/Users/rliaw/ray/python/ray/train/data_parallel_trainer.py", line 300, in preprocess_datasets
    self.preprocessor, self.datasets
  File "/Users/rliaw/ray/python/ray/train/_internal/dataset_spec.py", line 153, in preprocess_datasets
    new_datasets[key] = prep.transform(dataset)
  File "/Users/rliaw/ray/python/ray/data/preprocessor.py", line 143, in transform
    transformed_ds = self._transform(dataset)
  File "/Users/rliaw/ray/python/ray/data/preprocessors/chain.py", line 64, in _transform
    ds = preprocessor.transform(ds)
  File "/Users/rliaw/ray/python/ray/data/preprocessor.py", line 143, in transform
    transformed_ds = self._transform(dataset)
  File "/Users/rliaw/ray/python/ray/data/preprocessor.py", line 234, in _transform
    return dataset.map_batches(self._transform_pandas, batch_format="pandas")
  File "/Users/rliaw/ray/python/ray/data/dataset.py", line 495, in map_batches
    return Dataset(plan, self._epoch, self._lazy)
  File "/Users/rliaw/ray/python/ray/data/dataset.py", line 201, in __init__
    self._plan.execute(allow_clear_input_blocks=False)
  File "/Users/rliaw/ray/python/ray/data/_internal/plan.py", line 309, in execute
    blocks, clear_input_blocks, self._run_by_consumer
  File "/Users/rliaw/ray/python/ray/data/_internal/plan.py", line 672, in __call__
    fn_constructor_kwargs=self.fn_constructor_kwargs,
  File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 126, in _apply
    raise e from None
  File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 113, in _apply
    results = map_bar.fetch_until_complete(refs)
  File "/Users/rliaw/ray/python/ray/data/_internal/progress_bar.py", line 75, in fetch_until_complete
    for ref, result in zip(done, ray.get(done)):
ray.exceptions.RayTaskError(ValueError): ray::_map_block_nosplit() (pid=88535, ip=127.0.0.1)
  File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 449, in _map_block_nosplit
    for new_block in block_fn(block, *fn_args, **fn_kwargs):
  File "/Users/rliaw/ray/python/ray/data/dataset.py", line 457, in transform
    applied = batch_fn(view, *fn_args, **fn_kwargs)
  File "/Users/rliaw/ray/python/ray/data/preprocessors/concatenator.py", line 94, in _transform_pandas
    df.loc[:, self.output_column_name] = list(concatenated)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 692, in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1604, in _setitem_with_indexer
    self._setitem_with_indexer(new_indexer, value, name)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1635, in _setitem_with_indexer
    self._setitem_with_indexer_split_path(indexer, value, name)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1672, in _setitem_with_indexer_split_path
    self._setitem_with_indexer_2d_value(indexer, value)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1733, in _setitem_with_indexer_2d_value
    "Must have equal len keys and value when setting with an ndarray"
ValueError: Must have equal len keys and value when setting with an ndarray
Result for TorchTrainer_c1456_00000:
  date: 2022-08-04_10-42-07
  experiment_id: e77ebac889bb4883816946ccc51336bd
  hostname: Richards-MBP.local.meter
  node_ip: 127.0.0.1
  pid: 88526
  timestamp: 1659634927
  trial_id: c1456_00000
  
== Status ==
Current time: 2022-08-04 10:42:13 (running for 00:00:08.08)
Memory usage on this node: 26.7/64.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/34.48 GiB heap, 0.0/2.0 GiB objects
Result logdir: /Users/rliaw/ray_results/TorchTrainer_2022-08-04_10-42-05
Number of trials: 1/1 (1 ERROR)
+--------------------------+----------+-----------------+
| Trial name               | status   | loc             |
|--------------------------+----------+-----------------|
| TorchTrainer_c1456_00000 | ERROR    | 127.0.0.1:88526 |
+--------------------------+----------+-----------------+
Number of errored trials: 1
+--------------------------+--------------+--------------------------------------------------------------------------------------------------------------------+
| Trial name               |   # failures | error file                                                                                                         |
|--------------------------+--------------+--------------------------------------------------------------------------------------------------------------------|
| TorchTrainer_c1456_00000 |            1 | /Users/rliaw/ray_results/TorchTrainer_2022-08-04_10-42-05/TorchTrainer_c1456_00000_0_2022-08-04_10-42-05/error.txt |
+--------------------------+--------------+--------------------------------------------------------------------------------------------------------------------+

2022-08-04 10:42:13,563 ERROR tune.py:754 -- Trials did not complete: [TorchTrainer_c1456_00000]
2022-08-04 10:42:13,563 INFO tune.py:759 -- Total run time: 8.32 seconds (8.08 seconds for the tuning loop).
Traceback (most recent call last):
  File "/Users/rliaw/ray/doc/source/ray-air/examples/pytorch_tabular_starter.py", line 118, in <module>
    result = trainer.fit()
  File "/Users/rliaw/ray/python/ray/train/base_trainer.py", line 347, in fit
    raise result.error
types.RayTaskError(ValueError): ray::TrainTrainable.train() (pid=88526, ip=127.0.0.1, repr=TorchTrainer)
  File "/Users/rliaw/ray/python/ray/tune/trainable/trainable.py", line 347, in train
    result = self.step()
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 417, in step
    self._report_thread_runner_error(block=True)
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 589, in _report_thread_runner_error
    raise e
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 289, in run
    self._entrypoint()
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 365, in entrypoint
    self._status_reporter.get_checkpoint(),
  File "/Users/rliaw/ray/python/ray/train/base_trainer.py", line 458, in _trainable_func
    super()._trainable_func(self._merged_config, reporter, checkpoint_dir)
  File "/Users/rliaw/ray/python/ray/tune/trainable/function_trainable.py", line 684, in _trainable_func
    output = fn()
  File "/Users/rliaw/ray/python/ray/train/base_trainer.py", line 372, in train_func
    trainer.preprocess_datasets()
  File "/Users/rliaw/ray/python/ray/train/data_parallel_trainer.py", line 300, in preprocess_datasets
    self.preprocessor, self.datasets
  File "/Users/rliaw/ray/python/ray/train/_internal/dataset_spec.py", line 153, in preprocess_datasets
    new_datasets[key] = prep.transform(dataset)
  File "/Users/rliaw/ray/python/ray/data/preprocessor.py", line 143, in transform
    transformed_ds = self._transform(dataset)
  File "/Users/rliaw/ray/python/ray/data/preprocessors/chain.py", line 64, in _transform
    ds = preprocessor.transform(ds)
  File "/Users/rliaw/ray/python/ray/data/preprocessor.py", line 143, in transform
    transformed_ds = self._transform(dataset)
  File "/Users/rliaw/ray/python/ray/data/preprocessor.py", line 234, in _transform
    return dataset.map_batches(self._transform_pandas, batch_format="pandas")
  File "/Users/rliaw/ray/python/ray/data/dataset.py", line 495, in map_batches
    return Dataset(plan, self._epoch, self._lazy)
  File "/Users/rliaw/ray/python/ray/data/dataset.py", line 201, in __init__
    self._plan.execute(allow_clear_input_blocks=False)
  File "/Users/rliaw/ray/python/ray/data/_internal/plan.py", line 309, in execute
    blocks, clear_input_blocks, self._run_by_consumer
  File "/Users/rliaw/ray/python/ray/data/_internal/plan.py", line 672, in __call__
    fn_constructor_kwargs=self.fn_constructor_kwargs,
  File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 126, in _apply
    raise e from None
  File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 113, in _apply
    results = map_bar.fetch_until_complete(refs)
  File "/Users/rliaw/ray/python/ray/data/_internal/progress_bar.py", line 75, in fetch_until_complete
    for ref, result in zip(done, ray.get(done)):
ray.exceptions.RayTaskError(ValueError): ray::_map_block_nosplit() (pid=88535, ip=127.0.0.1)
  File "/Users/rliaw/ray/python/ray/data/_internal/compute.py", line 449, in _map_block_nosplit
    for new_block in block_fn(block, *fn_args, **fn_kwargs):
  File "/Users/rliaw/ray/python/ray/data/dataset.py", line 457, in transform
    applied = batch_fn(view, *fn_args, **fn_kwargs)
  File "/Users/rliaw/ray/python/ray/data/preprocessors/concatenator.py", line 94, in _transform_pandas
    df.loc[:, self.output_column_name] = list(concatenated)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 692, in __setitem__
    iloc._setitem_with_indexer(indexer, value, self.name)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1604, in _setitem_with_indexer
    self._setitem_with_indexer(new_indexer, value, name)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1635, in _setitem_with_indexer
    self._setitem_with_indexer_split_path(indexer, value, name)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1672, in _setitem_with_indexer_split_path
    self._setitem_with_indexer_2d_value(indexer, value)
  File "/Users/rliaw/miniconda3/envs/raybuild/lib/python3.7/site-packages/pandas/core/indexing.py", line 1733, in _setitem_with_indexer_2d_value
    "Must have equal len keys and value when setting with an ndarray"
ValueError: Must have equal len keys and value when setting with an ndarray

Versions / Dependencies

pandas==1.2.1
ray==2.0rc0 or master

Reproduction script

import ray
import numpy as np
import pandas as pd

from ray.data.preprocessors import StandardScaler
from ray.air.config import ScalingConfig

from ray.data.preprocessors import Concatenator, Chain

# Load data.
dataset = ray.data.read_csv("s3://anonymous@air-example-data/breast_cancer.csv")

# Split data into train and validation.
train_dataset, valid_dataset = dataset.train_test_split(test_size=0.3)

# Create a preprocessor to scale some columns
columns_to_scale = ["mean radius", "mean texture"]
preprocessor = StandardScaler(columns=columns_to_scale)

preprocessor = Chain(
    preprocessor,
    Concatenator(exclude=["target"], dtype=np.float32),
)

preprocessor.fit_transform(train_dataset)

Issue Severity

High: It blocks me from completing my task.

@richardliaw richardliaw added bug Something that is supposed to be working; but isn't release-blocker P0 Issue that blocks the release P0 Issues that should be fixed in short order labels Aug 4, 2022
@richardliaw
Copy link
Contributor Author

Works for latest version of pandas, which is what we test in CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't P0 Issues that should be fixed in short order release-blocker P0 Issue that blocks the release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants