Pass multiple Locustfiles and allow selecting User and Shape class from the WebUI

[mikenester]

My team has a use case where we'd like to be able to pick User classes after spinning up the WebUI. The goal of the PR is to:

• add this functionality without changing existing workflows
• Run locust with multiple locustfiles. Primarily to be used with the Class Picker

The -f argument has been updated to allow for multiple locust files or a directory. All User classes in the specified locustfiles will be used by default.

The newly added --class-picker option enables the user to select the desired User classes & shape class in the Web Ui. If no selection is made for User classes, all User classes are used. The shape class defaults to the regular shape.

Other noteworthy changes:

• Error raised if the specified directory doesn't exist, or doesn't have any valid Locustfiles
• If the Class Picker is visible in the WebUI and there is no UserClass selection, /swarm will use all available UserClasses
• load_locustfile, is_shape_class, is_user_class were moved from main.py to util/load_locustfile.py to avoid a circular import in web.py
  • This is no longer necessary for this PR, but I kept it in to cut down on the size of main.py

Example:
Running locust -f locustfiles --class-picker with the following file structure:

├── locustfiles/
│   ├── file1.py
│   ├── file2.py
│   └── shape_classes/
│       ├── shape1.py
│       ├── shape1.py
│   └── more_files/
│       ├── file_3.py
│       ├── locust.py
│       ├── _ignoreme.py

[cyberw]

Cant we make this a non-breaking change by just putting the parameter last and defaulting to use parse_options() inside in create_web_ui() if nothing is provided?

[mikenester]

Sounds good to me, I'll give it a shot

[mikenester]

After reading your other comment, I realized that my approach here was probably very unnecessary. I'm already passing in show_locustfiles, which I can use for the same check in /swarm to see if the locustfile in the payload needs to be loaded.

[cyberw]

Nice! Couple comments, and I’ll need to do a full review when not on mobile :)

[cyberw]

Hmm. When I think about it - could this be solved by allowing multiple parameters for -f / —locusfile instead? It seems a more obvious and user friendly approach (if you want a whole directory you would just do -f mydir/*)

[mikenester]

Hmm. When I think about it - could this be solved by allowing multiple parameters for -f / —locusfile instead? It seems a more obvious and user friendly approach (if you want a whole directory you would just do -f mydir/*)

Ah nice, I think this is a great idea!

[cyberw]

Can you change it so that it works that way? I guess using parser.add_argument('-f', …, nargs='+', …)

Also, maybe we could make it so that you select Users in the GUI instead of locustfiles (and default to run all users). That would make a lot of sense, I think.

[mikenester]

Can you change it so that it works that way? I guess using parser.add_argument('-f', …, nargs='+', …)

Yup! I've already got that working locally, I just need to fix and add tests 😄 However, for the use case of passing in a directory, I add another flag --locustfile-is-directory since passing in some_directory/* doesn't grab files in child directories. So the use cases are:

• locust -f some_locustfile.py
• locust -f some_locustfile.py another_locustfile.py morefiles/third_locustfile.py
• locust -f some_directory --locustfile-is-directory

I think before I push up, however, I'm going to see if I can try to not use the flag by adding a check in parse_locustfile_option to see if the argument is a file or a directory.

Also, maybe we could make it so that you select Users in the GUI instead of locustfiles (and default to run all users). That would make a lot of sense, I think.

Great idea! I think this would be easier to read in the UI as well. I actually got started on the PR from an old thread I saw in the Issues where someone got that working so some degree, but never opened it up as a PR. I'm busy this week, so I might not have any commits up until Monday (just fyi).

[mikenester]

@cyberw I just pushed up a few commits, but this isn't ready for re-review yet. I still have to update the docs and fix some failing tests (I'm on an M1 and it seems like some tests fail no matter what locally, but are fine in Github Actions. Some due to slowness)

[bebo-dot-dev]

I've nothing constructive to add other than to say thank you for working on this pull request. I've been thinking about this feature for about a week so decided to take a look today to see if it had already been discussed .. and found this :)

[cyberw]

Merge with master for a fix for the build (flask made an api change in 2.2)

[mikenester]

I've nothing constructive to add other than to say thank you for working on this pull request. I've been thinking about this feature for about a week so decided to take a look today to see if it had already been discussed .. and found this :)

Thanks! I'm glad others have use for this as well 🤘

[mikenester]

@cyberw This is ready for re-review. I've updated with your requested changes, most notably that the UI allows you to pick from UserClasses instead of Locustfiles (which is a much better so thanks for the suggestion!). I also added in a ShapeClass picker which only accepts one value and defaults to the normal use-case and only appears when the UserClass picker is active.

Additionally, there is no need for any additional command-line arguments/flags. -f/--locustfile accepts:

• A single Locustfile (no changes to this logic)
  • locust -f some_locustfile.py
• Multiple Locustfiles
  • locust -f some_locustfile.py some_dir/another_locustfile.py
• A directory containing locustfiles
  • locust -f locustfiles

[cyberw]

Is it not possible to make headless work with multiple files?

[cyberw]

(Defaulting to run all of them, but also having the ability to specify a subset of Users, as with a single file)

[mikenester]

Ah so if --headless and userclass_picker_is_active==True, then use all UserClasses in the specified locustfiles? That makes sense! But, I'm just wondering how to handle ShapeClasses in the Locustfiles, specifically if there is more than one?

Defaulting to run all of them, but also having the ability to specify a subset of Users, as with a single file

I think defaulting to running them all is the only option here, as there wouldn't be a way to specify which UserClasses to use since the UI would be disabled. Unless I'm totally missing something ??? (likely since I'm still a Locust noob)

[cyberw]

Any trailing arguments are used to select from users (e.g. locust … User1 User2)

[cyberw]

If there is more than one shape class, then we can just crash

[cyberw]

Is this even needed now? Arent we just loading all the files and only changing Users and shape class at run time?

[mikenester]

I'm pretty confident that it is, because if multiple locustfiles are used, then main.py doesn't initiate the Environment with the UserClasses (see line 72 of main.py). So update_environment needs to update the user_classes and shape_class properties. The other ones can probably be removed, I forgot to test that out, but will do so now.

Additionally, the application breaks without this line environment.runner._users_dispatcher = None

[cyberw]

if multiple locustfiles are used, then main.py doesn't initiate the Environment with the UserClasses (see line 72 of main.py).

but why cant main.py initiate it? (Now that we’re not selecting which locustfile to run after startup)

[mikenester]

That's a good point. I modified main.py to:

if userclass_picker_is_active:
    # iterate through each locustfile and 'load' them
else:
    docstring, user_classes, shape_class = load_locustfile(locustfile)

I think the # iterate through each locustfile and 'load' them block would work for both scenarios. I'll update that.

But i think update_environment would still be needed to be called when the user selects a UserClass or Swarm class in the UI. So that the environment is updated with the UserClass/ShapeClass selection and also so that the following is called:

environment.runner.stop()
environment.runner._users_dispatcher = None

which causes this user dispatcher to be recreated with the selected UserClasses: https://github.com/locustio/locust/blob/master/locust/runners.py#L491

[cyberw]

But i think update_environment would still be needed to be called when the user selects a UserClass or Swarm class in the UI. So that the environment is updated with the UserClass/ShapeClass selection and also so that the following is called

Yes, that makes sense. I’m unsure about having user_picker_is_active as an implied effect of having multiple files specified though? Its a very UI-specific thing but impacts behaviour deep inside main. Maybe have it as a separate setting? Or just always have it on but hide the User selection more in the UI?

[mikenester]

I think the other change will resolve half of this comment. main will essentially only use userclass_picker_is_active to toggle the UI to show the UserClass & ShapeClass select fields. The other changes I made using that variable in main won't be necessary.

In terms of having user_picker_is_active as an implied effect of multiple files, I think the best alternative is to just have it as a CLI Option (which seems like that's what you might be suggesting)

[cyberw]

👍👍

[mikenester]

@cyberw I've made all the requested updates:

• Added --enable-userclass-picker option to explicitly trigger the UI selections. This cleans up the issues you had with userclass_picker_is_enabled behavior in main. There are no more conditional checks since Environment is always created user_classes and shape_class.
  • Now options.enable_userclass_picker is only used in main to enable the UI selections (line 262) and to log that the UserClass picker is enabled (line 445)
• The new workflow is that /swarm will update the environment only if the Userclass Picker is enabled AND those values are in the payload. This also lead to removing update_environment for a more simplified approach

There is a breaking change that was implemented with allowing multiple values to be passed into -f/--locustfile. I only just now caught it. Specifying User classes in the CLI cannot be done after the -f arguments, since it expects the values to be locustfiles.

Good:

• locust -f locustfiles --csv test_csv QuickstartUser1
• `locust QuickstartUser1 -f locustfiles

Bad:

• locust -f locustfiles QuickstartUser1
  • Error: Invalid file 'locustfiles'. File should have '.py' extension

I didn't update the docs on this yet as I wanted to see if I should keep it as is (maybe at a note in the error message), or perhaps add a flag to specify the User classes.

[cyberw]

This is starting to look really good. Anything else before we merge? Can you squash the commits that didnt really relate to the final product? Or I can squash everything into one commit

[mikenester]

This is starting to look really good.
Thanks! Your input made it 10x better than what I initially had

Anything else before we merge? Can you squash the commits that didnt really relate to the final product? Or I can squash everything into one commit

Yeah, actually. Incase you didn't see my other comment:
There is a breaking change that was implemented with allowing multiple values to be passed into -f/--locustfile. I only just now caught it. Specifying User classes in the CLI cannot be done after the -f arguments, since it expects the values to be locustfiles.

Good:

• locust -f locustfiles --csv test_csv QuickstartUser1
• locust QuickstartUser1 -f locustfiles

Bad:

• locust -f locustfiles QuickstartUser1
  • Error: Invalid file 'locustfiles'. File should have '.py' extension

I didn't update the docs on this yet as I wanted to see if I should keep it as is (maybe at a note in the error message), or perhaps add a flag to specify the User classes.

I'm thinking that the best move is to keep it as is and update the docs & error message, since that will affect the least amount of users.

[cyberw]

Oh, right. Yea we probably dont want to do that. Maybe use a comma separated list? Argparse doesnt seem to support it out of the box so SO recommends something like

https://stackoverflow.com/questions/52132076/argparse-action-or-type-for-comma-separated-list

[mikenester]

Oh, right. Yea we probably dont want to do that. Maybe use a comma separated list? Argparse doesnt seem to support it out of the box so SO recommends something like

https://stackoverflow.com/questions/52132076/argparse-action-or-type-for-comma-separated-list

Done. Excellent suggestion 🙏 . I also squashed all of the commits.

[cyberw]

Minor details:

• remove ”enable” from —enable-user-picker
• make the default to actually select all users (then you can also remove the message about no selected users resulting in all users)
• Update the image

[cyberw]

Btw, its kind of a ”user AND shape class” selector, isnt it? Maybe it should be named something different?

[mikenester]

Minor details:

remove ”enable” from —enable-user-picker

That's how I originally had it, but test_specify_config_file was failing due to an argparse error:

ambiguous option: --u=100 could match --users, --userclass-picker

make the default to actually select all users (then you can also remove the message about no selected users resulting in all users)
Update the image

You're referring to the UI select box, right?

[mikenester]

Btw, its kind of a ”user AND shape class” selector, isn't it? Maybe it should be named something different?

Yup! I couldn't think of a better name 😐 . Open to suggestions

[cyberw]

Not sure if it works, but maybe —class-selector or —class-picker?

if that doesnt work then maybe —select-class or —pick-class

[mikenester]

@cyberw
I updated:

• --enable-userclass-picker -> --class-picker
• User class fields are pre-selected in the UI

[cyberw]

Looks good now! Can you just update the PR description?

[mikenester]

Looks good now! Can you just update the PR description?

Done 👍

[cyberw]

Thx!