Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-3166: Mimic Flow behavior in artillery #4771

Merged
merged 3 commits into from
Aug 6, 2024

Conversation

TimDiekmann
Copy link
Member

🌟 What is the purpose of this PR?

Flows run asynchronously, so to observe specific behavior and load-test the API we need to kimic some flows.

🔍 What does this change?

  • Patch artillery library to correctly use JSON: Key containing periods behave oddly artilleryio/artillery#879 (comment)
  • Add a flow scenario which simply creates two entities 10 times (one at the graph, one at GraphQL) - this is not a real flow behavior, but at least it's creating entities
  • Adjust session code to properly detect expiration dates
  • Remove unneeded/outdated type-dependency

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

@TimDiekmann TimDiekmann self-assigned this Aug 1, 2024
@github-actions github-actions bot added area/deps Relates to third-party dependencies (area) area/tests New or updated tests labels Aug 1, 2024
@@ -8,7 +8,7 @@ const OUT_DIR = "dist/esm";

const bundles: RollupOptions[] = [
{
input: "src/main.ts",
input: ["src/main.ts"],
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accidentally committed this, but it doesn't hurt, either, as it's easier to extend 😄

Comment on lines +32 to +34
- post:
name: Create entities
url: http://127.0.0.1:4000/entities/bulk
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept the graph endpoint to show how this is used

createEntity(ownedById: $ownedById, entityTypeId: $entityTypeId, properties: $properties)
}
variables:
ownedById: "{{ session.ownedById }}"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not strictly needed but we have it available.

const setAuthorizationHeader = (request: RequestParams, token: string) => {
if (!request.headers) {
request.headers = {};
let __oryKratosClient: FrontendApi | undefined;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a naming pattern we use for this? Or is there a better way? Generally I just tried to avoid creating the client multiple times.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd use one underscore (_oryKratosClient) but this is fine

Copy link
Contributor

github-actions bot commented Aug 1, 2024

Benchmark results

@rust/graph-benches – Integrations

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$1.47 \mathrm{ms} \pm 6.72 \mathrm{μs}\left({\color{gray}-0.009 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_one_depth

Function Value Mean Flame graphs
entity_by_id 5 entities $$25.5 \mathrm{ms} \pm 188 \mathrm{μs}\left({\color{gray}0.622 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$20.6 \mathrm{ms} \pm 84.9 \mathrm{μs}\left({\color{gray}-0.313 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$32.1 \mathrm{ms} \pm 149 \mathrm{μs}\left({\color{lightgreen}-40.884 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$1.59 \mathrm{s} \pm 4.08 \mathrm{ms}\left({\color{red}453 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$81.6 \mathrm{ms} \pm 435 \mathrm{μs}\left({\color{gray}2.33 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$79.7 \mathrm{ms} \pm 573 \mathrm{μs}\left({\color{gray}0.830 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$108 \mathrm{ms} \pm 645 \mathrm{μs}\left({\color{gray}-2.042 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$99.9 \mathrm{ms} \pm 616 \mathrm{μs}\left({\color{gray}-2.335 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$41.4 \mathrm{ms} \pm 378 \mathrm{μs}\left({\color{gray}0.433 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$95.5 \mathrm{ms} \pm 474 \mathrm{μs}\left({\color{gray}-2.779 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$90.9 \mathrm{ms} \pm 554 \mathrm{μs}\left({\color{gray}-0.079 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$42.8 \mathrm{ms} \pm 433 \mathrm{μs}\left({\color{gray}-0.182 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$67.1 \mathrm{ms} \pm 311 \mathrm{μs}\left({\color{gray}-0.475 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$59.2 \mathrm{ms} \pm 485 \mathrm{μs}\left({\color{gray}-0.057 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$38.2 \mathrm{ms} \pm 246 \mathrm{μs}\left({\color{gray}-0.899 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$56.2 \mathrm{ms} \pm 329 \mathrm{μs}\left({\color{gray}2.12 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$50.6 \mathrm{ms} \pm 616 \mathrm{μs}\left({\color{gray}1.47 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$16.3 \mathrm{ms} \pm 211 \mathrm{μs}\left({\color{gray}-0.752 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$17.5 \mathrm{ms} \pm 237 \mathrm{μs}\left({\color{lightgreen}-6.897 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$16.7 \mathrm{ms} \pm 230 \mathrm{μs}\left({\color{lightgreen}-7.657 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$16.9 \mathrm{ms} \pm 289 \mathrm{μs}\left({\color{gray}1.80 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$16.1 \mathrm{ms} \pm 204 \mathrm{μs}\left({\color{gray}-2.180 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$16.4 \mathrm{ms} \pm 181 \mathrm{μs}\left({\color{lightgreen}-5.223 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$16.6 \mathrm{ms} \pm 206 \mathrm{μs}\left({\color{gray}-3.134 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$16.2 \mathrm{ms} \pm 212 \mathrm{μs}\left({\color{lightgreen}-10.587 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$16.8 \mathrm{ms} \pm 230 \mathrm{μs}\left({\color{gray}2.47 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_zero_depth

Function Value Mean Flame graphs
entity_by_id 5 entities $$1.98 \mathrm{ms} \pm 7.38 \mathrm{μs}\left({\color{gray}-0.790 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.94 \mathrm{ms} \pm 9.04 \mathrm{μs}\left({\color{gray}-0.332 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$2.21 \mathrm{ms} \pm 12.0 \mathrm{μs}\left({\color{gray}-0.170 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$4.10 \mathrm{ms} \pm 22.1 \mathrm{μs}\left({\color{lightgreen}-7.267 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$3.00 \mathrm{ms} \pm 90.6 \mathrm{μs}\left({\color{gray}3.54 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$1.93 \mathrm{ms} \pm 7.34 \mathrm{μs}\left({\color{gray}0.633 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$1.98 \mathrm{ms} \pm 45.0 \mathrm{μs}\left({\color{gray}1.14 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$2.04 \mathrm{ms} \pm 7.80 \mathrm{μs}\left({\color{lightgreen}-5.565 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$2.89 \mathrm{ms} \pm 14.8 \mathrm{μs}\left({\color{gray}-3.261 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$13.8 \mathrm{ms} \pm 162 \mathrm{μs}\left({\color{gray}2.21 \mathrm{\%}}\right) $$ Flame Graph

const setAuthorizationHeader = (request: RequestParams, token: string) => {
if (!request.headers) {
request.headers = {};
let __oryKratosClient: FrontendApi | undefined;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd use one underscore (_oryKratosClient) but this is fine

@TimDiekmann TimDiekmann added this pull request to the merge queue Aug 6, 2024
Merged via the queue into main with commit 2e63704 Aug 6, 2024
148 checks passed
@TimDiekmann TimDiekmann deleted the t/h-3166-mimic-flow-behavior-in-artillery branch August 6, 2024 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/deps Relates to third-party dependencies (area) area/tests New or updated tests
Development

Successfully merging this pull request may close these issues.

2 participants