Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New console command: random seeder #458

Merged
merged 7 commits into from
Feb 6, 2024

Conversation

josecelano
Copy link
Member

@josecelano josecelano commented Feb 6, 2024

It will upload random torrents to the Index. For testing purposes. It could be used to seed E2E test environments too.

  • Reorganize console mod to include more mods.
  • Move API client from testing to production.
  • Create scaffolding for a new console command: seeder(with clap and anyhow)
  • Use the API client to seed a live env with random torrents.

The console command is:

cargo run --bin seeder -- --api-base-url <API_BASE_URL> --number-of-torrents <NUMBER_OF_TORRENTS> --user <USER> --password <PASSWORD> --interval <INTERVAL>

For example:

cargo run --bin seeder -- --api-base-url "localhost:3001" --number-of-torrents 1000 --user admin --password 12345678 --interval 0

That command would upload 1000 random torrents to the Index using the user account admin with password 123456 and wait 1 second between uploads.

Output:

$ cargo run --bin seeder -- --api-base-url "localhost:3001" --number-of-torrents    3 --user admin --password 12345678 --interval 5
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/seeder --api-base-url 'localhost:3001' --number-of-torrents 3 --user admin --password 12345678 --interval 5`
2024-02-06T17:12:23.325055202+00:00 [seeder][INFO] Trying to login with username: admin ...
2024-02-06T17:12:23.666824220+00:00 [seeder][INFO] Logged as admin with account: admin 
2024-02-06T17:12:23.666870390+00:00 [seeder][INFO] Uploading 3 random torrents to the Torrust Index ...
2024-02-06T17:12:23.666875190+00:00 [seeder][INFO] Uploading torrent #1 ...
2024-02-06T17:12:23.666886180+00:00 [seeder][INFO] Uploading torrent with uuid: c6a4752a-d27c-4ba9-a367-cf41c9f22d4a ...
2024-02-06T17:12:23.735547434+00:00 [seeder][INFO] Uploaded torrent: {"torrent_id":54,"info_hash":"7e0a8e0461bd2fd46b55c95da95721c836ae9349"}
2024-02-06T17:12:28.735666148+00:00 [seeder][INFO] Uploading torrent #2 ...
2024-02-06T17:12:28.735738597+00:00 [seeder][INFO] Uploading torrent with uuid: a383519c-0018-43ec-82df-10044c61d46d ...
2024-02-06T17:12:28.793988941+00:00 [seeder][INFO] Uploaded torrent: {"torrent_id":55,"info_hash":"12f0f9edd2494d56a36c13b833aee37d1a0bb84d"}
2024-02-06T17:12:33.794118199+00:00 [seeder][INFO] Uploading torrent #3 ...
2024-02-06T17:12:33.794209398+00:00 [seeder][INFO] Uploading torrent with uuid: bc32812c-aab2-40b7-999e-d58c4070655c ...
2024-02-06T17:12:33.874166798+00:00 [seeder][INFO] Uploaded torrent: {"torrent_id":56,"info_hash":"4b9f6e1b2e4ddcb25b292ba0cb8561e2daa5b243"}

```console
src/console/
├── commands
│   ├── mod.rs
│   └── tracker_statistics_importer
│       ├── app.rs
│       └── mod.rs
├── cronjobs
│   ├── mod.rs
│   └── tracker_statistics_importer.rs
└── mod.rs
```

- We will create a new command to seed the Index with random torrents.
- This new strcuture is similar to what we have in the Tracker.
@josecelano josecelano added Testing Checking Torrust - Admin - Enjoyable to Install and Setup our Software labels Feb 6, 2024
@josecelano josecelano added this to the v3.0.0 milestone Feb 6, 2024
@josecelano josecelano linked an issue Feb 6, 2024 that may be closed by this pull request
Added `server` subfolder. We will add a client folder.
@josecelano
Copy link
Member Author

Hi @da2ce7 I'm going to clone the API client from testing code to production. I would keep the same code in tests. I do not want to couple tests to production code. In the future, we could:

  • Extract a common package that both the server and the client depend on with common types (requests and responses). We can call it torrust-index-api-contract.
  • Extract a server package (torrust-api-server) that depends on the torrust-index-api-contract package.
  • Extract a client package (torrust-api-client) that depends on the torrust-index-api-contract package.
  • Tests in the server can use the torrust-api-client package.

We can refactor this in the next milestone.

It will be use in a new console command (seeder).
New console command to upload torrent to the Index remotely by using the
API client.

```console
cargo run --bin seeder -- --api-base-url <API_BASE_URL> --number-of-torrents <NUMBER_OF_TORRENTS> --user <USER> --password <PASSWORD> --interval <INTERVAL>
```

For example:

```console
cargo run --bin seeder -- --api-base-url "localhost:3001" --number-of-torrents 1000 --user admin --password 12345678 --interval 0
```

That command would upload 1000 random torrents to the Index using the user
account `admin` with password `123456` and waiting `1` second between uploads.
@josecelano
Copy link
Member Author

ACK 324fac7

@josecelano josecelano merged commit 663bd6e into torrust:develop Feb 6, 2024
8 of 11 checks passed
@josecelano josecelano deleted the 453-load-test branch February 6, 2024 17:45
@josecelano josecelano self-assigned this Feb 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
- Admin - Enjoyable to Install and Setup our Software Testing Checking Torrust
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

New console command to upload random torrents to the Index: seeder
1 participant