Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Working GCP in gateway #58

Merged
merged 1 commit into from
Jan 14, 2022
Merged

Working GCP in gateway #58

merged 1 commit into from
Jan 14, 2022

Conversation

parasj
Copy link
Contributor

@parasj parasj commented Jan 14, 2022

A very simple PR that enables GCP support for the Gateway

To test:

$ source scripts/pack_docker.sh; skylark replicate-random aws:ap-northeast-1 gcp:us-central1-a --chunk-size-mb 8 --n-chunks 1024 --num-gateways 1 --num-outgoing-connections 64

=================================================
  ______  _             _                 _
 / _____)| |           | |               | |
( (____  | |  _  _   _ | |  _____   ____ | |  _
 \____ \ | |_/ )| | | || | (____ | / ___)| |_/ )
 _____) )|  _ ( | |_| || | / ___ || |    |  _ (
(______/ |_| \_) \__  | \_)\_____||_|    |_| \_)
                (____/
=================================================

            __exit__:24  | Cloud SSH key initialization: 6.46s
    replicate_random:118 | Instances will remain up and may result in continued cloud billing. Remember to call `skylark deprovision` to deprovision gateways.
      wait_for_ready:138 | Waiting for skylark-333700:gcp:us-central1-a:skylark-gcp-3e2f0698a55b478b9bb14208516cedf3 to be ready
      wait_for_ready:138 | Waiting for aws:ap-northeast-1:i-0e2d9e5b41994c304 to be ready
      wait_for_ready:140 | skylark-333700:gcp:us-central1-a:skylark-gcp-3e2f0698a55b478b9bb14208516cedf3 is ready
      wait_for_ready:140 | aws:ap-northeast-1:i-0e2d9e5b41994c304 is ready
            __exit__:24  | Provisioning instances and waiting to boot: 58.73s
       start_gateway:184 | Starting gateway aws:ap-northeast-1:i-0e2d9e5b41994c304: Installing docker
       start_gateway:184 | Starting gateway skylark-333700:gcp:us-central1-a:skylark-gcp-3e2f0698a55b478b9bb14208516cedf3: Installing docker
       start_gateway:208 | Starting gateway aws:ap-northeast-1:i-0e2d9e5b41994c304: Starting monitoring
       start_gateway:208 | Starting gateway skylark-333700:gcp:us-central1-a:skylark-gcp-3e2f0698a55b478b9bb14208516cedf3: Starting monitoring
       start_gateway:213 | Starting gateway skylark-333700:gcp:us-central1-a:skylark-gcp-3e2f0698a55b478b9bb14208516cedf3: Pulling docker image
       start_gateway:218 | Starting gateway skylark-333700:gcp:us-central1-a:skylark-gcp-3e2f0698a55b478b9bb14208516cedf3: Starting gateway container ghcr.io/parasj/skylark:local-4c0221b9868e0ee1adc4e9cbda577f7f
^[[C^[[C^[[       start_gateway:213 | Starting gateway aws:ap-northeast-1:i-0e2d9e5b41994c304: Pulling docker image
       start_gateway:218 | Starting gateway aws:ap-northeast-1:i-0e2d9e5b41994c304: Starting gateway container ghcr.io/parasj/skylark:local-4c0221b9868e0ee1adc4e9cbda577f7f
            __exit__:24  | Install gateway package on instances: 91.66s
    replicate_random:126 | Provisioned path aws:ap-northeast-1 -> gcp:us-central1-a
    replicate_random:128 | 	[aws:ap-northeast-1] http://13.115.242.234:8888/container/f5a3baccbe3f
    replicate_random:128 | 	[gcp:us-central1-a] http://35.206.81.151:8888/container/88ca16d2bad1
            __exit__:24  | Building chunk requests: 0.01s
 send_chunk_requests:241 | Sending 1024 chunk requests to 13.115.242.234
            __exit__:24  | Dispatch chunk requests: 2.19s
    replicate_random:141 | 8.00GByte replication job launched
    monitor_transfer:286 | Web dashboard running at http://54.92.213.136:8080
    monitor_transfer:330 | 35/1024 chunks done (0.27 / 8.00GB, 0.43Gbit/s, ETA=144s)
    monitor_transfer:330 | 285/1024 chunks done (2.23 / 8.00GB, 0.81Gbit/s, ETA=56s)

{"total_runtime_s": 23.721418, "throughput_gbits": 2.697983737734397, "monitor_status": "completed", "success": true}

@parasj parasj requested a review from samkumar January 14, 2022 08:20
@parasj parasj self-assigned this Jan 14, 2022
@parasj parasj added the gateway label Jan 14, 2022
Copy link
Contributor

@samkumar samkumar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Feel free to merge this.

@parasj parasj merged commit b7ec2d8 into main Jan 14, 2022
@parasj parasj deleted the dev/paras/gcp-gateway branch January 14, 2022 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants