Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add mode skip build in docker image yarn build phase #1024

Merged
merged 1 commit into from
Dec 14, 2023

Conversation

iccicci
Copy link
Contributor

@iccicci iccicci commented Dec 14, 2023

Context

SDK packages are organized in a way that all the dependencies are handled at root level: i.e. all the dependencies are tracked in /yarn-project.nix and /yarn.lock and installed in /node_modules, included dependencies required only at client side and ones required only for e2e testing.

The drawback of this is that all these dependencies not required at back-end side are installed and built inside our Docker image.

Some of these dependencies need to be downloaded and rebuild. With a high throughput line, this is not a big issue, but in case of internet problems or for those who have poor lines, this can result in a huge waste of time.

Proposed Solution

Actually proposed by @mkazlauskas;

adding --mode=skip-build to yarn build step in Docker image makes the docker build time on my local env to drop from about 1200" to about 200"!

This may have some drawbacks we agreed we can tolerate:

  • potential runtime errors, if it's using libraries that require native build. We have pretty good test coverage though, so not too worried about this
  • potential performance degradation: some libraries have a fallback to js implementation if native libraries are not available

Copy link

Standard DiffPost

This PR would generate the following kubectl diff:

Preview
diff -u -N /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-backend /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-backend
--- /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-backend	2023-12-14 07:24:00.586643801 +0000
+++ /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-backend	2023-12-14 07:24:00.586643801 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "88"
   creationTimestamp: "2023-08-11T18:16:12Z"
-  generation: 88
+  generation: 89
   labels:
     app: backend
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-backend
@@ -89,7 +89,7 @@
           value: "false"
         - name: USE_KORA_LABS
           value: "true"
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:7px9fipq9m2f71dab50rm88zmdn99xq9
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:cwwbhmk0a1fkqvikmlsnvzdh6x6y3lxw
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-coingecko-proxy /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-coingecko-proxy
--- /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-12-14 07:24:00.906645220 +0000
+++ /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-12-14 07:24:00.906645220 +0000
@@ -8,7 +8,7 @@
   labels:
     app: coingecko-proxy
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-coingecko-proxy
diff -u -N /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-projector /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-projector
--- /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-projector	2023-12-14 07:24:01.222646621 +0000
+++ /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-projector	2023-12-14 07:24:01.222646621 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "88"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 88
+  generation: 89
   labels:
     app: handle-projector
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-handle-projector
@@ -73,7 +73,7 @@
               name: handle-owner-user.dev-preview-dbsync-db.credentials.postgresql.acid.zalan.do
         - name: PROJECTION_NAMES
           value: handle
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:7px9fipq9m2f71dab50rm88zmdn99xq9
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:cwwbhmk0a1fkqvikmlsnvzdh6x6y3lxw
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-provider /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-provider
--- /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-provider	2023-12-14 07:24:01.538648022 +0000
+++ /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-provider	2023-12-14 07:24:01.538648022 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "55"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 55
+  generation: 56
   labels:
     app: handle-provider
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-handle-provider
@@ -81,7 +81,7 @@
           value: handle
         - name: USE_KORA_LABS
           value: "true"
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:7px9fipq9m2f71dab50rm88zmdn99xq9
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:cwwbhmk0a1fkqvikmlsnvzdh6x6y3lxw
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-pg-boss-worker /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-pg-boss-worker
--- /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-pg-boss-worker	2023-12-14 07:24:01.854649421 +0000
+++ /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-pg-boss-worker	2023-12-14 07:24:01.854649421 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "88"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 88
+  generation: 89
   labels:
     app: pg-boss-worker
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-pg-boss-worker
@@ -95,7 +95,7 @@
           value: pool-delist-schedule,pool-metadata,pool-metrics,pool-rewards
         - name: STAKE_POOL_PROVIDER_URL
           value: http://dev-preview-cardanojs-backend.dev-preview.svc.cluster.local
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:7px9fipq9m2f71dab50rm88zmdn99xq9
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:cwwbhmk0a1fkqvikmlsnvzdh6x6y3lxw
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-projector /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-projector
--- /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-projector	2023-12-14 07:24:02.162650786 +0000
+++ /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-projector	2023-12-14 07:24:02.162650786 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "88"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 88
+  generation: 89
   labels:
     app: stake-pool-projector
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-stake-pool-projector
@@ -73,7 +73,7 @@
               name: stakepool-owner-user.dev-preview-dbsync-db.credentials.postgresql.acid.zalan.do
         - name: PROJECTION_NAMES
           value: stake-pool,stake-pool-metadata-job,stake-pool-metrics-job,stake-pool-rewards-job
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:7px9fipq9m2f71dab50rm88zmdn99xq9
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:cwwbhmk0a1fkqvikmlsnvzdh6x6y3lxw
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-provider /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-provider
--- /tmp/LIVE-3489817743/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-provider	2023-12-14 07:24:02.478652189 +0000
+++ /tmp/MERGED-2058196679/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-provider	2023-12-14 07:24:02.478652189 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "85"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 85
+  generation: 86
   labels:
     app: stake-pool-provider
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-stake-pool-provider
@@ -83,7 +83,7 @@
           value: http://dev-preview-cardano-stack-metadata.dev-preview.svc.cluster.local
         - name: USE_TYPEORM_STAKE_POOL_PROVIDER
           value: "true"
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:7px9fipq9m2f71dab50rm88zmdn99xq9
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:cwwbhmk0a1fkqvikmlsnvzdh6x6y3lxw
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-3489817743/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-backend-monitor /tmp/MERGED-2058196679/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-backend-monitor
--- /tmp/LIVE-3489817743/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-backend-monitor	2023-12-14 07:24:02.810653664 +0000
+++ /tmp/MERGED-2058196679/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-backend-monitor	2023-12-14 07:24:02.810653664 +0000
@@ -5,7 +5,7 @@
   generation: 2
   labels:
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     instance: primary
   name: lace-backend-monitor
   namespace: dev-preview
diff -u -N /tmp/LIVE-3489817743/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-handle-provider-monitor /tmp/MERGED-2058196679/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-handle-provider-monitor
--- /tmp/LIVE-3489817743/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-handle-provider-monitor	2023-12-14 07:24:03.118655035 +0000
+++ /tmp/MERGED-2058196679/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-handle-provider-monitor	2023-12-14 07:24:03.118655035 +0000
@@ -5,7 +5,7 @@
   generation: 2
   labels:
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     instance: primary
   name: lace-handle-provider-monitor
   namespace: dev-preview
diff -u -N /tmp/LIVE-3489817743/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-stake-pool-provider-monitor /tmp/MERGED-2058196679/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-stake-pool-provider-monitor
--- /tmp/LIVE-3489817743/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-stake-pool-provider-monitor	2023-12-14 07:24:03.426656404 +0000
+++ /tmp/MERGED-2058196679/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-stake-pool-provider-monitor	2023-12-14 07:24:03.426656404 +0000
@@ -5,7 +5,7 @@
   generation: 2
   labels:
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     instance: primary
   name: lace-stake-pool-provider-monitor
   namespace: dev-preview
diff -u -N /tmp/LIVE-3489817743/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-backend /tmp/MERGED-2058196679/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-backend
--- /tmp/LIVE-3489817743/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-backend	2023-12-14 07:24:03.742657811 +0000
+++ /tmp/MERGED-2058196679/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-backend	2023-12-14 07:24:03.746657828 +0000
@@ -19,7 +19,7 @@
   labels:
     app: backend
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-backend
diff -u -N /tmp/LIVE-3489817743/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-coingecko-proxy /tmp/MERGED-2058196679/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-coingecko-proxy
--- /tmp/LIVE-3489817743/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-12-14 07:24:04.058659215 +0000
+++ /tmp/MERGED-2058196679/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-12-14 07:24:04.058659215 +0000
@@ -16,7 +16,7 @@
   labels:
     app: coingecko-proxy
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-coingecko-proxy
diff -u -N /tmp/LIVE-3489817743/v1.Service.dev-preview.dev-preview-cardanojs-backend /tmp/MERGED-2058196679/v1.Service.dev-preview.dev-preview-cardanojs-backend
--- /tmp/LIVE-3489817743/v1.Service.dev-preview.dev-preview-cardanojs-backend	2023-12-14 07:23:59.318638180 +0000
+++ /tmp/MERGED-2058196679/v1.Service.dev-preview.dev-preview-cardanojs-backend	2023-12-14 07:23:59.318638180 +0000
@@ -5,7 +5,7 @@
   labels:
     app: backend
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-backend
diff -u -N /tmp/LIVE-3489817743/v1.Service.dev-preview.dev-preview-cardanojs-coingecko-proxy /tmp/MERGED-2058196679/v1.Service.dev-preview.dev-preview-cardanojs-coingecko-proxy
--- /tmp/LIVE-3489817743/v1.Service.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-12-14 07:23:59.618639510 +0000
+++ /tmp/MERGED-2058196679/v1.Service.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-12-14 07:23:59.622639527 +0000
@@ -5,7 +5,7 @@
   labels:
     app: coingecko-proxy
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-coingecko-proxy
diff -u -N /tmp/LIVE-3489817743/v1.Service.dev-preview.dev-preview-cardanojs-handle-provider /tmp/MERGED-2058196679/v1.Service.dev-preview.dev-preview-cardanojs-handle-provider
--- /tmp/LIVE-3489817743/v1.Service.dev-preview.dev-preview-cardanojs-handle-provider	2023-12-14 07:23:59.930640891 +0000
+++ /tmp/MERGED-2058196679/v1.Service.dev-preview.dev-preview-cardanojs-handle-provider	2023-12-14 07:23:59.930640891 +0000
@@ -5,7 +5,7 @@
   labels:
     app: handle-provider
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-handle-provider
diff -u -N /tmp/LIVE-3489817743/v1.Service.dev-preview.dev-preview-cardanojs-stake-pool-provider /tmp/MERGED-2058196679/v1.Service.dev-preview.dev-preview-cardanojs-stake-pool-provider
--- /tmp/LIVE-3489817743/v1.Service.dev-preview.dev-preview-cardanojs-stake-pool-provider	2023-12-14 07:24:00.250642311 +0000
+++ /tmp/MERGED-2058196679/v1.Service.dev-preview.dev-preview-cardanojs-stake-pool-provider	2023-12-14 07:24:00.250642311 +0000
@@ -5,7 +5,7 @@
   labels:
     app: stake-pool-provider
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: 7fe1e6d1edd2308c80bbb339168b5565d7fd1125
+    app.kubernetes.io/version: fb752e6a2ef4f632af3c36cda22bc228e5c2072c
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-stake-pool-provider

@iccicci iccicci marked this pull request as ready for review December 14, 2023 15:37
Copy link
Contributor

@mchappell mchappell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems ok to me, nice write-up. Can we document this somewhere for later reference?

@iccicci iccicci merged commit 837be7c into master Dec 14, 2023
8 checks passed
@iccicci iccicci deleted the chore/docker-build-speed-up branch December 14, 2023 16:02
iccicci added a commit that referenced this pull request Dec 14, 2023
iccicci added a commit that referenced this pull request Dec 15, 2023
iccicci added a commit that referenced this pull request Dec 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants