FAQs
Here we put some questions that are frequently mentioned by our users/contributors.
Q: Should we include failure deployments when calculating DF?
A: No. DF is supposed to measure the frequency we ship value to the target environments. A failure deployment does not ship value to the target environments hence should not be considered.
Q: Should we include "empty builds" (i.e. the builds which include no new commits) when calculating DF?
A: Guess a better approach is including the empty builds when calculating DF, but ignore it Lead Time.
Q: Is there a better way to calculate MLT instead of calculate the mean value of multiple deployments? Shouldn't we consider the weight of different deployments due to the various batch size?
A: Correct. Batch size need to be considered. Refer to the latest algorithm to see how it works in our system.
Q: How should we compute "delayed deployments"? Does "delayed deployments" result in negative impact to the MLT value?
A: When compute MLT value, commits are selected base on the time range in which the deployment occurs instead of the build trigger.
i.e. When deployment occurs, compute its lead time. Otherwise no.
There's one special case though - imagine we have two deployments, where the deployment#1 happens before deployment #2. However if we trace back to their build time respective, It is possible though the time when build#1 occurs is actually after build#2.
Q: Is "an aborted build" a failure build? Should it be included in CFR?
A: No. Since for each failure we would also want to calculate its TTR value. An aborted build doesn't necessarily result in a failure to which team wants to apply a fix.
Q: When calculating CFR, except for the failure execution to target environment, should we also calculate the ones failed at preceding stages? For example, failed at build stage.
A: No.
Because it doesn't sound reasonable for counting an execution as failure when it did not even attempt to deploy to the target environment. On the other hand, technically it is also cumbersome to identify the order of different stages.
Q: For consecutive failures, do we measure the TTR time for all of them, or the first failure only?
A: For consecutive failures, we measure the TTR value for first failure only.