Skip to content

Commit

Permalink
doc: doc update
Browse files Browse the repository at this point in the history
  • Loading branch information
Dengyu-Wu committed Jun 28, 2024
1 parent bf514af commit 214dfb9
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions docs/cutoff/topk.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,35 @@
## Optimal Cutoff Timestep (OCT)

$$
\begin{align}
g(\boldsymbol{X}) = \arg\min_{\hat{t}}\{ \forall \hat{t}_1 > \hat{t}: \mathbf{1}(f(\boldsymbol{X}[\hat{t}_1])= \boldsymbol{y})\}
\end{align}
$$

## Top-K Gap for Cutoff Approximation

The defination of $Top_k(\boldsymbol{Y}(t))$ as the top-$k$ output occurring in one neuron of the output layer,

$$
\begin{align}
Y_{gap}= Top_1(\boldsymbol{Y}(t)) - Top_2(\boldsymbol{Y}(t)),
\end{align}
$$

which denotes the gap of top-1 and top-2 values of output $\boldsymbol{Y}(t)$. Then, we let $ D\{\cdot\}$ denote the inputs in subset of $D$ that satisfy a certain condition. Now, we can define the confidence rate as follows:

$$
\begin{align}
\textit{Confidence rate: } C(\hat{t}, D\{Y_{gap}>\beta\}) = \frac{1}{|D\{Y_{gap}>\beta\}|}\sum_{\boldsymbol{X}\in D\{Y_{gap}>\beta\}} (g(\boldsymbol{X}) \leq \hat{t}),
\end{align}
$$

The algorithm searches for a minimum $\beta \in \mathbb{R^+}$ at a specific $\hat t$, as expressed in the following optimization objective:

$$
\begin{align}
\arg\min_{\beta} C(\hat t, D\{Y_{gap} > \beta\}) \geq 1-\epsilon,
\end{align}
$$

where $\epsilon$ is a pre-specified constant such that $1-\epsilon$ represents an acceptable level of confidence for activating cutoff, and a set of $\beta$ is extracted under different $\hat t$ using training samples.

0 comments on commit 214dfb9

Please sign in to comment.