2023-10-19

Accretion Net

Suppose $\phi^* $ and $\theta^* $ of any converged encoder-decoder $A^*(x) = x \mapsto z \sim q_{\phi^{ * }}(z|x) \mapsto \hat{x} \sim p_{\theta^{ *}}(x|z)$ have captured important features of $x$.

Given sufficiently large gating function $g_{\omega}$ with sufficiently optimized $\omega$, we can reuse features of $A^*$ in any untrained $A$:

$$ x \mapsto z \sim q_{\phi}(z|x) $$

$$ z \mapsto z’ \sim A^*(g_{\omega}(z)) $$

$g_{\omega}(z)$ converges to interpretable $A^*$ input

$$ z’ \mapsto \hat{x} \sim p_{\theta}(x|z’) $$

and finally

$$ A^* = A $$

It’s important that $g$ is general enough to be able to call $n$ number of $A^*$’s $m$ times where $n, m \geq 1$.