Mark who I met in machine learning study meetup had recommended me to study a research paper about discrete variational autoencoder. I have read today. As so does variational inference, it includes many mathematical equations, but what the author wants to tell was very straightforward. Two previous posts, Variational Method, Independent Component Analysis, are relevant to the following discussion. Autoencoder To understand the paper, above all, we need to know what the autoencoder is and what variational autoencoder is.

Continue reading

Generative model Generative model is a model for generating all variables including outputs. I will give a very simple example with strong assumptions. Data $\boldsymbol{x^{(n)} } $ are generated by an unknown matrix, $\boldsymbol{G}$. $$ \boldsymbol{x} = \boldsymbol{G}~\boldsymbol{s} $$ The goal is to find the source variable $\boldsymbol{s}$. we assume that the number of sources is equal to the number of observations We assume that the latent variables are independently distributed, with marginal distributions We assume that the vector $\boldsymbol{x}$ is generated without noise for simplicity.

Continue reading