Binary Hopfield net using Hebbian learning We want to study Hopfield net from the simple case. Hopfield net is a fully connected feedback network. A feedback network is a network that is not a feedforward network, and in a feedforward network, all the connections are directed. All the connections in our example will be bi-directed. This symmetric property of the weight is important property of the Hopfield net. Hopfield net can act as associative memories, and they can be used to solve optimization problems.

Continue reading

Why Ising model : 3 reasons for relevance Studying Ising model can be useful to understand phase transition of various systems. Hopfield network or Boltzmann machine to the neural network is just a generalized form of Ising model. Ising model is also useful as a statistical model in its own right. Ising model $\boldsymbol{x}$ is the state of an Ising model with $N$ spins be a vector in which each component $\boldsymbol x_n$ takes values $-1$ or $+1$.

Continue reading

Atom Git and Github have become one of the essential part of programmers. I just mechanically commit just after I finish a code or change some setting, and it saved me so many times. It also provides great environment for teamwork and management of the projects. This Jekyll blog is also built upon the Github service. Atom is the text editor developed by Github. And this is great. When I made up my mind to transit to data science.

Continue reading

Single neuron still has a lot to say In the post of the first neural network tutorial, we studied a perceptron as a simple supervised learning machine. The perceptron is an amazing structure to understanding inference. In the post of the first neural network tutorial, I said I would leave you to find the objective function and and draw the plot of it. I just introduce here. Objective function and its contour plot.

Continue reading

Slice sampling algorithm A single transition $(x,u) \rightarrow (x',u')$ of a one-dimensional slice sampling algorithm has the following steps. (1). evaluate $P^* (x)$ (2). draw a vertical coordinate $u' \sim$ Uniform$(0,P^* (x))$ (3). create a horizontal interval $(x_l, x_r)$ enclosing $x$  3a. draw $r \sim$ Uniform$(0,1)$  3b. $x_l := x-rw$  3c. $x_r := x+(1-r)w$  3d. while $(P^* (x_l) > u')$ ${x_l := x-rw}$  3e. while $(P^* (x_r) > u')$ ${x_r:= x+w}$

Continue reading