rnn attention memory The RNN gives an attention distribution which describe how we spread out the amount we care about different memory positions. rnn attention old memory new memory write value The RNN gives an attention distribution, describing how much we should change each memory position towards the write value. interpolation amount shift filter RNN controller attention mechanism query vector attention from previous step new attention distribution