V ( s t ) V(s_t)~ V ( s t ​ ) ↩ \hookleftarrow~ ↩ r t r_{t} r t ​ + + + γ V ( s t + 1 ) \gamma V(s_{t+1}) γ V ( s t + 1 ​ ) ↩ \hookleftarrow~ ↩ r t r_{t} r t ​ + + + γ E [ r t + 1 ′ ] \gamma \mathop{\mathbb{E}} \bigr[ r’_{t+1} \bigl] γ E [ r t + 1 ′ ​ ] + + + γ 2 E [ V ( s t + 2 ′ ′ ) ] \gamma^2 \mathop{\mathbb{E}} \bigr[ V(s’’_{t+2}) \bigl] γ 2 E [ V ( s t + 2 ′ ′ ​ ) ] ↩ \hookleftarrow~ ↩ r t r_{t} r t ​ + + + γ E [ r t + 1 ′ ] \gamma \mathop{\mathbb{E}} ~ \bigr[ r’_{t+1} \bigl] γ E [ r t + 1 ′ ​ ] + + + γ 2 E E [ r t + 2 ′ ′ ] \gamma^2 \mathop{\mathbb{EE}} ~ \bigr[ r’’_{t+2} \bigl] γ 2 E E [ r t + 2 ′ ′ ​ ] + +~ + .