Confusing Vol. 2’s “value iteration with approximation” with standard neural network training. Fix: Remember – you are approximating expected future cost , not just mapping states to actions. Always keep the Bellman equation in mind.
In the neon-drenched corridors of Neo-Kyoto, Elias Thorne wasn’t just a coder; he was a "path-optimizer." While others fought for scraps in the physical world, Elias lived in the high-stakes realm of stochastic systems dynamic programming and optimal control vol 2 pdf
| You are... | Will Vol. 2 help? | |------------|-------------------| | Graduate student in controls/OR | ✅ Essential for research | | ML engineer working on RL | ✅ Yes – provides theory behind Sutton & Barto | | Practitioner with huge state space (e.g., inventory, robotics) | ✅ Yes – focuses on approximations | | Beginner looking for light reading | ❌ No – start with Vol. 1 or Sutton & Barto | Confusing Vol
Absolutely, but with a caveat.
Getting lost in convergence proofs. Fix: Flag those sections for later. First understand what the algorithm computes, then why it might converge. Always keep the Bellman equation in mind