他说的“你的第一本”，指的是楼主书单里的第一本书：Jorge Nocedal 和 Stephen J. Wright 写的《Numerical Optimization》。读起来相当有意思。
He said "your #1", making reference to the first book in OPs list: "Numerical Optimization" by Jorge Nocedal and Stephen J. Wright. It's pretty fun to read.
……Paul https://www.springer.com/us/book/9780387922799 关于凸规划的建模技巧，可以看 Stephen Boyd 和 Lieven Vandenberghe 的《Convex Optimization》https://stanford.edu/~boyd/cvxbook/ 基础非线性优化算法，看 Nocedal, Jorge, Wright, S. 的《Numerical Optimization》https://www.springer.com/us/book/9780387303031 我其实从没找到过终极的优化宝典，但上面这些参考书往往能给出正确的建模技巧……
…Paul https://www.springer.com/us/book/9780387922799 For modeling tricks for convex programming, Convex Optimization by Stephen Boyd and Lieven Vandenberghe https://stanford.edu/~boyd/cvxbook/ For basic nonlinear optimization algorithms, Numerical Optimization by Nocedal, Jorge, Wright, S. https://www.springer.com/us/book/9780387303031 I've never really found the ultimate optimization book, but the references above tend to have the right modeling techniques,…
Nocedal's text, 'Numerical Optimization' is the standard for that field. As he notes, I've always been surprised that more techniques in ML do not leverage the Hessian to get quadratic convergence rates. Nevertheless, the most interesting…
…infimum. The background necessary to understand this book is real analysis and as a quick aside I think Rudin's Principles of Mathematical Analysis is the best for this. Second, is Nocedal and Wright's Numerical Optimization book. It provides a good overview of the powerful algorithms in optimization that we should be using. Now, it's weakness is that it often cheats and uses a stronger constraint qualification than we're…
……Linnainmaa BP 算法的实现细节，但通过类似雅可比-向量积之类的东西，对纯反向模式自动微分有各种各样的改进，能节省海量的内存。Nocedal 的《Numerical Optimization》涵盖了大量关于如何实现自动微分的内容，我可以毫不犹豫地向任何感兴趣的人推荐它。[1] https://www.reddit.com/r/MachineLearning/comments/e5vzun/d_j...
…implementation details of Linnainmaa's BP, but there are all sorts of improvements to plain reverse mode auto-grad through stuff like Jacobian-Vector products that save an enormous amount of memory. Nocedal's "Numerical Optimization" covers a great deal on how to implement autograd, and I can easily recommend it to anyone interested. [1] https://www.reddit.com/r/MachineLearning/comments/e5vzun/d_j...
…Recipes offers very quickly. There are some good books in the list you linked, but that list is also very outdated. Notably absent: any of Trefethen's books (ATAP in particular), Nocedal and Wright's "Numerical Optimization", Stoer & Bulirsch, anything about convex optimization, anything about the finite element method, anything about geometry or CAD, anything about Monte Carlo, anything about special functions, and on and…
他说的“你的第一本”，指的是楼主书单里的第一本书：Jorge Nocedal 和 Stephen J. Wright 写的《Numerical Optimization》。读起来相当有意思。
He said "your #1", making reference to the first book in OPs list: "Numerical Optimization" by Jorge Nocedal and Stephen J. Wright. It's pretty fun to read.
……Paul https://www.springer.com/us/book/9780387922799 关于凸规划的建模技巧，可以看 Stephen Boyd 和 Lieven Vandenberghe 的《Convex Optimization》https://stanford.edu/~boyd/cvxbook/ 基础非线性优化算法，看 Nocedal, Jorge, Wright, S. 的《Numerical Optimization》https://www.springer.com/us/book/9780387303031 我其实从没找到过终极的优化宝典，但上面这些参考书往往能给出正确的建模技巧……
…Paul https://www.springer.com/us/book/9780387922799 For modeling tricks for convex programming, Convex Optimization by Stephen Boyd and Lieven Vandenberghe https://stanford.edu/~boyd/cvxbook/ For basic nonlinear optimization algorithms, Numerical Optimization by Nocedal, Jorge, Wright, S. https://www.springer.com/us/book/9780387303031 I've never really found the ultimate optimization book, but the references above tend to have the right modeling techniques,…
Nocedal 的著作《Numerical Optimization》是该领域的标杆。正如他指出的那样，我一直很惊讶为什么机器学习中没有更多技术去利用 Hessian 矩阵来获得二次收敛速度。不过，最有趣的……
Nocedal's text, 'Numerical Optimization' is the standard for that field. As he notes, I've always been surprised that more techniques in ML do not leverage the Hessian to get quadratic convergence rates. Nevertheless, the most interesting…
……下确界。读懂这本书所需的背景知识是实分析，顺便提一句，我认为 Rudin 的《数学分析原理》是学这个最好的书。第二本是 Nocedal 和 Wright 的《Numerical Optimization》。它很好地概述了我们应该使用的那些强大的优化算法。不过，它的缺点是经常耍赖，使用了比我们实际……更强的约束规整条件。
…infimum. The background necessary to understand this book is real analysis and as a quick aside I think Rudin's Principles of Mathematical Analysis is the best for this. Second, is Nocedal and Wright's Numerical Optimization book. It provides a good overview of the powerful algorithms in optimization that we should be using. Now, it's weakness is that it often cheats and uses a stronger constraint qualification than we're…
……Linnainmaa BP 算法的实现细节，但通过类似雅可比-向量积之类的东西，对纯反向模式自动微分有各种各样的改进，能节省海量的内存。Nocedal 的《Numerical Optimization》涵盖了大量关于如何实现自动微分的内容，我可以毫不犹豫地向任何感兴趣的人推荐它。[1] https://www.reddit.com/r/MachineLearning/comments/e5vzun/d_j...
…implementation details of Linnainmaa's BP, but there are all sorts of improvements to plain reverse mode auto-grad through stuff like Jacobian-Vector products that save an enormous amount of memory. Nocedal's "Numerical Optimization" covers a great deal on how to implement autograd, and I can easily recommend it to anyone interested. [1] https://www.reddit.com/r/MachineLearning/comments/e5vzun/d_j...
……《Numerical Recipes》很快就能提供的内容。你链接的列表里有一些好书，但那个列表也已经严重过时了。明显缺席的有：Trefethen 的任何书（尤其是 ATAP）、Nocedal 和 Wright 的《Numerical Optimization》、Stoer & Bulirsch、任何关于凸优化的书、任何关于有限元方法的书、任何关于几何或 CAD 的书、任何关于蒙特卡洛的书、任何关于特殊函数的书，等等等等……
…Recipes offers very quickly. There are some good books in the list you linked, but that list is also very outdated. Notably absent: any of Trefethen's books (ATAP in particular), Nocedal and Wright's "Numerical Optimization", Stoer & Bulirsch, anything about convex optimization, anything about the finite element method, anything about geometry or CAD, anything about Monte Carlo, anything about special functions, and on and…