Timeline for Can I minimize a mysterious function by running a gradient decent on her neural net approximations? [closed]
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 23, 2023 at 18:11 | comment | added | Red shoes | The problem is that if $d$ is a descent direction for NN, it may be an ascent direction for $F$. | |
| Sep 27, 2022 at 15:45 | comment | added | Geordie Williamson | Hi Vladimir, I just wanted to comment that the proof will be in the pudding. It's a pretty cheap thing to try, and you should just try to do it! I have tried this on several problems where it did horribly, and presumably it does well on others. The most crucial thing will be trying to have some understanding of whether the neural net will be a reasonable approximation of your function. This depends a lot on what kind of function you are trying to optimize. | |
| Sep 23, 2022 at 16:52 | history | closed |
Yoav Kallus Stefan Kohl♦ |
Not suitable for this site | |
| Sep 23, 2022 at 16:28 | comment | added | D.W. | Cross-posted: ai.stackexchange.com/q/37166/1794, mathoverflow.net/q/431052/37212, cs.stackexchange.com/q/154308/755. Please do not post the same question on multiple sites. | |
| Sep 23, 2022 at 2:48 | comment | added | Vladimir Zolotov | @YoavKallus Thanks for the link. The thing I'm looking for is definitely a gradient-free optimization method. But the thing is that if you plug a tricky function in one of methods from wiki it will probably just never find anything. I mean I assume that ppl from machine learning are using neural nets for a reason another that they are unaware of Nelder-Mead routine. | |
| Sep 23, 2022 at 2:11 | review | Close votes | |||
| Sep 23, 2022 at 17:07 | |||||
| Sep 23, 2022 at 1:58 | comment | added | Yoav Kallus | This is not the appropriate venue for this question, but I believe what you're looking for is just a gradient-free optimization method? Many numerical libraries implement one or more such methods (e.g. scipy has a Nelder-Mead routine). Maybe start here: en.wikipedia.org/wiki/Derivative-free_optimization | |
| Sep 23, 2022 at 1:26 | history | asked | Vladimir Zolotov | CC BY-SA 4.0 |