Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self-play
Por um escritor misterioso
Last updated 19 outubro 2024
Results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Recently, AlphaZero has achieved outstanding performance in playing Go, Chess, and Shogi. Players in AlphaZero consist of a combination of Monte Carlo Tree Search and a Deep Q-network, that is trained using self-play. The unified Deep Q-network has a policy-head and a value-head. In AlphaZero, during training, the optimization minimizes the sum of the policy loss and the value loss. However, it is not clear if and under which circumstances other formulations of the objective function are better. Therefore, in this paper, we perform experiments with combinations of these two optimization targets. Self-play is a computationally intensive method. By using small games, we are able to perform multiple test cases. We use a light-weight open source reimplementation of AlphaZero on two different games. We investigate optimizing the two targets independently, and also try different combinations (sum and product). Our results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Moreover, we find that care must be taken in computing the playing strength. Tournament Elo ratings differ from training Elo ratings—training Elo ratings, though cheap to compute and frequently reported, can be misleading and may lead to bias. It is currently not clear how these results transfer to more complex games and if there is a phase transition between our setting and the AlphaZero application to Go where the sum is seemingly the better choice.
Strength and accuracy of policy and value networks. a Plot showing
Policy or Value ? Loss Function and Playing Strength in AlphaZero
reference request - How do neural networks play chess
Does the neural net of AlphaZero only evaluate the score of a
AlphaGo Zero – How and Why it Works – Tim Wheeler
Reimagining Chess with AlphaZero, February 2022
Policy or Value ? Loss Function and Playing Strength in AlphaZero
AlphaZero Explained · On AI
AlphaGo Zero – How and Why it Works – Tim Wheeler
AlphaDDA: strategies for adjusting the playing strength of a fully
Reinforcement learning is all you need, for next generation
Acquisition of chess knowledge in AlphaZero
PDF) Brick Tic-Tac-Toe: Exploring the Generalizability of
MuZero Intuition
Recomendado para você
-
Acquisition of Chess Knowledge in AlphaZero19 outubro 2024
-
Acquisition of chess knowledge in AlphaZero19 outubro 2024
-
Chess's New Best Player Is A Fearless, Swashbuckling Algorithm19 outubro 2024
-
Mastering the game of Go without human knowledge19 outubro 2024
-
DeepMind's game-playing AI just beat 50-year-old record in computer science19 outubro 2024
-
GitHub - timvvvht/AlphaZero-Connect4: An asynchronous implementation of AlphaZero, a self-play reinforcement learning algorithm.19 outubro 2024
-
AlphaZero-Inspired Game Learning: Faster Training by Using MCTS Only at Test Time19 outubro 2024
-
🔵 AlphaZero Plays Connect 419 outubro 2024
-
Diversifying AI: Towards Creative Chess with AlphaZero19 outubro 2024
-
Training AlphaZero for 700,000 steps. Elo ratings were computed from19 outubro 2024
você pode gostar
-
One Piece Episode 333 Recap19 outubro 2024
-
Skovyboi Survivor Lyrics19 outubro 2024
-
HOW DOES JUSTIN BIEBER KNOW THE LYRICS TO LOSE YOURSELF BY EM⁉️👀✓ IS19 outubro 2024
-
Live Laugh Ligma Sticker for Sale by OnlyTheBest4U19 outubro 2024
-
Pin em calender19 outubro 2024
-
God of War Ragnarok's first few hours made me laugh and cry19 outubro 2024
-
Wild Hearts receberá patch com correção de desempenho na próxima19 outubro 2024
-
Alone-Alan Walker (tradução)19 outubro 2024
-
Call of Duty World War 2 (COD WW2) Chain Smoker Trophy / Achievement19 outubro 2024
-
Build Our Machine: REMAKE - Comic Studio19 outubro 2024