Large language models (LLMs) excel at various tasks, but strategic planning, like mastering a board game, remains a challenge. This research explores how LLMs can be trained for both external and internal planning to improve their strategic reasoning skills, using board games as a testing ground. Researchers introduced a novel Transformer model, the "multi action-value (MAV)" model, pre-trained on game data for chess, Chess960, Connect Four, and Hex. This model acts as a world model, value function, and policy function simultaneously, predicting the game state, legal moves, and win probabilities. Two planning approaches were tested: external and internal search. External search uses the MAV model to guide Monte Carlo Tree Search (MCTS) without external game engines. This approach proved remarkably effective, even with a limited number of simulations, achieving Grandmaster-level performance in chess with a similar search budget to human players. Internal search distills the search process directly into the LLM by training it on linearized search trees. This approach allows the model to execute a search procedure within a single model call, showing performance improvements that scale with the search budget. Results showed significant improvements in gameplay across all tested games using both external and internal search methods. The MAV model demonstrated accurate state tracking and legal move prediction, minimizing hallucinations. External MCTS significantly boosted playing strength, even with low simulation counts. Internal search further enhanced performance by allowing the model to self-correct and explore multiple lines of reasoning. This research offers exciting insights into how search-based planning can enhance LLM reasoning abilities, suggesting potential applications in more complex real-world decision-making scenarios. However, the methods still rely on extensive game data and strong game engines for training, posing challenges for broader applications. Further research will explore more complex search procedures and integrate natural language communication capabilities into these specialized models, potentially paving the way for more versatile and powerful LLM agents.