Imagine a world where AI doesn't just follow instructions, but designs its own strategies to solve complex problems. This isn't science fiction—it's the exciting potential of automated heuristic design (AHD). Traditionally, crafting effective heuristics required deep domain expertise, a time-consuming manual process. But what if we could automate this? Recent research explores the use of large language models (LLMs), like those powering ChatGPT, to evolve their own problem-solving heuristics through a technique called Evolutionary Program Search (EPS). Researchers are investigating whether LLMs, when coupled with search strategies inspired by biological evolution, can truly discover novel and efficient problem-solving approaches. Early results suggest the simple act of prompting an LLM repeatedly isn't enough. While the performance of standalone LLMs improves with a greater number of prompts (a larger "query budget"), there's a limit to what they can achieve on their own. The true potential seems to lie in combining the generative power of LLMs with structured search. Imagine the LLM proposing different solution strategies (heuristics represented as computer programs), and an evolutionary algorithm selecting and refining the most promising ones, generation after generation. The study tested this approach across various problems like the Admissible Set problem (a complex mathematical puzzle), Online Bin Packing (optimizing how to fit items into bins), and the classic Traveling Salesman Problem (finding the shortest route). The results are intriguing. While no single LLM or EPS method reigns supreme across all problems, the combination of LLMs and evolutionary search significantly outperforms standalone LLMs, especially on tasks like the Traveling Salesman Problem. This suggests that even a less powerful LLM, when combined with an efficient search strategy, can outperform a much larger model working alone. This is still early-stage research. It’s computationally expensive, there are no guarantees it will work, and further optimization is needed. But the findings open doors to a future where AI can potentially not just execute code, but design its own algorithms to tackle some of the world’s most intricate challenges. The prospect of AI-driven algorithmic discovery is still unfolding, but it’s undoubtedly a fascinating area to watch.