Imagine teaching a computer to make decisions, not by feeding it mountains of data, but by simply describing the task. That’s the surprising premise of new research exploring how large language models (LLMs) can build decision trees—a fundamental machine learning model—from scratch, without any training data. Traditionally, decision trees learn by analyzing datasets, identifying patterns to create a branching structure of if-then rules. This new research flips the script, asking LLMs to create these trees using only their existing knowledge and a description of the features involved. The results are intriguing. On certain small datasets, these "zero-shot" decision trees actually outperform those trained on data. This opens exciting possibilities for leveraging LLMs when data is scarce or privacy is paramount, particularly in fields like healthcare. The research also dives into creating embeddings from these LLM-generated trees. These embeddings are compact representations that capture relationships between features, useful for powering other machine learning models. Remarkably, these "zero-shot" embeddings perform comparably to embeddings derived from traditional, data-trained trees. While the research focuses on small datasets and a simple prompting method, it serves as a powerful demonstration of the potential of LLMs as automated model generators. Further improvements in prompting strategies, combined with the ongoing development of even larger and more capable LLMs, could unlock even more powerful applications in the future. This could revolutionize how we build AI models, making them accessible to a broader range of users and applications.