Identifying different species of swallows might seem like a trivial task for humans, but can AI do it? Recent large vision-language models (LVLMs) have shown remarkable abilities in understanding images and reasoning about them, but their performance on fine-grained object classification, such as distinguishing between similar animal species, hasn't been thoroughly studied. This research introduces FOCI (Fine-grained Object ClassIfication), a new benchmark designed to test these models on challenging multiple-choice problems related to fine-grained object recognition. The benchmark utilizes existing datasets of various objects, including aircraft, flowers, food, pets, and cars, as well as domain-specific subsets from ImageNet-21k focusing on animals, plants, food, and man-made objects. The researchers tested twelve different LVLMs on FOCI, using four candidate answers (one correct, three similar incorrect options) to evaluate their performance. The study found that these powerful AI models, while strong in general image understanding tasks, often struggle with this level of specific classification. Intriguingly, basic CLIP models, which provide the image encoders for these LVLMs, significantly outperformed the larger, supposedly smarter models. This discrepancy suggests a misalignment between the image encoder and the large language model within the LVLMs when it comes to identifying subtle differences. The research also highlighted that larger training datasets for aligning the models with finer-grained details appear to be more critical for this task than for general image understanding. Simply put, even if the image encoder 'sees' the correct object, the LLM doesn’t always 'understand' it with the same accuracy. Additional experiments confirmed that using captions that specifically name the objects and incorporating fine-grained classification objectives during training can improve accuracy. This work demonstrates that FOCI reveals a distinct skill set in AI models that existing benchmarks overlook. Recognizing fine differences, like those between two swallow species, remains an ongoing challenge, demanding better alignment strategies and targeted training data to bridge the gap between human visual perception and AI image interpretation.