'The Very Hungry Caterpillar,' by Eric Carle, is one I have used when teaching Kindergarten.
The story teaches counting to five, the names of the days of the week, and different types of food.
Chicka, Chicka, Boom Boom, by Martin, Bill, Jr. Each letter of the alphabet runs up to the coconut tree. This book is great because it has big, colorful illustrations, and a rhythmic chorus.
Just a few that I have used, and the children enjoy.