I wouldn't worry over this. I personally didn't start schooling my children until they were 5 and then I only used the Rod & Staff ABCDEF workbooks. In 1st grade is when I started Abeka Phonics and at that time they both were able to catch on. Please don't let him stress over this. He's only 4 and will have plenty of time to learn. I can't get either of my two girls (now 6th and 3rd) to put a book down now. A little bit at a time is much better IMHO. Hope this helps.
