I love all the books, so am probably completely biased...
But, I think that Chamber is Secrets is probably to one I like the least, so I can see where you are coming from. As others have said, from Prisoner of Azkaban, it gets so, so, very good. Though, I do love CoS, mostly because of...