Will he change, or am I in denial?

He was hurt in his last relationships. He is all freaked out because he is starting to like me more than he thought he would. He tells me he doesn't think he can fall in love again and thought we should "end" it as he doesn't want to hurt me. Now it has ONLY been 2 months and we are not even really dating. It was decided from the beinging that it was a casual relationship and I am ok with that. I told him there was no time limit on where this "relationship" has to go in the next few months. He likes me and I know he is at the point where he COULD fall in love. As soon as he has a day off from work or any free time, he spends it with me and tells me he loves my company and that I am his Mrs. Perfect. But then he tells me he might not ever be able to "love" me and doesn't want the relationship to be one sided when the the comes. IF it comes. Should I listen to what he said and let him go to avoid possible heartache later on both sides, or do I stick around to see?.