Is my boyfriend falling in love with me?

My bf and I have been dating for 4 months. He is 25, I am 32 and I have 2 kids from a previous relationship whom he adores. When we first started dating we agreed not to rush into anything, have fun and just see what happens. About 2 months into it he started making jokes about the future - moving in, marriage, being a father to my kids. He always made the comments sound like he was joking but occasionally did sound serious like "if we end up together" or "if this works out". About 2 weeks ago his best friend moved in with his girlfriend who has 2 young children and my bf said the thought of that was scary to him. When I got back from being away for a few days he told me he thinks I want more of a serious relationship than he does. I have never said this or talked about the future but he said it was just a vibe. He always breaks it off with a girl when she starts to have feelings and he doesn't feel the same (usually after dating 2 months) and says he thinks my feelings are stronger than his and that he doesn't want anthing serious. He has only had one serious relationship a few years ago and she broke his heart. So I told him I don't want to rush, let's have fun, take our time and see what happens. He was ok with that but I think he has been scared that I might end it because since then he has been so attentive. He has been telling me he misses me when we don't see each other for a few days and we spent an amazing weekend together and he told me he has the best time with me and how great I am and apologized for "freaking out". He also starting joking about the future again. Is he saying those things to make sure I don't leave him or is he realizing he does feel more for me or what. I just don't understand, was he feeling something all along and maybe got a little scared? I am just going to see what happens but I really don't want to get hurt because I am falling for him. What do you think?