I think the Doctor first subconsciously realized how he felt about Rose in Dalek. When he seals the door and Rose doesn’t make it through, you can see something change in his face that’s much more than him just feeling guilty. He’s terrified. Terrified of losing her. He hasn’t admitted it to himself at that point, but when the Dalek manipulates him by telling him to save “the woman he loves,” he certainly doesn’t argue with that. Later in that episode, when Rose refuses to let him kill the Dalek, she says “What the hell are you turning into?” That stops him right in his tracks. He wants so badly to hide the monster he thinks he is from Rose, to change into something halfway lovable.
I also think that’s the precise moment Rose begins to really fall for the Doctor. She sees him do something unthinkable for her: he lowers his weapon in front of a Dalek. The creature [partially] responsible for the destruction of his home, his race, and his sanity, and he lowers his weapon because Rose asked him to. At that moment I think she sees a glimpse of his potential, his inner soul that is very different from the hardened man the Time War has made him, someone who can love, someone who wants to love.
Of course Rose doesn’t make her first move until The Doctor Dances. She’s acting very bold, reckless even (perhaps as part of a defense mechanism after the traumatizing events of Father’s Day), and she uses her flirting with Jack to test the waters with the Doctor. He still has his guard up, and he does until The Parting of the Ways. He uses a regeneration to save her, and he does it without a second thought. He loves her. He’s admitted it to himself.
In Tooth and Claw, it’s clear they have some sort of understanding about their feelings for each other, even if it’s unclear whether or not they have discussed it. Their joking, their body language, and their constant eye contact tells us for sure that they’re in love.
In The Impossible Planet, we see that the initial joy of being in love has faded a bit, because they both seem to realize that their relationship can’t be simple. Rose suggests that they live a domestic life, and you can see the agony in the Doctor’s eyes. He knows he can’t live with her, but he isn’t ready to hurt her or let her go. Rose too seems to understand this, and she puts on a façade of casual acceptance, even though we sense her heartbreak and he does too. When he says “Oh she knows” as he’s about to go into the pit, I don’t at all believe that means he’s said “I love you” to her before, at least not in the context of romantic love. I think he’s trying to convince himself that he’ll never have to say it, because that will make it real. He hopes that they can continue silently knowing without acting on it, because that would only lead to pain.
In The Satan Pit, The Doctor says he believes in Rose. That’s when we know he’s accepted his love for her, because he’s verbalizing it. This is his equivalent of a confession. He has to confess to Satan himself rather than to a God because he feels that he isn’t worth any more than that due to his past crimes, along with the crimes he knows he will soon commit. He knows his love for her will do terrible damage to both of them, and that it would be selfish to tell her, and he is confessing that sin to the only being he feels he is worthy of confessing to, the Devil himself.
When he says he believes in Rose, he is saying that he believes his heart for once. He believes what his heart has been telling him all along, and he chooses it over his head. It is not practical to love Rose, it is not logical, it is not useful, but he BELIEVES his love for her. For the first time he believes that something really can exist without explanation, and that the meaning of something does not lie solely in his knowledge of it. He is choosing faith. He does not know love, he BELIEVES it. He is trusting his own heart. And that is why their love is so beautiful.
Friendly reminder that while Eleven was here
Nine and Rose were up here
The same day, two different regenerations
why am i crying