Millie or Silverstream?

I've been really upset about Graystripe.

So he loves Silverstream, right? In the Last Hope, he's still mourning her. And then why does he dump her and go with Millie?