Stupid guy doin stupid things.......but my brothers best friend?

Well my brother's best friend wen he comes over allways sleeps in the guest room. But like in the middle of the night he would come into my room really late at night and feel me up in my bed and try to kiss me. I always tell him to Piss off or ***** off and he ends up going. I have a bf and he knows and besides i dont like him he's a jerk. im just afraid one time he wont go away, should i tell my brother. ( my older brother is 19 turning 20 on saturday and his mate is 19) I just dont want to ruin my brother and his friends relationship.