Would my cat be happier living with me or my mom?

Im 21 and moving out of my parents house. Rudy my cat has become apart of the family since i adopted him last year. Hes 3 years old now. He loves my mom and I and we love him. But im moving out in a couple months and I dont trust apartment complexes as I would be worried leaving Rudy there cooked up and the thought of a fire running through my mind and him being stuck in the apartment :( its a weird fear I have of apartments. He sleeps with me everynight as my mom doesnt like him licking himself waking her up at night so she wont let him in. Hes used to being alone from 8-6 on weekdays. Should I leave him wiht my mom as she will be alone? And probably will miss him even though she wont admit it im sure..id be lonely at night without him though..lol.