When the Snatchers bring Hermione to Malfoy Manor where only Draco remains, he orders them to leave her, and they obey. It would be foolish not to obey. He keeps her there with him, but can't fathom how ungrateful she's being. After all she is safe here, provided with gourmet meals, a luxurious bed, even a grand library at her disposal. Draco beleives he is keeping her from a far worse fate, it is about time she believes it, too. (I do not own these characters, they belong to Queen Rowling, I am merely playing with them)