Sirius Black makes the smart choice and doesn't go after Pettigrew when James and Lily are killed. Instead he takes Harry from the cottage and raises him away from Dumbledore's reach as Albus tries to gain control over Harry's life to use him as a weapon when the Dark Lord finally returns. With Dumbledore failing in this Harry is raised by Sirius at Black manor living in secret to avoid the attention of the wizarding world and Dumbledore until Harry is ready to return at the age of 11 and throw Dumbledore for a loop.