Hiruzen looked after Toji until he was 4 years old. Toji was given his own apartment to live in. He lived on the lousy allowance Hiruzen gave him.
His jinchuriki status was leaked to the village by Danzo. When the villagers learned of it. They started hating him. He was prohibited from entering most of the shops in the village.
When Toji was around five, Teuchi spotted him peering in near closing time. It was raining heavily. Teuchi recognized him as the boy that everyone hated, but he didn't care. There was a cold, hungry child at his door and Teuchi wasn't gonna let that stand. So he invited Toji into the warm, dried him off and gave him a big bowl of fresh ramen.
In the process, becoming the first person who ever treated Toji with kindness.
Toji thanked him. Teuchi told him to come back anytime.
Toji lived a simple boring life. When he wasn't training he would stay at home or go to Ichiraku for free meals.
Two years later
Hiruzen told him he was going to enroll in the academy. He told Toji it would be a good opportunity to make friends. Naruto told him he had no interest in making friends. The only thing he cared about was money. Hiruzen then told him if he graduated from the academy he would become a ninja and be able make a lot of money. Toji immediately reconsidered and told him he would join the academy.When Toji joined the academy he was ignored by other academy students because their parents warned them to stay away from him. Toji simply didn't care about this.
A few weeks later he saved his classmate Hinata from three bullies.
"If you really are a Hyuga then show us your Byakugan but if you not gonna don't look at us."
"Yeah your eyes are creepy"
"I bet you actually some kinda monster."
"Yeah a Byakugan monster."
The three bullies laughed. Hinata started to cry.
"If you gonna pick on someone. Try me." Toji
The bullies turned around.
"Who are you."
"Nobody important" Toji said as he before he appeared in front of them.
"He's fast" they thought
Toji grabbed two of them on the shoulder. They could feel their bones breaking.
"Get out of my sight." Toji said with a menacing glare
He let the two go. They immediately ran. The Third bully followed them.
"Thank you" Hinata said after wiping her tears
"There's no need to thank me." Toji said before disappearing
From that day Hinata became interested in Toji. She started to watch him closely.
Toji excelled in all practical aspects of the academy curriculum except ninjutsu.Instructors and students were awe in of how good his taijutsu was and also his proficiency with weapons. He was average when it came to his studies(due to lack of interest).
5 years later
When Toji turned 10 the Shinagami gave him the two weapons. At age 11 Toji met the fox. He didn't care about his parents murderer. He simply told it to stay out of his way.
It was time for the graduation exam.
Okay class we will now start the graduation exam. Naruto you can sit
this one out you pass. The Hokage considers you qualified to become a ninja" Iruka