The 3 midoriya children did not have a happy childhood. As far as Izuku could remember, he along with his older and twin sister had to suffer day in and day out. Izuku had very little happy memories as a kid. His parents dumped all their responsibilities on his oldest sister while they drank their lives away.
They had very little hope of their lives ever improving until that fateful day when things all changed, and for the better.
I do not own MHA, it is owned by Kōhei Horikoshi(I would like to keep my money thank you :) )