Well I would say try to talk to him about how your feeling, I was in a relationship myself that is similar in the way of we were together for two years but no one thought it would last, and in those two years we never went out once, I broke up with him in may because I got sick of never seeing or talking to him, the biggest problem we had was that we didn’t communicate how we were feeling, Which is why you should just be outright and say what you feel, if he really cares about you he’ll hear you out and you can work together to fix the problem but if not I would suggest to break up with him, you deserve better than to always be put second