He sat in the same place every day, the bottom of the garden. He always hoped today was the day she would come. Yet his heart knew this was not so, he lost her - too pig headed to say he was sorry, sorry for making her cry, sorry for the lies he weaved "Oh Maisie I will leave my wife, you know I love you".
He had no intention of leaving his wife, he loved her you see, just the sex wasn't there, the intimacy, but he loved her in other ways, the companionship, the silence when needed.
With his mistress it was all hot passion she made him feel like a man, then she made the mistake of falling in love with him, that was never on the cards - so no he would never leave his wife for her. Now with his wife's passing, he thinks of those times, and in his lonely years he wishes he left when he had the chance.