IN LOVE WITH MY SISTER'S EX

Hi RTers, my name is Sharon (not real name) from Delta State, I'm 24 years old and a graduate. When i was smaller at about age 14, Emaa was my elder sister's boyfriend who is now married with 2 kids. They dated for about 3 years before they broke up. From what i knew then, the breakup was due to infidelity from Emma's a part. They were really cool together and we all thought they would end up together, but that was not to be. The breakup was devastating, i can still remember how my sister cried her eyes out and all.
Emma now 32 years old traveled to the US after his schooling and came back about a year ago, He works in an auditing firm and looks financially okay.

I met him when i did an interview with his company 5 months ago. I never knew he worked there, he walked up to me and introduced himself to me (of course he looked familiar but i couldn't really place his face), he looked so handsome and i was genuinely impressed. He took me out thereafter, i never really thought anything could come out of the date but he made me laugh like i've never done before. We talked for such a long time that i didnt even realise how far gone the time before he dropped me off. I grew very fond of him over the next 2 months that i didnt even realised when i fell in love with him. During the course of one of our discussions we talked about how he treated my sister 10  years ago, and he said he regretted what he did, but he was obviously very immature then and wouldn't do such a thing in his present age. Although a lot of people say Men are liars, but looking at him as he said all that, i couldn't help but believe him. 

He asked me to Marry him 2 weeks ago, and even if ever fibre in my body wants him so much, i can't but think of what people and my sister would think of me. I just couldn't say yes. I told him to give me sometime to think of it. I've been trying to talk to my elder sister about him but i haven't been able. I really love him and he seems to love me too and very responsible. 

Dear RTers, i need good advice on how to go about this dilemma. It would hurt me so much to lose him but i don't want to have problems with my family either.