I am going through something much similar. We got married this past april and haven't even see ovulation happening!!!! Our infertility is on my side but he was very hesitant to go in for testing. I finally got him to talk to me and he was afraid!!! Yeah, afraid that he would have a low count or some other problem and then he would feel he would be to blame if we couldn't have kids of our own. We were very stressed for a while and we had our time where all we did was fight and blame eachother for things--we couldn't even have a conversation without fighting!!!
Your dh needs to understand what you are going through right now and he needs to understand how much you love him and that if the IF is on his end that you will love him just the same. Tell him that you need time to feel conforted with him again, you both need time together--even if it is just laying on the couch talking. Make sure you both talk, get every concern,frustration and fear out and don't take it the wrong way!!! We were directing our frustration at eachother and not at the real cause--feeling inadequate that we weren't preg yet.
Hang in there, there is a light at the end of the tunnel...we aren't there yet but we can talk about everything now without getting angry and starting a fight. It is possible!!!
