It's common for the weight-loss suggestion by physicians to irritate patients, especially if the doc has no 'bedside manner' skills (an unfortunately common situation, I'm afraid.) However, the truth is that women who weigh either too little or too much can have problems conceiving -- and not expressly because of their weight per se.
It's easier to understand if you look at a person's weight as a symptom of other problems, some of which might be hormonal and could impact fertility. Another important point to make is that the reproductive system is just one of many in the body, and the most optimal setting for conception is a body in which all systems are 'go'.
On the more positive flip-side, a study was just published that showed no difference in pregnancy outcomes for women who were overweight when donor eggs were used. This likely highlights the fact that it's not so much a uterine issue, but an ovulation issue. Ovulation, of course, is so very, very dependent on a well-functioning hormonal system, the very system that can go awry either as a result of over or under weight or as a precursor.
Still, yes, docs have a hard time expressing all of the above in a sensitive, sympathethic manner. There's no substitute for people skills, even in the most highly trained physician!