DH hates my parents - HELP - conception, pregnancy, baby, toddler ...

Hi all.
Im just needing some advice and a place to vent on an issue that has me confused, sad and feeling guilty. Sorry for its length in advance.

In a nutshell: my husband 'hates' (his words) my parents.
Meanwhile I love my parents and honestly think they are 2 of the best people on the planet (not that they are perfect but that they are nice people who love us and would never purposely cause problems/rifts between DH and I or themselves and DH).
Now onto the specific problem:
My family (including all extended family) live 4hours drive away at the other end of the state so I try to visit every 2 - 3months and stay for a week at a time. The problem is knowing DH's feelings about my family I feel guilt whether he comes with us or stays home by himself.
I have gone without DH many times and have twice since her birth just taken DD and myself and that's fine with me however DH wont ring, barely replies to text or facebook messages while we are gone and when I questioned him the last time I went about it he claims that he cant as it reminds him how much he misses us and makes him sad, lonely, depressed etc.
Or he can come with us and then I feel guilty that he uses up his holidays with 'people he hates'.
Now I will say this: I do get a bit annoyed at my DH when he comes as he makes it tense by not speaking unless spoken too, by not pitching in with anything (even DD's first birthday party etc.) and whilst staying there he doesnt get up til 11am, he then either play games on his phone, visits people with me (always given the option - do you want to), Dad usually has an arvo off work and they play golf, we take DD swimming etc. and he then spends all night with my brother playing xbox/ps3 late into the night. So I dont see how he can 'hate' it that much however I just saw on his facebook that during our previous trip he was messaging someone how much he hated it and his inlaws etc. It makes me sad that he publicly declares how much he hates my parents despite them never doing anything bad to him (or me) and them going out of their way to do stuff he likes (i.e golf) and trying to get to know him.

I know in my heart I shouldnt feel guilty about visiting family as DH is an adult, this was the situation when he met me (parents living away) and he knew how important family was to me. However I cant help it and I feel that this will be an ongoing issue throughout our marriage/DD's childhood (I should add my family is big on family holidays, celebrating events so I can only assume we will spend more time with them as our child/ren get older)

Sorry for my ramble but I guess I just dont know what to do - do I confront DH about his attitude/behaviour whilst around them, do I just tell him to stay home barring special events, or do I just leave it and hope he doesnt have an anxiety/depression episode as he spends more time with them in the future (he wont go to the dr about them so I dont know what to call it exactly however about once or twice a year he seems to have a mini breakdown and all this sort of stuff come out/up and becomes a huge issue for a few weeks).
HELP

What do you do if you are in a similar situation?

Source: http://www.bubhub.com.au/community/forums/showthread.php?490064-DH-hates-my-parents-HELP

kids choice awards ncaa final four 2012 uk vs louisville university of kansas buckeye west side story final four 2012