Polygamy dilutes that intimacy and undivided attention, turning marriage into something fundamentally different. If you take the position that limiting the number of spouses to one is an "injustice", then a liberal society must eventually follow the conclusion that *any* limit on the number of spouses must also be an "injustice"... whole towns could wind up married to each other.
Further, saying that polygamists "want more of what they already got" sounds alot like telling Homosexuals that "they want more of what they already got cause they have the right to marry anyone that they want so long as they are of the opposite sex."
Well then I guess one can make a lot of non-comparable positions sound a lot alike--even if some of them have 3X as many syllables and ignore concepts like
love and attraction, which on this topic is a whopper so I have to congratulate you for trying so hard.
As for your comment about the 'hassle', I personally don't find your glib dismissal of the mountain of legal and financial confusion resulting from legalized polygamy to be particularly conscientious or apt. On top of that, there is the insensitivity of putting millions of otherwise monogamous couples into a situation where a person can try to pressure their spouse to accept a third person into their sexual, emotional, legal and financial life... and do so legally.
Marriage is about intimacy, attentiveness and exclusivity. These are in large part why most people agree with gay marriage... because those qualities are retained. But polygamy would hurt all three.
Finally, the law in many places didn't define marriage as being only heterosexual. But it has indeed defined marriage as monogamous. Gays aren't trying to alter the established structure of marriage in our society, while polygamists are.