With the vSwitch and port groups configured with default policies, this shouldn't happen, except the ports on the physical switch(es) are not configured properly (e.g. spanning-tree).
Please provide some details about the virtual setup and how the ports on the physical switch are configured.
André