Piggies are stressed by any complete change of environment and any changes in the group.
A change of environment means that they have to learn all the safe spots again that prey animals like guinea pigs have established a a sort of map in their brain. For some piggies that's quick learning, for others not so much.
A change in the group, no matter if somepig is new or somepig has left the group is a massive stressor to every member of the group. The hierarchy has to be worked out again.
Of course there are individual differences. Some piggies are just more relaxed, others are extremely skittish. That's due to genetics, their personal experiences, their hormonal status and their age.