is it normal that one of my guinea pigs (Fudge) is eating poop every once in a while, and he actually swallows it.
Yes, that is perfectly normal and indeed necessary.
Grass/hay is very nutritious but very hard to break down. All animals for who this is bulk of their diet have had to come up with solutions.
Hay makes about 80% of the food daily intake. This is why guinea guinea pigs have such a large gut - much larger in comparison than a human gut. And they need to use to twice to make the most of their main food source.
Caecotrophs (redigested poos) are the poos that contain the not yet fully broken down hay/grass fibre from the first run through the gut. They are produced at a different time to the waste poos and have a slightly different texture and colour. They are generally picked up directly from the anus because they are so precious unless a piggy can no longer do this (arthritis or an accident/disability).
You get waste poos from both runs through the gut. But because the digestive process takes a long time; the waste poo output generally lags 1-2 days behind the food intake. That is why weighing daily at the same time (instead of the life-long once weekly health monitoring weigh-in) is so important when it comes to feeding support; otherwise you are lagging badly behind with your care. You can also never judge the hay intake just by eye.
Weight - Monitoring and Management
Please make sure that your piggies always have access to unlimited hay. It is vital for their dental health (the back teeth have evolved to constantly grow at the ideal speed against the very abrasive silica in hay) and the gut is very much geared up to deal with mostly hay/grass fibre. Your piggies will also not be at risk of scurvy because fresh grass is rich in vitamin C and hay still contains, too. It is the reason why guinea pigs never had the need to make their own. A good hay based diet with some limited mainly green and leafy veg and just 1 tablespoon of pellets max can make all the difference between a healthy piggy reaching the lower end of the average life span or the higher end, or even living beyond it.
I hope that this helps you seeing some of the connections between the general advice?
Here is more interesting information on guinea pigs as a species:
Guinea Pig Facts - An Overview