BIG HUGS
You have done everything you can and you haven't failed him in any way. As owners, we can only do our best but we cannot win every battle by sheer determination only - the outcome is ultimately out of our control, sadly.
Please be kind with yourself and give yourself time to rest, to grieve and being kind with yourself. Intense soul-searching and feelings of failure/guilt are typical for the onset of the grieving process. You wouldn't have them so strongly if you didn't love so deeply as they are the other side of the same coin. It is also normal that your emotions and the blues/emotional and physical exhaustion that come at some point once the adrenaline runs out are much more pronounced since you have given so much of yourself. It is not something you walk away from lightly. We are here if you struggle with others not understanding that it is the strength of your bond that determines how much you grieve and not the species.
Our grieving guide may help you understand better and make sense of what you are going to experience in the coming days and weeks:
Human Bereavement: Grieving, Processing and Support Links for Guinea Pig Owners and Their Children
We have got a Rainbow Bridge section for those owners who would like to post a tribute to a beloved pet whenever it feels right for them