Pete regrets being saved. He regrets it because a clean slate is infinitely more easy to dirty, because while he may no longer perform any misdeeds that can land him back at the penal farm, he knows what he is, and he supposes the man upstairs must know, too.
He supposes the man upstairs must also know that he didn't even get saved for the right reason. Perhaps if he went to church and told a parson this, he'd say that the lord doesn't care why a man is saved, he just cares that he wants to be saved. Pete, however, feels as though that tune might change if he told the parson he got saved because he saw the man he wants more than any treasure or restaurant out west, get saved himself.
It still keeps him up at night, long after the cook-fire's gone out, how Delmar looked after getting baptized. How he'd waded towards him and Everett, arms outstretched, drenched and ecstatic and holier than anything Pete's ever seen, before or since. How the mid-morning sun threw itself over him, how he shined and seemed to drip with gold, clothes sticking to him in a way that reminds Pete of the pewter statues of the saints in the shoddy chapel of the penal farm.
He didn't make for the river until Delmar mentioned heaven. Even if Pete never tells him, even if he's somehow able to take all this wanting to the grave, the idea of not going to the same place as Delmar after all is said and done is too awful to think on. So he runs in, even before Delmar heaves himself out of the river, following in his wake because maybe, just maybe, some of the water that passed over Delmar's mouth will pass over his. And perhaps this baptism will change him, make him better, make him holy.
And for a minute, maybe more, Pete Hogwallup does feel holy, until it hits him that the only reason he's let himself be baptized is because he wants another man, a man he dreams of laying with, in the very way that both the good book and the state of Mississippi prohibit. It doesn't help that Everett's being awful antagonistic over the state of their clothes, either. But then Delmar catches his eye, and gives Pete that smile that says "he doesn't get it, but you do" and Pete has to wonder how anything that feels as right as this can be considered a sin.