Just as Doctor Glas has no means of communication but his diary, Herzog, with really no one to talk to, has to mentally write these letters. Perhaps we all resort to such tricks to express emotions that we would be embarrassed to talk about to other people.
The free indirect style is very effective, mixing the author's and the character's voices until the boundaries are all blurred. Interestingly, when Herzog starts to recollect so vividly his past, he suspects that "these acute memories are probably symptoms of disorder". The depiction of his family is very moving, and it is clear that the Jewish collective memory is a great part of Herzog's cultural and emotional inheritance: "so we had a grear schooling in grief".
Sarcasm is a great defense mechanism (a wink to Sterling), much used by Jews (I was frequently reminded of Woody Allen). On personal improvement, Herzog says: "I expect to be in great shape on my deathbed. The good die young, but I have been spared so that I may end my life as good as gold".
On religion, it's clear Herzog is an agnostic: "There is always an overawing power, namely, one's terror".
Herzog is a man with an excess of self-consciousness, if that is possible. Clearly, he has defects as a romantic partner, but he seems to be a fundamentally honest, ethical person, He certainly is not lacking in the capacity for love and affection, as his relations with Junie, his daughter, and his brother show. However, he has problems relating to other persons, especially women. Of Daisy we get to now little. Madeleine is a horrible, horrible woman (at least from the perspective we get, but, in any case, she does cheats and she does try to keep Herzog away from Junie). Sono, the Japanese, sounds like an excellent woman, as well as the sexy Ramona, and Herzog himself knows that he is attracted by conflict, more than by tenderness and affection. That maight be a somewhat pathological condition.
Happiness, the difficulty of identifying, finding and keeping it, is of course one of the central subjects of modern art. "Some people are at war with the best things in life and pervert them into fantasies and dreams".
However, by the end, Herzog has found that unhappiness has no virtue in it (it may be the consequence of virtue, but it's not usually the case): "the advocacy and praise of suffering take us in the wrong direction and those of us who remain loyal to civilization must not go for it".
For the moment, just the rambling thoughts of a reader.