N.18 “Correlation energy”

The error in Hartree-Fock is due to the single-determinant approxi­mation only. A term like “Hartree-Fock error“ or “single-determinantal error” is therefore both precise, and immediately under­standable by a general audience.

However, it is called “correl­ation energy,” and to justify that term, it would have to be both clearer and equally correct mathemati­cally. It fails both requirements miserably. The term correl­ation energy is clearly confusing and distracting for non­specialist. But in addition, there does not seem to be any theorem that proves that an independently defined correl­ation energy is identical to the Hartree-Fock single determinant error. That would not just make the term correl­ation energy disingenuous, it would make it wrong.

Instead of finding a rigorous theorem, you are lucky if standard textbooks, e.g,. [32,28,44] and typical web references, offer a vague quali­tative story why Hartree-Fock under­estimates the repulsions if a pair of electrons gets very close. That is a symptom of the disease of having an incom­plete function represen­tation, it is not the disease itself. Low-parameter function represen­tations have general difficulty with representing localized effects, whatever their physical source. If you make up a system where the Coulomb force vanishes both at short and at long distance, such correl­ations do not exist, and Hartree-Fock would still have a finite error.

The kinetic energy is not correct either; what is the correl­ation in that? Some sources, like [28] and web sources, seem to suggest that these are “indirect” results of having the wrong correl­ation energy, whatever correl­ation energy may be. The idea is apparently, if you would have the electron-electron repulsions exact, you would compute the correct kinetic energy too. That is just like saying, if you computed the correct kinetic energy term, you would compute the correct potential too, so let’s rename the Hartree-Fock error “kinetic energy inter­action.” Even if you computed the potential energy correctly, you would still have to convert the wave function to single-determinantal form before evaluating the kinetic energy, otherwise it is not Hartree-Fock, and that would produce a finite error. Phrased differen­tly, there is absolutely no way to get a general wave function correct with a finite number of single-electron functions, whatever corrections you make to the potential energy.

Szabo and Ostlund [44, p. 51ff,61] state that it is called correl­ation energy since “the motion of electrons with opposite spins is not correlated within the Hartree-Fock approxi­mation.” That is incom­prehensible, for one thing since it seems to suggest that Hartree-Fock is exact for excited states with all electrons in the same spin state, which would be ludicrous. In addition, the electrons do not have motion; a stationary wave function is computed, and they do not have spin; all electrons occupy all the states, spin up and down. It is the orbitals that have spin, and the spin-up and spin-down orbitals are most definitely correlated.

However, the authors do offer a “clarifi­cation;” they take a Slater determinant of two opposite spin orbitals, compute the proba­bility of finding the two electrons at given positions and find that it is correlated. They then explain: that’s OK; the exchange requirements do not allow uncorrelated positions. This really helps an engineer trying to figure out why the “motion” of the two electrons is uncorrelated!

The unrestricted Hartree-Fock solution of the dissociated hydrogen molecule is of this type. Since if one electron is around the left proton, the other is around the right one, and vice versa, many people would call the positions of the electrons strongly correlated. But now we engineers understand that this “does not count,” because an uncorrelated state in which electron 1 is around the left proton for sure and electron 2 is around the right one for sure is not allowed.

Having done so well, the authors offer us no further guidance how we are supposed to figure out whether or not electrons 1 and 2 are of opposite spin if there are more than two electrons. It is true that if the wave function

$$\Psi({\skew0\vec r}_1,{\textstyle\frac{1}{2}}\hbar,{\skew0... ...-{\textstyle\frac{1}{2}}\hbar,{\skew0\vec r}_3,S_{z3},\ldots)$$

is represented by a single small determinant, (like for helium or lithium, say), it leads to uncorrelated spatial proba­bility distributions for electrons 1 and 2. However, that stops being true as soon as there are at least two spin-up states and two spin-down states. And of course it is again just a symptom of the single-determinant disease, not the disease itself. Not a sliver of evidence is given that the supposed lack of correl­ation is an important source of the error in Hartree-Fock, let alone the only error.

Koch and Holthausen, [28, pp.22-23], address the same two electron example as Szabo and Ostlund, but do not have the same problem of finding the electron proba­bilities correlated. For example, if the spin-independent proba­bility of finding the electrons at positions ${\skew0\vec r}_1$ and ${\skew0\vec r}_2$ in the dissociated hydrogen molecule is

$${\textstyle\frac{1}{2}}\vert\psi_{\rm {l}}({\skew0\vec r}_... ...\vec r}_1)\vert^2\vert\psi_{\rm {l}}({\skew0\vec r}_2)\vert^2$$

then, Koch and Holthausen explain to us, the second term must be the same as the first. After all, if the two terms were different, the electrons would be distin­guishable: electron 1 would be the one that selected $\psi_{\rm {l}}$ in the first term that Koch and Holthausen wrote down in their book. So, the authors conclude, the second term above is the same as the first, making the proba­bility of finding the electrons equal to twice the first term, $\vert\psi_{\rm {l}}({\skew0\vec r}_1)\vert^2\vert\psi_{rm{r}}({\skew0\vec r}_2)\vert^2$. That is an uncorrelated product proba­bility.

However, the assumption that electrons are indistin­guishable with respect to mathematical formulae in books is highly controversial. Many respected references, and this book too, only see an empirical requirement that the wave function, not books, be anti­symmetric with respect to exchange of any two electrons. And the wave function is anti­symmetric even if the two terms above are not the same.

Wikipedia, [[23]], Hartree-Fock entry June 2007, lists electron correl­ation, (defined here vaguely as “effects” arising from from the mean-field approxi­mation, i.e. using the same $v^{\rm {HF}}$ operator for all electrons) as an approxi­mation made in addition to using a single Slater determinant. Sorry, but Hartree-Fock gives the best single-determinantal approxi­mation; there is no additional approxi­mation made. The mean “field” approxi­mation is a consequence of the single determinant, not an additional approxi­mation. Then this reference proceeds to declare this correl­ation energy the most important of the set, in other words, more important that the single-determinant approxi­mation! And again, even if the potential energy was computed exactly, instead of using the $v^{\rm {HF}}$ operator, and only the kinetic energy was computed using a Slater determinant, there would still be a finite error. It would therefore appear then that the name correl­ation energy is sufficiently impenetrable and poorly defined that even the experts cannot necessarily figure it out.

Consider for a second the ground state of two electrons around a massive nucleus. Because of the strength of the nucleus, the Coulomb inter­action between the two electrons can to first approxi­mation be ignored. A reader of the various vague quali­tative stories listed above may then be forgiven for assuming that Hartree-Fock should not have any error. But only the unrestricted Hartree-Fock solution with those nasty, “uncorrelated” (true in this case), opposite-spin “electrons” (orbitals) is the one that gets the energy right. A unrestricted solution in terms of those perfect, correlated, aligned-spin “electrons” gets the energy all wrong, since one orbital will have to be an excited one. In short the “correl­ation energy” (error in energy) that, we are told, is due to the “motion” of electrons of opposite spins not being “correlated” is in this case 100% due to the motion of aligned-spin orbitals being correlated. Note that both solutions get the spin wrong, but we are talking about energy.

And what happened to the word “error” in “correl­ation energy error?” If you did a finite difference or finite element compu­tation of the ground state, you would not call the error in energy “truncation energy;” it would be called “truncation error” or “energy truncation error.” Why does one suspect that the appro­priate and informative word “error” did not sound “hot” enough to the physicists involved?

Many sources refer to a reference, (Löwdin, P.-E., 1959, Adv. Chem. Phys., 2, 207) instead of providing a solid justifi­cation of this widely-used key term themselves. If one takes the trouble to look up the reference, does one find a rigorously defined correl­ation energy and a proof it is identical in magnitude to the Hartree-Fock error?

Not exactly. One finds a vague quali­tative story about some perceived “holes” whose mathemati­cally rigorous definition remains restricted to the center point of one of them. However, the lack of a defined hole size is not supposed to deter the reader from agreeing whole­heartedly with all sorts of claims about the size of their effects. Terms like “main error,”, “small error,” “large correl­ation error” (qualified by “certainly”), “vanish or be very small,” (your choice), are bandied around, even though there is no small parameter that would allow any rigorous mathematical definition of small or big.

Then the author, who has already noted earlier that the references cannot agree on what the heck correl­ation energy is supposed to mean in the first place, states “In order to get at least a formal definition of the problem, ...” and proceeds to redefine the Hartree-Fock error to be the “correl­ation energy.” In other words, since correl­ation energy at this time seems to be a pseudo-scientific concept, let’s just cross out the correct name Hartree-Fock error, and write in “correl­ation energy!”

To this author’s credit, he does keep the word error in “correl­ation error in the wave function” instead of using “correl­ation wave function.” But somehow, that particular term does not seem to be cited much in literature.