Last year I wrote blog posts criticising two papers from the same group about electrodiffusion modelling in dendritic spines. One was a review in Nature Reviews Neuroscience (article, blog), the other an analytical/modelling study in Neuron (article, blog). More in hope than expectation, I drew my concerns to the attention of the respective editors. I was pleasantly surprised: the issues were taken seriously at both journals and, after sufficiently positive reviews, the resulting exchanges of correspondence have now appeared. Neither journal showed me the authors’ reply (this is standard, if slightly unfair, procedure), so below I give a brief reaction to those replies. I also append a few reflections on the editorial process. Finally, I have learnt a few interesting things through these discussions; I list them at the end.
“The new nanophysiology…” (Nature Reviews Neuroscience)
Because of space restrictions and possible referee fatigue, my letter was restricted to the most serious errors. The gist of my comments was: physiological solutions contain large numbers of both negative and positive ions, not just a few positive ions; electroneutrality is unavoidable under physiological conditions; and that there were several problems in the equations, including a nonsensical redefinition of capacitance.
In their response (“Electrodiffusion and electroneutrality” section), the authors backpedal a bit on their suggestion that electroneutrality should not be assumed when modelling ionic behaviour in spines (the French have a charmingly appropriate expression about drowning a fish). They try to suggest that their article was about (uncontroversial) electrodiffusion rather than electroneutrality. However, in the original article, they state: “Indeed, if this assumption is not made it can be shown that there can be long-range electrostatic interactions over distances much larger than the Debye length…” (in reality the assumption does hold and the long-range interactions do not occur). Moreover, all of the equations (Box 1) and simulations (Fig. 3, Box 2) involve or were intended by the authors to involve situations without electroneutrality. Even in their response, the authors still try to claim that “… electroneutrality may break down at the tens of nanometre scale…” (it doesn’t).
Alongside this woolly discussion, the authors suggest that the only mobile anions inside cells are about 7mM chloride ions. This statement is interesting from two points of view. Firstly, it is very obviously false. The cytosol contains 25mM HCO3-, about 20mM of glutamate and aspartate combined, several phosphate species (ATP, ADP, AMP, inorganic phosphate, phosphocreatine…), lactate and many other metabolites with net negative charges. These certainly represent several tens of mM and are quite respectably mobile. Secondly, even if we take the authors’ line of thought to its logical extreme and imagine all intracellular anions to be immobile, that would only extend the Debye length to about 1nm, still providing excellent screening over an extremely short range (nanometres, not tens or hundreds of nanometres). Such immobile anions are not represented in the authors’ model, but, if they were present the bulk of matching positive and negative ions would still ensure the accuracy of the electroneutrality approximation. Finally, the combination of anion immobility and electroneutrality would also prevent any alterations of total ion concentrations when synaptic current flows, yet the authors argue elsewhere (the Neuron paper also criticised here), as do the Yuste group, independently, that this effect is significant. Oops!
It appears that I misunderstood the purpose of Box 1. I am grateful that the authors have now clarified (in “Boundary conditions matter“) that its equations differ from those used elsewhere in the article. In addition to being irrelevant within the article, the equation system in Box 1 still seems to be internally inconsistent and is therefore of doubtful relevance to anything at all. Thus, the authors affirm that in addition to a zero electric flux condition over most of the spine head, they did indeed ground (set V = 0) at a disk where the entrance to the spine neck would be. I don’t believe these boundary conditions can be satisfied with any distribution of positive charges only in the spine head. A challenge for the authors: produce such a solution for a single positive charge, verifying the boundary conditions. Where should that charge be positioned? (Some ambiguity may arise if external charges are allowed; the authors never specified what lies outside the dielectric sphere.)
In describing the same zero electric flux boundary condition, the authors make the supremely bizarre statement that “The latter condition models an ideal capacitor where the permittivity of the membrane bilayer would be zero” (with a reference to my letter!?). A glance at the formula for the capacitance of a parallel-plate capacitor
Capacitance = (Permittivity)(Area)/(Separation)
suggests that this capacitor would be ideal in the sense of having zero capacitance and therefore not existing at all. To be honest, I’m completely lost here.
Regarding my criticism that their exciting “Non-classical behaviour of membrane capacitance in a nanocompartment” contained no membrane and was only non-classical because they had introduced a new and useless definition of capacitance (not because of the nanocompartment), the authors take the opportunity to repeat what was in their article. They confirm that they have redefined capacitance (the response section is entitled “Redefining capacitance“), but don’t explain what utility the new definition might have beyond allowing them to “[find] it in other cases, such as fluctuation of the membrane [sic?] of a dendrite…” Indeed, that work is one of a series of papers (mostly from their group or irrelevant) adduced in support of their combative conclusion that “Nanophysiology is happening“.
“Deconvolution of Voltage Sensor Time Series…” (Neuron)
For this paper, too, my letter was much shorter than the blog post initially submitted; it was in fact restricted to just three points:
- The authors attempt to solve underdetermined equations for the spine neck resistance.
- Instead of ‘extracting’ the value from experimental data via an optimisation, as claimed, it was set manually by the authors’ initial parameter choices (in other words, their ‘optimisation’ halts near the predetermined value).
- The authors model a spine neck using a cable equation with a sealed end instead of an electrical connection to the dendrite.
Amazingly, the authors’ response ignores these three issues entirely. Go see for yourself. It’s surprising that the journal was satisfied with such a non-response.
The editorial process
I thank and commend the editors for having reviewed and ultimately publishing the correspondence pieces. It takes bravery and rigour to allow criticism of one’s own output; many editors struggle enormously with this conflict of interest (hello, Nature Materials!). However, these affairs still expose some weaknesses in today’s editorial processes.
The concerns I raised are essentially mathematical; they are either right or wrong. Yet, even after specific re-review of these issues, the editorial processes of two major journals were unable to decide whether they were in fact right or wrong, preferring to leave “sophisticated readers” to sort things out for themselves. Clearly the original manuscripts were accepted without anybody actually understanding what they contained. That doesn’t surprise me, but it does jar with the verification function that journals are supposed (and claim) to perform. Disturbingly, the affairs also suggest a publication strategy to exploit this reviewing loophole: team up with a celebrity experimentalist, make some grand (or grand-sounding) claims, surround them with incomprehensible equations (correctness optional) and profit. Sadly, once published, it probably would be better for the authors’ careers to deny and obfuscate everything, to avoid any substantive correction and keep their references in glamour journals alive.
The affairs also exemplify what might be termed ‘publication hysteresis’: to get into a major journal you need referee unanimity (or so I’m always told), yet to get a paper retracted, it seems you also need referee unanimity (this I can confirm). That leaves a huge grey zone, where re-examination reveals papers that shouldn’t have been accepted but which aren’t retracted. Given the importance attached to papers in glamour journals, this feels like an abdication of responsibility. It is useful to recall the Committee on Publication Ethics (COPE) guidelines, which state that retraction should be considered if the editors “have clear evidence that the findings are unreliable, either as a result of misconduct (e.g. data fabrication) or honest error (e.g. miscalculation or experimental error)“. I have no doubt that many of the central claims in both papers are unreliable.
What have I learnt?
Analysing these papers has required quite a lot of effort. Some of the issues are complex and technical, and the deepest problems are rarely exposed with the greatest clarity! It’s fair to question whether it was an efficient use of my time. Indeed, a recurring criticism of critics is that they should spend more time being positive in their own work rather than wasting it being negative about others’. In today’s career structure, I very much doubt that I have been advancing my career optimally, if at all. Inevitably, one tends to create enemies, which is risky (well, risky for an academic career). But, I also believe that we should change that career structure so that publishing low-quality work becomes a net negative. I don’t see how that can be achieved without calling out such work; the current approach of imagining it will be possible to ignore bad papers during career evaluations or grant application reviews is simply not realistic when those papers have been published in glamour journals like Nature Reviews or Neuron. Surprisingly often, one finds oneself in the position of seemingly being the only person in the world who is interested, able and, crucially, willing to analyse critically some piece of work. I think we all have a duty to share our expertise in such cases; the PubPeer platform allows one to do so anonymously if desired.
If what I have done is peer review, it’s of a very different kind to standard pre-publication peer review. I have certainly spent much, much more time on this than on any paper I have refereed. The detail and understanding attained is correspondingly deeper and, hopefully, more useful to others. Note also that there was a strong bias in selecting what to review: this was something I found interesting and where I felt I could make a useful contribution. Rapid reviews of random papers seem quite superficial and boring in comparison. I prefer the new method.
I believe that direct, immediate, public confrontation of ideas (not necessarily of people) allows much more rapid distillation of the truth and therefore accelerates scientific progress. Despite my overall negative stance in this affair, this clarification of ideas has nevertheless caused me to learn about and understand new concepts and, maybe, to identify questions for future research, on which a few thoughts now follow.
I hadn’t realised quite how significant the synaptic sodium influx into a spine could be. I was impressed by the extent to which electroneutrality causes potassium ions in a spine to be rapidly expelled by that entry of sodium.
The suggestion by the authors that a counterflow of anions within the spine can cause a gradient of total ionic concentration is plausible, although ultimately its electrical significance seems to be relatively limited. That gradient cannot be established without mobility of anions.
The fact that an excitatory current is carried uniquely by positive charges may largely prevent (non-capacitive) flow of anions when modelled correctly, which may alter the apparent resistivity experienced by the synaptic current, at least it could at low frequencies; this remains to be explored.
The discussion forced me to go through the intracellular ionic composition again. Some anions seem to be missing. Back-of-the-envelope calculations (which need formalising) suggest that negative charges on proteins only supply a low concentration. The authors’ remark in their response that many of the intracellular anions are on membrane lipids is interesting; a first calculation suggests they are numerous. How concentrated under the membrane are the counter-ions? Are they osmotically active?
Referee #3 