The conclusion of the authors, that LaTeX submissions should be restricted to mathematical journals only, is not even remotely supported by their experiment. Furthermore, they suggest .pdf submissions as an alternative, when .pdfs are the standard format that finished LaTeX files are transmitted in. How could this glaringly obvious oversight be made?

In response to criticism about the methodological flaws, that is, the flaws in copying short documents with specific formatting being the task, which LaTeX was never ever meant to be used to to, the authors say:
"Another rationale for using predefined texts is that this is the only way to realize a fair comparison between the two software alternatives. Asking people to generate own texts would be methodologically inadequate, because this would result in a high variability between texts and therefore would not allow reasonable comparisons between the two software tools.

We understand that people who are not familiar with the experimental methods of psychology (and usability testing) are surprised about our decision to use predefined texts. However, this is the normal and methodologically common way to minimize the effects of different types of texts which otherwise would cover the pure differences of software usability."

This is a completely unacceptable justification. It amounts to "we understand that our tasks were dissimilar to those performed in the real world, this is the way we've always done it. Deal with it." It doesn't matter at all what the standards in the discipline are for testing; this particular experiment reveals absolutely nothing about the efficiency of the two alternatives. LaTeX is not meant for quickly copying fancy custom formatting. In fact, formatting is done automatically specifically because the content of the document is meant to be separate from the formatting.

The authors do not mention bibliography management, inline references and figures, version control, graphics, and the countless LaTeX packages that add functionality that is completely missing in Word. Nor do they discuss the ability to interface between either alternative and Matlab, R, etc.

In other words, the experiment does nothing approaching testing the relative usefulness in writing a scientific paper, which is absolutely the question that should be being asked. There is a reason for the universal condemnation in the comments and on blogs.

Finally, the authors even note they have collaborated with computer scientists and that the experience led them to realize LaTeX wasn't worth the time. This as a defense is as absurd as, or more so, than the entire article itself. All this demonstrates is that they were frustrated by having to use a document preparation system with which they were not familiar. Perhaps the computer scientists were also at fault for being less flexible, but the notion that tax dollars could be saved by banning LaTeX has less than an infinitesimal amount of evidence in this paper.