Reviewer #1's Review

“I think this is a nice method of analyzing gene expression data and provides some interesting, and novel, results.I have two principal comments (1 and 2), whilst the rest of the comments are relatively minor (3 onwards)
Comment 1:
Since a significant part of the paper is a new way of analyzing the data, I would recommend the paper has a ‘Methods’ section in the main text (possibly after the ‘Introduction’). This section should give a general overall summary of the approaches used to analyse the data. Namely,
Getting gene states (incl. using RMA)
Pathway consistency score and pathway activity score (incl. which scores are used in which circumstances)
Identifying pathways that distinguish phenotypes (incl. classifiers and feature analysis)
Using pathways / subtype signatures to predict disease outcome (incl. unsupervised clustering and Kaplan Meier analysis)
Where you have currently described methods in the ‘Results’ section, they could be moved to the ‘Methods’ section. The supplementary material can still be used to give a more detailed description of methods.
It would also be useful to provide references (or a brief description) for feature analysis, unsupervised clustering and Kaplan Meier analysis.
Also, for clarity, it may be worth mentioning that each pathway has its own score in each sample
(data set).
Comment 2:
In ‘Supplementary Methods’, I believe your formula’s for p(x | Up), p(x | Down), p(x), p(Up | x) and p(Down | x) are incorrect. In particular, I believe we should have the property
p(Up | x) + p(Down | x) = 1 (which is not true in your version).
Below (in a, b and c), I have tried to explain this in more detail. Is this just a typo in the manuscript? In this case, do your algorithms use valid probability distributions? In this case, can you correct the typos in the supplementary methods.
Or
Did you use these probability distributions in your algorithms? If so, the probabilities associated with your gene states will be lowered, possible affecting your results. In this case, I recommend reproducing your results to check your conclusions are still valid.
a)
For p(x | Up) and p(x | Down), I think it should just be
p(x | Up) = f(x | aU, bU)
p(x | Down) = f(x | aD, bD)
They are conditional on us already knowing the gene is Up / Down, and therefore should not be multiplied by p(Up) / p(Down) respectively
b)
I think p(x) should be as follows
p(x) = p(Up)*p(x | Up) + p(Down)*p(x | Down)
c)
p(Up | x) would then loose one of the ‘NU / N’ (= p(Up)) terms in the numerator. i.e. We would have
p(Up | x) = p(Up)*f(x | aU, bU) / {p(Up)*f(x | aU, bU) + p(Down)*f(x | aD, bD)}
This would give the following property.
p(Up | x) + p(Down | x) = 1
In your version, it is likely to be the case that p(Up | x) + p(Down | x) < 1. This cannot be true since, you are only choosing between one of two states (Up and Down).”

n.b. These are the more general comments made by the reviewer when reviewing the originally submitted version of this paper. The manuscript was revised before publication. Specific minor points addressed during revision of the paper are not shown.