No effect of multi-axis dot pattern symmetry on subjective duration

Previous work has shown that symmetrical stimuli are judged as lasting longer than asymmetrical ones, even when actual duration is matched. This effect has been replicated with different methods and stimuli types. We aimed to a) replicate the effect of symmetry on subjective duration, and b) assess whether it was further modulated by the number of symmetrical axes. There was no evidence for either effect. This null result cannot be explained by reduced statistical power or enhanced floor or ceiling effects. There is no obvious stimulus-based explanation either. However, we are mindful of the reproducibility crisis and file drawer problems in psychology. Other symmetry and time perception researchers should be aware of this null result. One possibility is that the effect of symmetry on subjective duration is limited to very specific experimental paradigms.

I would consider myself relatively expert in both the topics and methods used in this manuscript. I work on models of human time perception and frequently use human participants in tasks with designs and analyses similar to those used here. I re-read the Ogden et al. 2016 paper in preparation.
Consult also the following for alternative/related checklists: 10.1038/s41562-019-0772-6 10.31234/osf.io/edh2s 10.1177/2515245918806489 Transparency (30/50) 1. Are the data presented in the paper available on a public repository? (make sure this is noted in "availability statement" or similar) yes/no/comment The data are available at a linked osf project site, which is great. The data are in a .xlsx file, with the different data (along with summary and figures) for each experiment on different tabs of the file. Correct me if I'm wrong, but these appear to be the only the cell-wise averages? It would be helpful if there were more notes/comments about what the presented data are etc, so that someone grabbing the data can immediately understand what they are looking at without any guesswork. If it is indeed the cell-wise summary, it would be better it the raw trial by trial data were also included.
2. Is the code to reproduce the results available with the data? (make sure this is noted in "availability statement" or similar) yes/no/comment As above, the analyses to produce summary statistics are present on the .xlsx file, but the relation to the raw trial by trial data is not present. I additionally note that the authors have provided the materials for running the experiment. I didn't check whether these work or are accurate to the provided description. There is a justification based on partial-eta-square. This is good. I would like to see a more thorough explanation of the process and justification here. Daniel Lakens had a nice blog post (and follow up paper) several years ago about how no one talks about power and so no one knows about it. It would be great to provide more details (and include materials on osf) that support the provided justification. In a very brief way, yes. I understand the motivation to present the paper in view of a metascience position, potentially for fear that a reviewer would reject on the basis of not being novel. Given the context and the place it has been submitted, I would consider providing more background on the original motivation of the study. Little is given in way of justification for looking at number of folds and it seems this was the original motivating factor. I understand that this kind of comes to nothing given the results, but given that it seems you set out to investigate this (rather than looking to disconfirm the previous results) some background motivating your thinking is probably warranted.
14. Are the presented hypotheses based on theories or assumptions that are reasonable or supported by the literature? 1 The presented hypothesis is quite narrow (perhaps reflecting my comment from directly above). The details relating both to time and gestalt are a little murky. The "arousal" idea for internal clocks is just a description of the results, in the literature (including the cited literature) always an ad hoc interpretation. Maybe again reflecting that the motivations aren't made entirely clear for this specific study, but the idea that these particular stimuli are going to be useful for this case because of their "perceptual goodness" (bottom of p.5) is not clearly justified to my reading.
15. Is the presented literature review a fair reflection of the existing literature? 2 yes/no/comment In so far as the metascience issues, I think so. As above, I think it would be appropriate to balance this discussion with the motivation for the study further. But what is there is fair.
16. Is previous work cited appropriately? 3 yes/no/comment As far as I can tell. I didn't follow through with every reference but didn't see anything inappropriate.

Results (25/35)
17. Have the analysis methods been applied correctly (as specified) as far as you can tell? 4 yes/no/comment As mentioned above, it would be good to have more clarity on how power was calculated.
Is it necessary to conduct both Shapiro-Wilk and K-S test? K-S test is potentially problematic for empirical distributions that are not necessarily normal (when comparing with normal) and is problematic (generally unreliable) for samples of this size.
A quick glance at the Fs and associated p-values seems correct but I didn't explicitly check whether these were correct.
18. Is the paradigm appropriate for testing the hypothesis? 5 yes/no/comment Again, as the hypothesis appears to be quite limitedis there evidence for an effect of "folds" on duration judgements. With this very limited question, then yes. How this extends to the other questions presented in the previous papers, I guess the authors are deliberately staying away from this.
19. Are there any clear methodological confounds/errors in the results? (provide a full explanation of why, citing references where necessary).
yes/no/comment I guess there is an issue conceptually with concluding in favour of the null in any substantive, broader way here because it is effectively a conceptual replication. Perhaps it is trivial in this instance, but there is no explicit manipulation check (on random vs reflection) presented here. There is also no contrast with the previously used stimuli (e.g. Ogden et al. 2016). Sure, this might be trivial in that they are very easy for participants to detect which one is the symmetrical one and which is not, and the salience of the symmetry is similar to previous studies, but there is no evidence presented for this which makes any stronger conclusion tricky (related to comments on discussion below). I think to make strong comment about whether this is indeed a failed replication or simply doing something slightly different (which reflects on the broader notion of visual symmetry) that this kind of information is necessary.
20. Are there any analyses of the presented data that you can think of that have not been conducted and would potentially reflect alternative interpretations? (provide a full description of the additional analyses required, citing references where necessary, along with explicit prediction of the results, and explanation of why these results are expected).

yes/no/comment
As above, very tricky given the "null" result. But the presented evidence is very straightforward. The larger concerns would be about manipulation checks. yes/no/comment I think again here it is difficult. The authors acknowledge that the stimuli used here are different from previous studies, but then assert that they are more salient (second paragraph, page 11). As above, I'm generally happy to accept this is true, but a stronger statement would be to show evidence that these are as salient, or at least "sufficiently" salient, whatever that might mean.
24. Are there any other relevant relationships between the results observed in this study and other literature that have been overlooked by the authors? 6 yes/no/comment Nothing that comes to mind immediately. Again, the contrast conducted is very simple and there is no evidence that the manipulations (other than duration) have any effect. So the only thing to again mention is about symmetry in general and whether the manipulation was effective.
Breaking the rules of the template slightly here but possibly relevant to the issue of whether changing the report task is this paper https://psyarxiv.com/pg7bs/. This is not a request for citation, just a note related to the fact that you seem to be trying to defend that the analog scale versus verbal estimation wouldn't necessarily be a problem.

Wildcard (0/5)
Does the paper develop/deploy a method in a new and/or interesting way? yes/no/comment Does the paper provide a link between known results in an interesting way? yes/no/comment Is there generally something about this paper that makes you go "Wow, I wish I had thought of that"? yes/no/comment

Minor comments
Please identify basic mistakes that you notice like typos, grammatical errors, or sections of text that could otherwise be expressed more clearly. Provide suggested alternative phrasing if you wish.
1. On page 7 when describing the stimulus "The SD of dots…". Consider not using abbreviation for clarity.