PONE-D-19-20674

Unconscious categorization of sub-millisecond complex images

PLOS ONE

Dear Mr. Beauny,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

All three reviewers found your manuscript interesting and novel. However, they have also found a number of methodological issues that need to be addressed before the paper is ready for publication. Enclosed are the views of the experts to whom I turned for evaluation of your manuscript (one of them is Julian Matthews).

One particularly important issue is data availability – if you could share the data as a supplementary file or deposit it on a one of the research data sharing platforms, that would be great.

We would appreciate receiving your revised manuscript by Jan 17 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Magdalena Ewa Król, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For more information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially sensitive information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

3. We note that Figure 2 in your submission contain copyrighted images. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

1.    You may seek permission from the original copyright holder of Figure 2 to publish the content specifically under the CC BY 4.0 license.

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

2.    If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors report the findings of a psychophysical experiment aimed at assessing unconscious perception of complex visual scenes. Critically, the authors employ a tachistoscope capable of precise, sub-millisecond stimulus presentation, and outline a novel method for inferring unconscious perception by 'reductio ad absurdum' that combines perceptual awareness ratings and psychometric threshold comparison.

Their method takes as a starting point that objective performance on a task should be explainable by performance on only those trials that a participant reports being conscious (on the conservative assumption that unconscious performance is at chance-level). The authors apply this logic to a novel analysis technique for inferring conscious performance on the basis of subjective and objective performance thresholds (“surplus coefficient” or SC). Using SC, the authors reveal that the conservative position that unconscious performance is at chance-level cannot hold. Critically, “the proportion of trials for which participants reported being able to identify the content of the visual scene was not sufficient to explain global objective categorization over all trials.”

In general, the authors do a fantastic job of outlining the existing literature in their introduction, the assumptions their study adopts, and the novelty their method brings. The authors employ a tachistoscope to target two types of conscious perception: detection and identification. This procedure allows them to psychophysically distinguish two thresholds of visual awareness with respect to urban and natural scenes. Firstly, the threshold between "not seeing" and "seeing", and secondly, the threshold between "not conscious of content" and "conscious of content". I believe this procedure is a worthwhile contribution to this literature. However, I have reservations about the other major element of their paper—the novel method for inferring unconscious perception. This is why I consider the data to only partly support their conclusions.

Major issue:

My concern is the derivation of “surplus coefficient” (SC). The goal of this analysis is to determine the theoretical level of performance on “conscious” trials that would be required to explain “objective performance”, assuming chance-level performance on “unconscious” trials. The upshot of this process is that when SC is greater than 0, it is impossible to explain “objective performance” on the basis of conscious trials alone because “conscious performance” needs to be greater than 100% accurate. This logic seems to hold in the context of this experiment when we consider conscious “detection”. This is because the target of a detection report is the mere perception of a stimulus—we can assume conscious performance will be roughly equivalent for all stimuli on conscious trials. This is not the case for conscious “identification”. In this context, the target of the report is one of 160 urban or 160 natural scenes. It is very likely that this judgment is easier or harder for certain stimuli, even under the assumption that conscious access is a dichotomous process.

Given this, it seems mistaken to assume “conscious performance” will be fixed for all conscious trials (and vice versa for “unconscious performance”) when deriving SC (“we verified whether conscious performance could, on its own, explain objective performance”). Despite this concern, I think it’s unlikely stimulus variability entirely discounts the authors’ findings in the “identification” condition. However, it will affect the replicability of this study and extensions were a different set of stimuli used. This issue should be addressed when discussing the results and when using the phrase “direct unconscious perception” in their Discussion - it seems SC , in its present form, is not quite appropriate in the case of conscious identification.

Stimulus variability is a common issue in the psycholinguistic literature that is accounted for by treating stimuli as a random factor in the error term of a mixed model. This approach may be appropriate here, for instance modelling objective performance as a function of psychometric thresholds, slopes, and conscious and unconscious performance as well as including stimuli and subjects as random factors. Another advantage of this approach is that it allows the inclusion of all trials in the model, and would be robust to changes in the proportion of conscious and unconscious trials + changes in threshold as the experiment progresses.

Minor issues:

As per PLOS ONE guidelines, “contacting author” is not sufficient data availability. I recommend uploading the data to Figshare in .csv format along with a ‘readme’ .txt document that describes each variable.

I was a bit uncertain about the response process and timing in Phase 2, especially for stimulus “present” trials. My understanding is that participants made a “natural” vs. “urban” judgment and a subjective report on every trial (i.e., “absent”, “catch”, and “test” trials). The critical difference was that on “catch” trials the presentation time was determined from responses in Phase 1. But was this the time corresponding to “detection” or “identification” reports?

Line 201 “same stimuli stimulus” —> “same stimuli”

Line 307 “detecting and identify” —> “detecting and identifying”

Line 308 “on the subjective as” —> “on the subjective scale as”

Line 387 “Data are simulating” —> “Simulated data”

Line 399 “same than objective” —> “same as objective”

Line 412 “below than 60% correct” —> “below 60% correct”

Line 421 “was of 0.50” —> “was 0.5”

Line 515 “and of false” —> “and false”

Line 538 “enabled to investigate” —> “enabled us to investigate”

Reviewer #2: This paper describes a psychophysical experiment on unconscious categorization of complex visual images (natural vs. urban scenes) presented with exposure durations in the submillisecond range using a custom built LCD-tachistoscope. Subjective perception (detection/visibility) was evaluated in a first phase (no objective task), followed by a second phase (objective task) where participants had to decide what the stimulus was on each trial (forced choice categorization). In this second phase, subjects had also to perform after each trial the same subjective task as in the first phase (absence of stimulus, stimulus detection or stimulus identification). The authors explored the distribution of responses on the subjective scale for each stimulus exposure duration, through psychometric fitting, and compared it to objective responses.

Results indicate that participants are able to correctly decide about the content of a stimulus (that is, to correctly categorize it) even when they fail to identify the contents per se, thus suggesting direct unconscious perception (for this specific setup using complex visual scenes).

This study provides interesting novel results that have implications on the understanding of conscious and subliminal perception when using complex visual stimuli. The paper is well written, and the Methods and Results are thoroughly described.

I think the discussion could be improved by considering other results from other papers that have used a similar approach to render the stimuli subliminal. There is at least one other paper (to my knowledge) that has used a similar device (https://onlinelibrary.wiley.com/doi/10.1002/hbm.22716) with submillisecond exposure duration. In this experiment the authors used rather simple visual stimuli (checkerboards) and found that although the checkerboards were undetectable on a conscious level for the 250 microsecond duration (see their experiment 3), yet they still evoked a brain response (even if a small one only). The authors should relate their present findings to this, at least with respect to the method used.

Reviewer #3: This study presents an experiment in which subjects need to subjectively evaluate the visibility (phase 1) of natural or urban scenes. In a next phase, the need to objectively categorize these scenes and also give a subjective visibility evaluation. The surplus coefficient is introduced as new measure to explain the proportion of objective performance that cannot be explained by the subject’s performance on conscious trials only. The results show that subjects can classify scenes even when they fail to subjectively fail to see the contents of the stimulus.

Overall, I found this an interesting study to read with some intriguing results. However, I am not convinced that the use of the surplus coefficient is bringing a lot of novelty or a better measure to the field of consciousness research. I listed my comments below.

- I find I it hard to fully appreciate the surplus coefficient (SC) as the dependent measure. While I get the intuition of the SC, some aspects still remain obscure to me after several readings of the analysis paragraphs. I understand that what is called global performance in Figure 5 is the objective performance, and that the % conscious trials in the figure corresponds to the subjective performance (it would help if there is more consistency in the naming). If I understand correctly, the counterintuitive suggestion of an objective performance that needs to be higher than 100% is needed to fit equation (3) and because unconscious performance (UP) is assumed to be equal to 50% in this equation. When looking at the simulations, however, I get the impression that the authors could use a simpler measure than this. If it is assumed that consciousness is an all-or-none phenomenon, the threshold for subjective/objective consciousness can be calculated as the inflexion point of the logistic regression curve (see Del Cul et al., 2007, PLoS Biol). According to my understanding of the surplus coefficient it should be highly similar to the difference in these thresholds: If the objective threshold is lower than the subjective threshold, this suggests that there is some additional – unconscious - processing taking place without reaching the subjective threshold (or a positive SC; Fig. 5D). In the case of an ideal observer (Fig.5C), the objective threshold is equal to the subjective threshold (the difference between the two is 0; or the SC is undetermined), suggesting that there is no additional unconscious processing. In the unlikely case of a negative SC (Fig. 5B), the subjective threshold is lower than the objective threshold which is also highly unlikely as it would mean that subjects can’t objectively classify information that they subjectively label as conscious. The two measurements (difference between thresholds and the SC) seem to behave the same. Because the use of the SC as a new metric is one of the most important claims of this study, it would be nice to understand why the authors constructed this new measure rather than to directly compare the thresholds and why is it beneficial to use the SC. The results with the SC appear to be very much in line with more traditional analysis approaches (e.g., aren’t the results presented in Fig. 6 giving you the exact same conclusions about unconscious processing?).

- In the discussion, the authors write that their results show that unconscious performance is only observable when participants knew there was something on the screen (lines 592-597). This statement appears to be rather paradoxical - a conscious sensation is necessary to have unconscious processing? – and needs to be rewritten. Also, later in the discussion it is written that the current results strongly favor unconscious performance what seems to be contradicting what was written before. The authors need to spell out more in detail what they exactly refer to: Objective identification appears to be possible without subjective identification, but is impossible without subjective detection, or something like this.

- What is missing from the experiment is an objective detection task. It would have been informative to see how this thresholds relates to the subjective detection threshold and the objective identification threshold.

- Abstract, line 31: “without being to identify it”. Missing word.

- Page 9, line 201: “…stimulus stimulus…”. Remove stimulus.

- Page 19, line 412: “… who were below than 60%...”. Remove than.

- Page 19, line 420-421: “…mean objective performance was of 0.50…”. Remove of.

- Figure caption of figure 9 says that the yellow curve represents the proportion of correct responses on trials reported as having been subjectively detected. Is this not just the proportion of trials indicated as subjectively seen? Same for the red curve.

- Page 26, line 593:”…our results fail to show the necessity of unconscious categorization…”. Strange sentence.

- Page 26, line 600: It is concluded that subjective reports were less sensitive in phase 2 what would suggest that performing the objective task biases subjective responses. However, only a change in the slope was observed and no change in the threshold. Does this then reflect a change in bias?

- Page 27, line 623: “…a hypothesis that we infirm.” Should be confirm.

- Page 27, line 626: “…that performance on subjectively identify trials..”. Should be rewritten.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Julian Matthews

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

Unconscious categorization of sub-millisecond complex images

PONE-D-19-20674R1

Dear Dr. Beauny,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. 

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. At this point, if you could have a look at the few remaining comments of Reviewer 3.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Magdalena Ewa Król, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have revised their manuscript which is now clearer, addresses my comments, and appears to account for comments raised by my fellow reviewers. I also note that their data is now uploaded to the OSF with a comprehensive readme file.

One paper the authors may wish to raise when discussing the impact of image variability on subjective thresholds is Rahnev & Fleming (Neuroscience of Consciousness, 2019). The analysis of Rahnev & Fleming demonstrates that stimulus variability (albeit for simple gabor patches) introduced during staircase procedures can improve metacognitive ability (the correspondence between objective detection reports and subjective confidence ratings). Although there are several major differences in research design, the results of Rahnev & Fleming may lend further support to the conclusions in the present study. Assuming a tight relationship between conscious access and metacognitive ability, stimulus variability (rather than a limitation) may have enhanced subjective thresholds relative to conditions with low stimulus variability. This means the SC-estimates in the present study are possibly more conservative than they might be were stimulus variability controlled more precisely.

In any case, this is a fine addition to the literature on unconscious perception.

Reviewer #3: All of my previous comments have been addressed.

Some minor comments:

I would change stimulus intensity to stimulus duration. Intensity can also be varied by changing brightness, for example, and keeping the duration constant. Because this experiment explicitly manipulates duration, it should be written accordingly.

Line 614: “… a hypothesis that we infirm.” Infirm?

Line 647: ‘…performance on subjectively identified [identify] trials…”

The order of the figures is very confusing in the pdf but the figure numbers do not correspond anymore to the references in the text. This should be checked again. Also in the response to my comment, a reference is made to figure 5 but I guess this should be figure 4 in the revised manuscript.

From line 488 onwards, ‘hits rates’ should be changed to ‘hit rates’.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Julian Matthews

Reviewer #3: No