Peer Review History
Original SubmissionNovember 30, 2021 |
---|
PONE-D-21-37932Using RShiny to Develop a Dashboard using IPEDS, U.S. Census, and Bureau of Labor Statistics DataPLOS ONE Dear Dr. Perkins, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript within two months. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Barbara Szomolay Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 2. Please note that PLOS ONE has specific guidelines on software sharing (http://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-software) for manuscripts whose main purpose is the description of a new software or software package. In this case, new software must conform to the Open Source Definition (https://opensource.org/docs/osd) and be deposited in an open software archive. Please see http://journals.plos.org/plosone/s/materials-and-software-sharing#loc-depositing-software for more information on depositing your software. 3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide. 4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: No Reviewer #2: Yes ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: N/A ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: I thank the authors for the opportunity to read their paper. Overall, I appreciate their contribution and all the effort placed in the development of this dashboard. I agree with their assessment, so far, I am not aware of any interface dashboards using IPEDS, USCB, and BLS. In the following lines I offer some reactions but overall, I agree with their coding schemes. My main feedback would be to clarify the purpose and scope of the study. Their dashboard is functional, but it is not clear why would this paper need to be published at PONE? Are the authors interested in teaching researchers to develop similar applications? Are they interested in easing access to these data sources? If the latter is true, I recommend adding a functionality to download the data merged/compiled from their platform. This would be useful for the incorporation of these indicators from a multiplicity of data sources. I can definitely see master’s students taking advantage of this resource. Other questions that emerge are, why are their merging approaches limited to two institutional types (public 2- and 4-year colleges)? I am also wondering if the authors have considered relying on the college scorecard (https://collegescorecard.ed.gov/data) data for this resource has been standardized and compiles other data sources. This comment is based on the fact that the IPEDS data need to be standardized when conducting cross-sectional analyses. Although I understand that the current application relies on one academic year, feeding the application from the college scorecard may be an nice addition. In sum, I believe that the ability to download these data may be an important addition to their dashboard. Reviewer #2: I found this application very interesting and I applaud the effort to publish the methods behind developing this {shiny} application. These types of dashboards are vital for the research community and often do not receive the academic credit they deserve. Below are some of my comments and concerns. 1. I'm not sure the purpose of the paper is entirely clear: is it to describe how someone might create a dashboard like this? Or is it to describe the contents of this particular dashboard? At the moment it seems like a bit of a blend, but I found that hard to follow since the audience for the two uses would likely be quite different. If possible, I would try to narrow the focus to one or the other. If trying to describe *how* someone could build a dashboard like this, there needs to be more focus on how to develop a shiny dashboard. For example, the basic concepts like the `ui` and `server` were not well explained in the text. Similarly the `input$` `output$` system in {shiny} was not explained. Additionally, if this is the focus, less is needed on the background about the particular datasets chosen to integrate here (since presumably the next user would not be building this exact dashboard), and more attention could be paid to the general concepts of downloading, munging, and merging data together (with these datasets as a particular example, but not the main focus). If, on the other hand, the purpose of this paper is to *describe* the contents of this particular dashboard, then the focus on the code is not as necessary. 2. It is not entirely clear to me what the purpose of this application is. Is it just to allow users to explore this data? If it is for analysis (but presumably for people who are less keen on pulling all of the data themselves) it seems like it would need to have the ability to subset the large datasets for download. Currently, it seems all the user can do is calculate the correlation between two variables or compare a few variables between institutions. 3. The R code was hard to read and a bit inconsistent style-wise. I recommend using a linter to keep the code consistent (for example: https://github.com/r-lib/lintr) 4. I'm not sure "RShiny" is how RStudio would refer to this product (I think just Shiny for the product {shiny} for the package?). 5. Since the purpose of the paper is not entirely clear, I'm not sure if this review includes a review of the dashboard itself, but if so, here are a few comments: * On the "Institution map" page it says to "use the search box", however I do not see a search box on this page (other than the magnifying glass on the map itself?) * On the "Historic Enrollment" page, the x-axis could be cleaned up to just say 2015, 2016, 2017, 2018 (instead of `2015_Enrollment` etc.). Additionally, when more than one school is added, the numbers are completely obsured by the points on the graph. Either the points could be removed (and replaced with just the numbers), the numbers could be moved up a bit, or the user could just hover to see the numbers. * On the "Demographics" page, the y-axis could be cleaned up (to remove the underscore and also the word "Percent" since it is redundant with the x-axis). The ordering of the bars could be improved (maybe ordered by frequency based on the top selection?) to make it easier to read for a viewer. It is also hard to compare categories between institutions - maybe position_dodge rather than a facetted chart would make sense? Otherwise, perhaps flipping the direction so the categories are aligned across institutions to make it easier to compare between them. (The same for the Graduation and Retention tab) * In the "Dynamic Scatterplot" page, it is not clear why the points are colored by location type? It also would be nice to have a direct link on this page to the Data Dictionary since that is necessary to know what the X and Y variables are. Additionally, it would be nice to output the results in a Tabel rather than the `lm` output directly, if possible, since the intended audience is likely not familiar with R (it seems). * In the "Correlation Coefficients" page "variable" is forced to stay in the selection. It is trivial to force this in the dataset without relying on the user to not delete it (create a string of the names to pass to the selectizeInput that removed "variable" from the choice and similarly when picking the variables on the server side add "variable" back in). It would also be nice to have a direct link to the Data Dictionary from this page. ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. |
Revision 1 |
PONE-D-21-37932R1Using RStudio to Develop a Dashboard using IPEDS, U.S. Census, and Bureau of Labor Statistics DataPLOS ONE Dear Dr. Perkins, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Nov 18 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Sathish A.P. Kumar Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #2: (No Response) Reviewer #3: (No Response) ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #2: Yes Reviewer #3: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #2: N/A Reviewer #3: Yes ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #2: Yes Reviewer #3: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #2: Yes Reviewer #3: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #2: # Review I appreciate the updates, I have a few additional comments, please see below. 1. In the abstract, it looks like "R Shiny" was replaced with "RStudio" -- this may have been due to my previous comment be unclear -- I think this should say "R shiny dashboard", not "RStudio dashboard" 2. There are inconsistencies in how Shiny is referred to (for example RShiny, R shiny, RStudio, shiny etc) I would recommend replacing these all with "Shiny" when referring to the application and "shiny" when referring to the package. 3. The detailed background on the datasets does not seem necessary, particularly the historical context. The information on what the datasets provide, however, seems relevant. 4. I found the following explanation of a pipe confusing: "You’ll also notice the use of “%>%” in the code. That is a pipe that basically tells R to use the dataset that was previously stated, or to continue processing given what was done before." Perhaps something like: "The %>%, known as the "pipe" is a function from the magrittr package. It takes the object on the left hand side and "pipes" it into the first argument of the subsequent function. For example institutioninformation %>% select(UNITID) is equivalent to select(institutioninformation, UNITID). A benefit of the pipe is it can allow the code to be more readable than a series of nested functions." 5. Perhaps explans what the tidyverse is, i.e. "Once the data are queried, it may be necessary to recode variables using tidyverse, a suite of R packages used to manipulate data frames". 6. There are several times the authors refer to assigning an object in R as "names the table" -- technically this should be something like "a table is created called x" since the process is actually creating the table and naming the object something rather than just assigning a name. 7. The code on page 17 could be reduced to a series of left joins connected by the pipe (rather than creating something named "census" over and over again, ie: census <- left_join(employ, Health, by = "GEOID") %>% rename(GEOID = GEOID.x) %>% left_join(income, by = "GEOID") %>% left_join(TotalWhite, by = "GEOID") %>% left_join(Veteran, by = "GEOID") The same is true for all of the code on pages 18 and 19. It seems unnessesary to use the pipe if you aren't going to chain the functions together (I like the pipe, but if you are using it I would recommend coding like above). 8. In the getting started with shiny section, instead of saying you need to open a "shiny" file (this is not a file type), I would say you need to create a .R file. Small typos: 1. When referring to a function in text, it generally should have the open and closed brackets (or no brackets at all) (i.e obcDriverConnect( should be obcDriverConnect() or obcDriverConnect) 2. Page 15 look says "install. Packages" should be "install.packages" 3. Page 22 "page gives the complete programming language" should read "page gives the complete programming code" 4. Page 25 HTLM -> HTML Reviewer #3: The authors are to be commended on a producing a polished, well-thought out shiny application using R. The manuscript provides a discussion of how the authors coded a dashboard and what coding choices facilitated this. I have a few questions that if answered in the text would strengthen this manuscript as a resource for individuals who would replicate the development of these dashboards: 1. What process did the authors use to select these variables and not others to appear in the dashboards? There are many datapoints left out. 2. A main claim in the paper is that there isn’t a dashboard cited in the literature that combines US federal census, labor and education. Are there any studies that combine this data themselves without a dashboard? This would strengthen the argument that this dashboard was needed and provide a basis to evaluate the effectiveness of this dashboard in future years. 3. Some discussion is needed about the logistics of hosting and maintaining this dashboard. The authors seem to be using a free account on shinyapps.io. What are the limitations as the dashboard is used more frequently for this approach? Did the authors consider other deployment paths? (i.e. was cost the only factor, or were other deployment options evaluated). 4. What plans are there to maintain the data set and what is the workload? i.e. what effort is needed to add 2020 data and beyond? How long will that take given the initial code is in place? The response to a prior reviewer comment about 2-year or 4-year institutions I think misses the mark. This is a somewhat misleading variable in IPEDS that actually can lead to wrong conclusions if a researcher is unaware what it represents. IPEDs codes institutions as 4-year institutions if they have any bachelor’s degree programs. 2-year institutions (in the IPEDS data set) are those institutions with only associates degrees. Most times this is not actually what a researcher actually wants and excludes nearly all community & technical colleges in some states that award primarily associates degrees and less than two year certificates, but have a few bachelor’s degree programs. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #2: No Reviewer #3: No ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. |
Revision 2 |
Using R Shiny to Develop a Dashboard using IPEDS, U.S. Census, and Bureau of Labor Statistics Data PONE-D-21-37932R2 Dear Dr. Perkins, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Sathish A.P. Kumar Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #3: (No Response) ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #3: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #3: N/A ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #3: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #3: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #3: The authors have addressed my feedback in the prior review. I have no other feedback for this paper. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #3: No ********** |
Formally Accepted |
PONE-D-21-37932R2 Using R Shiny to Develop a Dashboard using IPEDS, U.S. Census, and Bureau of Labor Statistics Data Dear Dr. Perkins: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Sathish A.P. Kumar Academic Editor PLOS ONE |
Open letter on the publication of peer review reports
PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.
We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.
Learn more at ASAPbio .