Analyzing the relationship between productivity and human communication in an organizational setting

Though it is often taken as a truism that communication contributes to organizational productivity, there are surprisingly few empirical studies documenting a relationship between observable interaction and productivity. This is because comprehensive, direct observation of communication in organizational settings is notoriously difficult. In this paper, we report a method for extracting network and speech characteristics data from audio recordings of participants talking with each other in real time. We use this method to analyze communication and productivity data from seventy-nine employees working within a software engineering organization who had their speech recorded during working hours for a period of approximately 3 years. From the speech data, we infer when any two individuals are talking to each other and use this information to construct a communication graph for the organization for each week. We use the spectral and temporal characteristics of the produced speech and the structure of the resultant communication graphs to predict the productivity of the group, as measured by the number of lines of code produced. The results indicate that the most important speech and network features for predicting productivity include those that measure the number of unique people interacting within the organization, the frequency of interactions, and the topology of the communication network.

I do not have a sufficient understanding of voice measures to comment on the validity what seems to be heroic work by the authors to generate useful measures of communication networks and speech characteristics.
While I think a better understanding of how communication relates to productivity in organizations is a very important question, I am not convinced this study advances our understanding of this question. There are several reasons for this, which I separate out below.
-------------------------------------------------1. The authors do not provide sufficient information about the organization they are studying, except that the work being performed by employees is software programming. The authors argue that the type of work being performed should affect the relationship between communication and productivity, which I agree with, but so should, for instance, the organizational structure, and the sales or work cycles. From what I can gather on my own, it seems as though the software factory is a student run software consultancy which suggests a very flat organizational structure in which most conversations may be between "equals". It also suggests that work occurs when there is a project available, and otherwise no work occurs. Without understanding how the organization functions and how often work occurs in the organization, it is difficult to know the extent to which the findings generalize to other organizations.
Moreover, the authors mention that their voice data includes conversations between programmers and between programmers and clients. It seems to me like the type of communication that is desirable differs depending on whether a programmer is speaking with a fellow programmer or with a client. I may have missed a discussion of whether the authors drop the client-facing conversations, but I was surprised this wasn't emphasized more. Given that the measure of productivity used in the paper does not directly capture how satisfied a client was or how big a contract the programmers got, I don't think the conversations with clients should be included in the study.

Response:
We have added additional details about the organizational structure in the lines 77-108. We have provided details about the employees, projects and project handling processes. We acknowledge to the reviewer that the organization was indeed flat, with the managing engineer and everybody else on an equal level below.
As the authors point out, the study is not able to determine whether different types of communication patterns cause more or less productivity, but rather whether they can be useful to predict how well the organization is doing (in this case, how productive workers will be). For instance, communication frequency may be a proxy for how excited or engaged workers are in the job or how frustrated they are with the client's demands. If speech and communication network characteristics are easier to observe and measure than other things like employee attitudes, that using them to predict productivity seems like it could be a very useful tool for employers to improve performance by helping them catch potential problems before they become disasters.
However, I do not think the authors did enough to explain how feasible or practical it is for employers to adopt this in practice. I imagine many employees would object to being recorded while working, and it may be hard to retain high quality employees if they were required to record themselves while at work. Given the importance of communication frequency for predicting productivity, partial recordings may not be sufficient. Generating useful data out of the voice recordings would also be expensive timewise. Thus, it is unclear to me what the practical takeaways of the paper are.
explain the lack of findings on speech signal features? Some discussion of possible measurement error of these features might be helpful for understanding how much we can take away from the findings.
Response: This could have happened, but it would have been reflected in problems time-aligning recordings for our cross-correlation analyses, and we saw no evidence of it. Also, people involved in SF said that after the first week or so, members tended to forget the recorders. The same has been reported in other studies doing long-term recording of participants. We have mentioned this in lines 117-121 of our limitations section.
-------------------------------------------------5. I am also a bit concerned about mismeasurement of productivity in the lines of code changed may not capture code quality. The authors acknowledge that deletions may or may not improve code quality, but on the converse, additions may or may not improve code quality. At the extreme, if one programmer is particularly frustrated with a co-worker or client, they may write a number of confusing lines that could make it hard for subsequent programmers to successfully edit the code. There are some relatively standard measures of code quality that the authors could have external programmers evaluate the organization's code on (e.g. complexity, coupling).

Response:
We acknowledge lines of code is a measure of productivity, but not the only one, and agree this is a limitation. On the other hand, according to software engineering scholars we contacted, measuring software quality is far from straightforward or settled science. To our knowledge, there are not any gold standards, and what methods exist require qualitative analysis of code, which is not practical for the amount of code in the SF repository. The fact that there was a multitude of projects, varying in fundamental concepts, requirements and programming languages (the coding was done in multiple platforms, for example C, C++, Java, Fortran, Labview, etc.), complicates possible quality evaluation even more.