Figure 1.
Observed frequency of Fletcher's usage of the word ‘’ in his 15 plays, compared to that of the 153 plays by other authors in the text corpus dataset.
A significantly higher frequency of ‘’ usage by Fletcher is demonstrated, indicating ‘
’ as an appropriate choice of marker to assist in the classification of his plays. Fletcher's predilection for the word ‘
’ has been previously shown by Hoy [21].
Figure 2.
Observed frequency of Shakespeare's usage of the word ‘’ in his 28 plays, compared to that of the 140 plays by other authors in the text corpus dataset.
A significantly lower frequency of ‘’ usage by Shakespeare is demonstrated, indicating ‘
’ as an appropriate choice of marker to assist in the classification of his plays.
Table 1.
List of methods and their types.
Table 2.
20 highest ranking words by CM1 score (presented in descending order of score), for Fletcher, Jonson, Middleton and Shakespeare.
Table 3.
20 lowest ranking words by CM–1 score (presented in descending order of score), for Fletcher, Jonson, Middleton and Shakespeare.
Figure 3.
CM–1 scores for the 50 highest and 50 lowest ranked words for Fletcher, based on the 168 plays in the text corpus dataset.
The 20 highest and 20 lowest ranked words are shown in red and green respectively, and are presented in Tables 2 and 3. As expected, the CM–1 score for ‘’ is significantly higher than that of any other marker word.
Figure 4.
Difference between the cumulative CM–1 scores for Fletcher's 20 highest and 20 lowest scoring marker words, as presented in Tables 2 and 3.
Fletcher's plays are highlighted in green. It is observed that the majority of his plays score considerably higher than the majority of plays by the other authors. One notable exception, The Faithful Shepherdess, is considered to be of a significantly different genre to the remainder of Fletcher's plays, and has been omitted in two previous studies attempting to identify his stylistic signature [21], [36].
Figure 5.
CM–1 scores for the 50 highest and 50 lowest ranked words for Jonson, based on the 168 plays in the text corpus dataset.
The 20 highest and 20 lowest ranked words are shown in red and green respectively, and are presented in Tables 2 and 3. CM–1 ranks ‘’ and ‘
’ as words that Jonson distinctively overuses, in contrast to ‘
’ and ‘
’, which are distinctively underused.
Figure 6.
Difference between the cumulative CM–1 scores for Jonson's 20 highest and 20 lowest scoring marker words, as presented in Tables 2 and 3.
Jonson's plays are highlighted in green. Although not as evident as with Fletcher, Jonson's plays demonstrate an overall higher score than the majority of plays by the other authors. The worst scoring of Jonson's plays, The Case is Altered, is generally regarded as a stylistic anomaly among his works [35].
Figure 7.
CM–1 scores for the 50 highest and 50 lowest ranked words for Middleton, based on the 168 plays in the text corpus dataset.
The 20 highest and 20 lowest ranked words are shown in red and green respectively, and are presented in Tables 2 and 3. CM–1 ranks ‘’, ‘
’, ‘
’ and the demonstrative form of ‘
’ among the words that Middleton distinctively overuses; ‘
’ is ranked amongst the words that Middleton underuses, as opposed to plays by Jonson, for which ‘
’ is a strong positive marker.
Figure 8.
Difference between the cumulative CM–1 scores for Middleton's 20 highest and 20 lowest scoring marker words, as presented in Tables 2 and 3.
Eight of these marker words appeared among the ten word-variables determined earlier by Craig [34] (by discriminant analysis). Middleton's plays are highlighted in green. It is observed that the majority of his plays score higher than the majority of plays by the other authors. The worst scoring of Middleton's plays, A Game at Chess, is unusual stylistically among his works, being a satire on contemporary international politics [32], [33].
Figure 9.
CM–1 scores for the 50 highest and 50 lowest ranked words for Shakespeare, based on the 168 plays in the text corpus dataset.
The 20 highest and 20 lowest ranked words are shown in red and green respectively, and are presented in Tables 2 and 3. CM–1 ranks ‘’, ‘
’ and ‘
’ as words that Shakespeare distinctively overuses, in contrast to ‘
’ (as discussed by Craig [29]), ‘
’ and the infinitive form of ‘
’, which are distinctively underused.
Figure 10.
Difference between the cumulative CM–1 scores for Shakespeare's 20 highest and 20 lowest scoring marker words, as presented in Tables 2 and 3.
Shakespeare's plays are highlighted in green. It is observed that the majority of his plays score higher than by other authors, although the overall range of values is lower than for Fletcher, Jonson and Middleton. This supports previous research suggesting that Shakespeare generally adheres to the norms of the work of his peer group [30].
Figure 11.
Authorship classification performance of 50 methods evaluated in terms of Matthews' correlation coefficient for Fletcher, resulting from a 10-by-10 fold cross validation of his 20 highest and 20 lowest CM–1 scoring marker words.
The results of individual folds are presented in blue/green, with the average performance for each method in red.
Figure 12.
Authorship classification performance of 50 methods evaluated in terms of Matthews' correlation coefficient for Jonson, resulting from a 10-by-10 fold cross validation of his 20 highest and 20 lowest CM–1 scoring marker words.
The results of individual folds are presented in blue/green, with the average performance for each method in red.
Figure 13.
Authorship classification performance of 50 methods evaluated in terms of Matthews' correlation coefficient for Middleton, resulting from a 10-by-10 fold cross validation of his 20 highest and 20 lowest CM–1 scoring marker words.
The results of individual folds are presented in blue/green, with the average performance for each method in red.
Figure 14.
Authorship classification performance of 8 methods evaluated in terms of Matthews' correlation coefficient for Shakespeare, resulting from a 10-by-10 fold cross validation of his 20 highest and 20 lowest CM–1 scoring marker words.
These 8 methods were selected as those which yielded the best classification performance for Fletcher, Jonson and Middleton. The results of individual folds are presented in blue/green, with the average performance for each method in red. The performance across all 8 methods is demonstrated to be above 80%, with the best performing method () yielding classification performance of 99%.
Table 4.
20 highest (left) and lowest (right) ranking words by Welch's -test score for Shakespeare, presented in descending and ascending order of score respectively.
Table 5.
Performance results for the top 5 WEKA models, from the ensemble selected for Shakespeare authorship attribution based on performance against Fletcher, Jonson and Middleton (see Materials and Methods section).
Table 6.
Performance results for the top 5 WEKA models for both CM1 and Welch's -test scores, evaluated in terms of Matthews' correlation coefficient, specificity (precision) and sensitivity (recall).
Figure 15.
Frequency of occurrence of words appearing among the 20 highest scoring marker words for Shakespeare, resulting from a 10 fold cross validation.
This process involved the removal of 10% of plays by Shakespeare (3), and 10% of plays by other authors (14). The 20 highest (left) and lowest (right) scoring marker words were calculated for every possible triplet of removed plays by Shakespeare ( combinations), and for each, a random selection of 14 plays by other authors. The marker words determined across the full text corpus are highlighted in green. This demonstrates this selection of words as valid for classification, and that the CM–1 score is robust against the removal and addition of plays.
Figure 16.
Difference between the cumulative CM–1 scores for the 20 highest and 20 lowest scoring marker words for Fletcher, Jonson, Middleton and Shakespeare.
For each author, the left box represents the distribution of scores for their plays, and the right box the distribution of scores for plays by all other considered authors. The worst scoring play belonging to each author is indicated by a green cross. These are: a) The Faithful Shepherdess (Fletcher); b) The Case is Altered (Jonson); c) A Game at Chess (Middleton); and d) Love's Labour's Lost (Shakespeare).