https://orcid.org/0009-0004-5773-7561
Figures
Abstract
Behavioral pain scales have been helpful for standardized swine pain assessment. However, it is still unknown if observers’ experience influences the scale score. We conducted a pilot study to investigate how three different levels of swine experience influenced how observers scored castration pain in piglets using Unesp-Botucatu Pig Composite Acute Pain Scale (UPAPS). We used a database from UPAPS scores from pigs undergoing surgical castration in a previous study. Scores were attributed by six observers with Little to no experience (n = 2), Some experience (n = 2) and Extensive experience (n = 2). Reliability was estimated using the intraclass correlation coefficient, agreement was investigated by Bland-Altman analysis, predictive capacity was estimated using the area under the curve (AUC), and statistical differences were tested using a regression model. We found that intra-experience levels reliability were satisfactory (Little to no: 0.72, Some: 0.81, Extensive: 0.84), but inter-experience reliability was lower (0.42). Little to no experience observers had poor agreement with other observers, with a bias toward underscoring UPAPS (bias of 0.94 vs. Some, 1.17 vs. Extensive). Predictive capacity was similar between all observers (AUC, Little to no: 71.94%, Some: 76.10%, Extensive: 79.09%, p > 0.05). Regression model confirmed underscoring of Little to no experience observers (mean ± standard error; Little to no: 1.09 ± 0.14; Some: 2.02 ± 0.23; Extensive: 2.25 ± 0.22; p < 0.05). We concluded that minimal experience, as Some experience observers have in the swine industry, is sufficient for them to score UPAPS in a similar way than more experienced observers. The present pilot study supports the enhancement and implementation of UPAPS on farm and laboratory settings by minimally qualified observers, improving swine welfare in the short and long term.
Citation: da Silva GV, Lopez-Soriano M, Pairis-Garcia MD, Trindade PHE (2024) Less experienced observers assess piglet castration-induced acute pain differently than experienced observers: A pilot study. PLoS ONE 19(9): e0309684. https://doi.org/10.1371/journal.pone.0309684
Editor: Ewa Tomaszewska, University of Life Sciences in Lublin, POLAND
Received: May 17, 2024; Accepted: August 16, 2024; Published: September 4, 2024
Copyright: © 2024 Silva et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The database analyzed during this study and the R script were fully included in its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Pain perception is influenced by social, cultural and individual factors, therefore, pain diagnosis in non-human animals may be challenging [1]. Behavioral pain scales have been successfully implemented to reduce variability and improve objectivity in pain diagnosis, as well as promote good animal welfare in farm and laboratory settings [2]. Whether raised for pork production [3] or scientific research [4], pigs undergo painful procedures such as tail docking, teeth resection, surgical castration, ear tagging and notching [5–7]. These procedures result in acute pain experienced by pigs, which compromises individual animal welfare and puts into question the ethical obligation to minimize pain and suffering experienced by animals in our care and responsibility [8].
The Unesp-Botucatu Pig Composite Acute Pain Scale (UPAPS) was developed for diagnosing pain in weaned and pre-weaned piglets undergoing surgical castration [9, 10]. Since its development, UPAPS has been used to diagnose pain and evaluate analgesic drug efficacy [11, 12] and was considered the only scale with high validation evidence to assess acute pain in swine [2]. Although of great importance, all behavioral scales, including the UPAPS, are susceptible to some subjectivity due to observers’ individual experiences [13]. The UPAPS was initially developed and validated by senior researchers with extensive experience in pain assessment, however, in a real-world scenario, pig pain will likely be diagnosed by caretakers or producers with limited to no formal pain assessment training and experience. This phenomenon has been documented in previous work assessing pain in cats using observers with differing backgrounds [13] documenting differences between observers. Scale design, observers’ responding strategies, or familiarity with the species may all be factors in why observers assess pain differently [14]. Gaining insights into these differences from the swine perspective is important not only to ensure pain in pigs is appropriately assessed, but also refining or reducing UPAPS complexity is necessary for different experience levels. However, to date there are no publications addressing how UPAPS scores would differ from observers with little experience compared with the more experienced ones. Hence, the objective of this pilot study was to investigate if three different levels of swine experience (no experience, some, extensive) can influence observers to score surgical castration pain in piglets using UPAPS. We found that observers with less than three months of experience in the swine industry underscore pain in piglets.
Methods
The present study was approved by the Institutional Animal Care and Use Committee of North Carolina State University (#20–113). All piglets were handled following the Guide for the Care and Use of Agricultural Animal in Research and Teaching [15] and all piglets undergoing castration received pain management before the procedure. Data collected for this study was part of a larger project [12], thus contributing to the four R’s of animal experimentation [16, 17]. Reporting is in accordance with the ARRIVE guidelines [18].
Animals, housing conditions and surgical castration
A total of 29 Large White x Duroc cross male piglets (15 litters, approximately 2 piglets per litter) were enrolled in the study. Housing conditions and complete surgical castration procedure description can be found in the previously published study [12].
Pain assessment
Whole-body behavior was filmed continuously for 4 min at three timepoints. Timepoints were chosen to account for three levels of pain severity expressed as previously described [9, 10]. Timepoints included: 1 h before castration (pain-free), immediately after castration (severe pain), and 3 h post-castration (mild to moderate pain) [9–12]. Video records were obtained using a high-definition camera (Sony HDR-CX405®; New York, NY, USA) placed on a tripod approximately 30 cm from the crate at a 122-cm height. A total of 177 videos were obtained (29 per timepoint), 348 min in total, without editing video clips.
The UPAPS is composed of multiple behavioral responses used to diagnose and quantify pain. The UPAPS was previously validated in piglets [10] using five behavioral items specific to posture, interaction and interest in the surroundings, activity, attention to the affected area and miscellaneous behaviors. These behavioral items are descriptive and composed of four score levels: ‘0’ (normal behavior), and ‘1’, ‘2’ or ‘3’, according to the presence of pain-related behaviors (Table 1). Individual items were scored and total pain scores were calculated by summing the five behavioral items into one total score per piglet by timepoint (0–15).
Observers experience
Pain assessment was performed by six observers with three levels of experience, classified as (i) Little to no experience: observers with almost no work on farm or research experience with pigs (n = 2); (ii) Some experience: observers with less than one year of farm or research experience with pigs (n = 2); and (iii) Extensive experience: observers with more than a year of farm or research experience with pigs (n = 2). Little to no experience and Some experience observers were recruited through a veterinary summer scholars research program at the North Carolina State University. All observers underwent training for scoring on the UPAPS, provided by one of the observers with Extensive experience. During training, the trainer provided video examples, definitions, and descriptions of each behavior, and trainees were required to score 20 masked videos of piglets in painful and pain free states (10 videos of each).
Statistical analysis
Data were analyzed using R software within the integrated RStudio environment [19] (Version 4.2.2; RStudio, Inc., Boston, MA, USA). Functions and packages used were presented in the format ’package::function’ corresponding to the computer programming language in R. A significance of 5% was considered for all inference tests. A palette of colors distinguishable by people with common forms of color blindness was used in all figures (ggplot2::scale_colour_viridis_d).
Intraclass correlation coefficient (ICC), two-way random effects model, type agreement multiple observers, and its 95% confidence interval (CI) (irr::icc) were used to evaluate the inter-observer, inter-group and intra-group reliability of the UPAPS total sum for experience level groups (Little to no experience, Some experience, and Extensive experience). The interpretation of ICC was as follows: very good: 0.81–1.0; good: 0.61–0.80; moderate: 0.41–0.60; reasonable: 0.21–0.4; and poor ≤ 0.2 [20].
Bland-Altman test for repeated measures [21] and Lin’s concordance correlation coefficient (CCC) [22] (SimplyAgree::agree_reps) were used to verify the agreement of UPAPS total sum assessed by each experience level group, as proposed previously [23]. As there were three groups and Bland-Altman analysis only allows pairwise comparisons, it was required three analyses to evaluate the agreement between all three possible combinations. Bland-Altman analysis is suitable for detecting bias referring to the difference between two experience level groups [21]. In addition, the Bland-Altman analysis provides the limit of agreement (LoA), which indicates the expected range where most differences between experience level groups should occur [21]. A simple linear regression (stats::lm) was conducted to analyze the proportional bias between experience levels [24]. Proportional bias represents an increase in the difference between experience level groups evaluated at higher or lower UPAPS total sum. The difference in the UPAPS total sum between two experience level groups was used as a response variable, and the average of the UPAPS total sum between the two groups was used as an explanatory variable. Three simple linear regressions were required to cover for the three possible combinations between the three experience level groups. The heteroskedasticity of the linear model was tested with the Breusch Pagan test (olsrr::ols_test_breusch_pagan).
Multilevel negative binomial modeling (lme4::glmer.nb) was employed to investigate the influence of timepoint and experience level (explanatory variables) on UPAPS total score (response variable). The best combination of fixed and random effects were identified according to the lowest Bayesian information criterion (stats::BIC) using preliminary models. Best-fit preliminary model did not consider the interaction between predictive variables, in other words, the dynamic of the UPAPS total sum for each experience level group was same for all timepoints. Piglets nested within each litter were considered as random effects. Bonferroni correction was used for adjusting the multiple comparisons in the post-hoc test (lsmeans::lsmeans and multcomp::cld).
Multilevel binomial modeling (lme4::glmer) was employed to investigate the influence of experience level group (explanatory variable) on UPAPS behaviors as dummy variables (binomial response variable). Dummy variables were created using fastDummies::dummy_columns for each score level. For example: Posture item was transformed into four items: Posture 0, Posture 1, Posture 2 and Posture 3, and each one of them was a binary variable of ‘0’, for when the score is not given, and ‘1’, for when the score is given. Bonferroni correction was also used for adjusting the multiple comparisons in the post-hoc test (lsmeans::lsmeans and multcomp::cld).
Receiver operating characteristic (ROC) curve was fitted to investigate experience level groups discriminative capacities. Response variable was Condition, a new binary variable that assumed ‘0’ when the assessment was performed in a video recorded at 1 h before castration (no pain) and ‘1’ when the assessment was performed in a video recorded immediately after castration (severe pain), similar as conducted previously [10, 25]. The 3 h post-castration timepoint was not considered in this analysis. Explanatory variable was UPAPS total sum. ROC curves were then used to estimate area under the curve (AUC), sensitivity and specificity. AUC were compared statistically using DeLong test.
Results
Reliability
Overall inter-observer reliability was “very good”, intra-experience level reliability was “good” and “very good” and inter-experience level reliability was “moderate” (Table 2). Score distribution of each observer is presented in Fig 1A.
(A) Overall observers scoring on UPAPS total sum boxplot across experience levels. (B) UPAPS total score by different levels of experience: little to no experience, some experience and extensive experience. (Letters indicate statistical differences (p < 0.05) found in the Bonferroni post-hoc test (a>b). UPAPS: Unesp-Botucatu Pig Composite Acute Pain Scale).
UPAPS total sum modeling
Parameters of the negative binomial modeling are presented in S1 Table. In the Bonferroni post-hoc test, Little to no experience level assessed a lower UPAPS total sum than Some experience and Extensive experience level (Fig 1B). Post-hoc analysis also confirmed that a higher UPAPS total sum was present immediately post-castration, followed by 3 h post-castration and lower values 1 h before castration considering all observer levels (S1 Fig).
Bland-Altman analysis
Bland-Altman analysis included three agreement assessments: Little to no experience vs Some experience (Fig 2), Little to no experience vs Extensive experience (Fig 3) and Some experience vs Extensive experience (Fig 4). All three had a high bias, combined with an unbalanced limit of agreement between Little to no experience and other experience levels (Table 3). The concordance correlation coefficient was higher in the agreement between Some experience vs Extensive experience (0.72), and lower between Little to no experience vs Extensive experience (0.44). Agreement beyond the limit of agreement ranged from 9.20 to 17.82% in all agreements (Table 3). Only agreements with Little to no experience level showed proportional bias (p < 0.001, S2 Table), and all three agreements had heteroskedasticity (p < 0.001, S2 Table).
Solid lines represent bias, dashed lines represent the lower and upper limit of agreement, dotted lines represent the 95% and 90% confidence intervals for bias and lower and upper limit of agreement, respectively. Green line represents the simple linear regression line. UPAPS: Unesp-Botucatu Pig Composite Acute Pain Scale.
Solid lines represent bias, dashed lines represent the lower and upper limit of agreement, dotted lines represent the 95% and 90% confidence intervals for bias and lower and upper limit of agreement, respectively. Purple line represents the simple linear regression line. UPAPS: Unesp-Botucatu Pig Composite Acute Pain Scale.
Solid lines represent bias, dashed lines represent the lower and upper limit of agreement, dotted lines represent the 95% and 90% confidence intervals for bias and lower and upper limit of agreement, respectively. Yellow line represents the simple linear regression line. UPAPS: Unesp-Botucatu Pig Composite Acute Pain Scale.
Predictive capacity of experience levels
Extensive experience had the greater AUC of 79.09%, compared to 71.94% and 76.10% from Little to no experience and Some experience, respectively (Table 4). Sensitivity increases along with experience level. Specificity was greater than 91% in all three experience levels. All AUC combinations had no significant (p > 0.05) difference based on DeLong test.
UPAPS behaviors modeling
Parameters of the multilevel binomial regression for each UPAPS behavior were presented in S2 Table. Post-hoc analysis showed experience level differences for each UPAPS behavior (S3 Table).
When comparing observer experience to specific subitems in the scale, Posture 1 was identified and scored most by observers within the Little to no experience category, followed by observers with Some experience and Extensive experience (Fig 5). Posture 3, Interaction 1, Activity 0, and Head below the line of the spinal column (Miscellaneous behavior) were scored less frequently by observers with Little to no experience compared with the other two levels. Interaction 0 (normal interaction), and an absence of behaviors scored within the attention to the affected area and miscellaneous category was more common for observers with Little to no experience and Some experience compared to Extensive experience. Wags tail continuously and intensely was scored the most by observers within the Extensive experience category.
(A) Posture, (B) Interaction, (C) Activity, (D) Attention and (E) Miscellaneous items. UPAPS: Unesp-Botucatu Pig Composite Acute Pain Scale.
Discussion
Advancing pain assessment in swine is an important component for safeguarding animal welfare for livestock used in research and farm settings [8, 26]. The Unesp-Botucatu Pig Composite Acute Pain Scale (UPAPS) is the first validated pain scale used to measure pain in pre-weaned piglets using behavioral assessments scored by observers with extensive species-specific experience [9, 10]. However, to date there are no publications addressing how UPAPS scores would differ from observers with little experience compared with the more experienced ones. Therefore, the objective of this pilot study was to investigate how three different levels of swine experience (no experience, some, extensive) influenced how observers scored castration pain in piglets using UPAPS.
In the current study, three experience levels were evaluated for assessing pain behavior in castrated piglets. During analyzing all six observers combined and comparing paired observers within experience levels, the intraclass correlation coefficient (ICC) demonstrated a “very good” reliability [14]. These results are similar to previous work demonstrating “good” and “very good” ICCs when comparing pain assessment between different observer groups such as cat owners, veterinarians, veterinary students, nurses and cat caregivers using the Feline Grimace Scale [27–29]. It was already expected, as previous studies using UPAPS also demonstrated satisfactory inter-observer reliability [9, 10, 12]. In contrast to the good intra-experience reliability in our study, inter-experience across different experience levels demonstrated only a “moderate” reliability, suggesting pain assessment and diagnosis were influenced by the observers’ experience level with swine. This is likely due to a dilution effect of variance when comparing all six observers simultaneously [30]. The variability when comparing six observers at the same time may be masking the disagreement between observers. In order to more appropriately address this variation, a regression model and Bland-Altman analysis were conducted to further investigate the data.
When comparing pain assessment between experience levels using a regression model and Bland-Altman analysis, Little to no experience observers underscored UPAPS and demonstrated higher bias (greater than one point) [28] and tended to demonstrate greater proportional bias compared to the two other experience levels [21]. This was in contrast to a previous study that observed minimal bias when comparing agreement of the Feline Grimace Scale between cat owners, veterinarians, veterinary students and nurses [28]. In addition to the bias, the limit of agreement (LoA), which was interpreted in relation to the analgesic threshold [31, 32], was unbalanced when comparing Little to no experience observers to Some and Extensive experience observers. Applying this interpretation to our results, the LoA of the little to no experience observers spanned beyond the UPAPS optimal cut-off point of a score equals to 4 [9, 10] and, if used in a clinical or farm settings could make pain diagnosis more difficult, particularly for piglets scoring around the cut-off point. In all comparisons, LoA spanned four, thus indicating that diagnosis around the cut-off point was uncertain between the experience levels. However, when interpreting LoA in agreement with proportional bias, it was thought that LoA should be hyperbolic rather than linear, that was, LoA was wider in the beginning and at the end of the scale points, but narrowed in the middle, where there was a cut-off point.
Deviations in pain assessment scores by Little to no experience observers could also be demonstrated by changes in the ROC curve used to estimate predictive capacity. Although all experience levels had statistically equivalent AUC, sensitivity of Little to no experience observers was lower than the other experience levels and, in a farm setting, would resulted in 13–17% of piglets being misdiagnosed as pain-free (false negative). The results of this study were in contrast with previous work that consistently demonstrated that less experienced observers (i.e. undergraduate students, younger veterinarians and recent graduate students) typically reported higher pain ratings and indicators for analgesic use compared to more experienced professionals [33] in cattle [34–38], dogs [39, 40] and cats [40].
In our study, Little to no experience observers had less than three months of experience working with pigs and these differences may be attributed to the fact that this study was evaluating experience levels with the species and not comparing professional expertise or skill. Less experienced observers like students and early career veterinarians would still likely had extensive experience working with or caring for dogs, cats, horses and/or cattle compared to the limited cultural and logistical opportunities to interact with pigs in the United States, the home country of the observers in the present study. Pigs are not often considered pets or companion animals and opportunities to interact and observe pigs’ behavior are limited. Therefore, comparing the results from this study to previous work might not be appropriate, given non-formal experience with previously assessed species was not addressed.
On the other hand, the results from the current study were similar to work evaluating pain behaviors in rats using the Rat Grimace Scale. Observers with no experience underscored pain behaviors compared to experienced professionals [41], and, as discussed earlier, may be associated with limited casual or informal interaction with rats. Future work should take into account how pain assessment in pigs was influenced beyond experience, and better understand the social and cultural factors that influence our relationships and exposure to the animals we are assessing [1].
Differences between experience levels may also be attributed to scale design. The UPAPS is a questionnaire assessment that needs some level of subjective interpretation by the observer [42]. From an ethological hierarchy [43] standpoint, the scale comprises behaviors more and less complex, which in addition to the scale length, may had reflected in less experienced observers missing behaviors. This was demonstrated by Little to no experience observers tending to score lower on Miscellaneous item, which required attention to and interpretation of multiple behaviors at once. In addition, satisficing is a type of responding bias that occurs when respondents preferably select the first options and are more likely to pay less attention to the latter options [44]. Satisficing would promote underscoring as was found for Little to no experience observers and identifying ways to minimize this bias by simplifying the scales is needed. Recent work from our lab [25, 45] suggests that not all behavioral items are equally weighted and the display of key behaviors such as Wags tail and Posture 1, behaviors that we found differences among experience levels, were important to accurately identify pain states.
Limitations of this study also included some aspects of the experience levels. Because training was provided by an experienced observer, our analysis can only compare if a less experienced observer was able to assess more or less like the experienced observers, however, there is no consensus if naive evaluations are better or worse than those from experienced ones. Thus, the naiveness hypothesis could be further tested in an experiment without training. Another limitation was the low number of observers in each experience level, which can be quite individual. Although we assumed a clear criterion to classify the observers’ experience, in a real-life scenario, the types and intensities of months of experience in the swine industry would differ for each observer. In addition, observers’ general knowledge, age, and personality can contribute to individual variation. This was a pilot study that intended to explore the observers’ experience in the pig acute pain assessment context and can be considered a proposal for research methodology. A larger sample size would be beneficial for estimating differences and addressing diverse backgrounds, cultural and social aspects in future studies.
Conclusion
Observers with less than three months of experience in the swine industry do not assess pigs’ acute pain response using UPAPS behaviors in an equivalent manner to experienced observers. However, Some experience (3–12 months experience working with pigs) would permit equivalent pain assessment and diagnosis compared to those with extensive experience. The development and refinement of behavioral scales to assess acute pain should be designed and applicable by users with various backgrounds. Beginner-friendly or less complex scale could enhance the implementation of acute pain monitoring in farm and laboratory settings.
Supporting information
S1 Fig. UPAPS score 1 h before castration, immediately post castration and 3 h post castration.
(Letters indicate statistical differences (p < 0.05) found in the Bonferroni post-hoc test (a>b>c). UPAPS: Unesp-Botucatu Pig Composite Acute Pain Scale).
https://doi.org/10.1371/journal.pone.0309684.s001
(DOCX)
S1 Table. UPAPS score modeling using negative binomial multilevel regression.
https://doi.org/10.1371/journal.pone.0309684.s002
(DOCX)
S2 Table. Linear regression model parameters for determining proportional bias and Breusch Pagan test for heteroskedasticity.
https://doi.org/10.1371/journal.pone.0309684.s003
(DOCX)
S3 Table. UPAPS behaviors means ± standard errors of the means across experience levels.
https://doi.org/10.1371/journal.pone.0309684.s004
(DOCX)
S2 Appendix. Script in R programming language used for the analyses in this study.
https://doi.org/10.1371/journal.pone.0309684.s006
(DOCX)
References
- 1. Robinson CL, Phung A, Dominguez M, Remotti E, Ricciardelli R, Momah DU, et al. Pain Scales: What Are They and What Do They Mean. Curr Pain Headache Rep. 2024 Jan 1;28(1):11–25. pmid:38060102
- 2. Tomacheuski RM, Monteiro BP, Evangelista MC, Luna SPL, Steagall PV. Measurement properties of pain scoring instruments in farm animals: A systematic review using the COSMIN checklist. PLoS One. 2023 Jan 20;18(1):e0280830. pmid:36662813
- 3. Food and Agriculture Organization of the United Nations. Meat Market Review—Emerging trends and outlook 2022.
- 4. Bergen WG. Pigs (Sus Scrofa) in Biomedical Research. Adv Exp Med Biol. 2022;1354:335–43. pmid:34807450
- 5. Camerlink I, Ursinus WW. Tail postures and tail motion in pigs: A review. Applied Animal Behaviour Science. 2020 Sep 1;230:105079.
- 6. Ison SH, Clutton RE, Di Giminiani P, Rutherford KMD. A Review of Pain Assessment in Pigs. Frontiers in Veterinary Science [Internet]. 2016 [cited 2023 Jul 27];3. Available from: https://www.frontiersin.org/articles/10.3389/fvets.2016.00108 pmid:27965968
- 7. Steagall PV, Bustamante H, Johnson CB, Turner PV. Pain Management in Farm Animals: Focus on Cattle, Sheep and Pigs. Animals (Basel). 2021 May 21;11(6):1483. pmid:34063847
- 8. Rioja-Lang FC, Connor M, Bacon HJ, Lawrence AB, Dwyer CM. Prioritization of Farm Animal Welfare Issues Using Expert Consensus. Front Vet Sci. 2020 Jan 10;6:495. pmid:31998770
- 9. Luna SPL, Araújo AL de, Neto PI da N, Brondani JT, Oliveira FA de, Azerêdo LM dos S,et al. Validation of the UNESP-Botucatu pig composite acute pain scale (UPAPS). PLOS ONE. 2020 Jun 1;15(6):e0233552. pmid:32480399
- 10. Robles I, Luna SPL, Trindade PHE, Lopez-Soriano M, Merenda VR, Viscardi AV, et al. Validation of the Unesp-Botucatu pig composite acute pain scale (UPAPS) in piglets undergoing castration. PLoS One. 2023;18(4):e0284218. pmid:37053294
- 11. Lopez-Soriano M, Rocha Merenda V, Esteves Trindade PH, Loureiro Luna SP, Pairis-Garcia MD. Efficacy of transdermal flunixin in mitigating castration pain in piglets. Front Pain Res [Internet]. 2022 Nov 11 [cited 2024 May 17];3. Available from: https://www.frontiersin.org/articles/10.3389/fpain.2022.1056492 pmid:36438445
- 12. Lopez-Soriano M, Merenda VR, Anderson S, Trindade PHE, Leidig MS, Messenger K, et al. Efficacy of inguinal buffered lidocaine and intranasal flunixin meglumine on mitigating physiological and behavioral responses to pain in castrated piglets. Front Pain Res (Lausanne). 2023 Jun 6;4:1156873. pmid:37346473
- 13. Adami C, Filipas M, John C, Skews K, Dobson E. Inter-observer reliability of three feline pain scales used in clinical practice. Journal of Feline Medicine and Surgery. 2023 Sep 1;25(9):1098612X231194423. pmid:37747309
- 14.
Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A Practical Guide to Their Development and Use. Oxford University Press; 2015. 415 p.
- 15. Federation of Animal Science Societies. Guide for the Care and Use of Agricultural Animals in Research and Teaching. 4th ed. 2020.
- 16. Banks RE. The 4th R of research. Contemp Top Lab Anim Sci. 1995 Jan;34(1):50–1. pmid:16457577
- 17.
Russell WMS, Burch RL. The principles of humane experimental technique. London: Methuen & Co. Ltd.; 1959.
- 18. Sert NP du, Ahluwalia A, Alam S, Avey MT, Baker M, Browne WJ, et al. Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0. PLOS Biology. 2020 Jul 14;18(7):e3000411. pmid:32663221
- 19. R Core Team. R: A language and environment for statistical computing. 2022;
- 20.
Altman DG. Practical Statistics for Medical Research.: Chapman and Hall/CRC; 1990. 624 p.
- 21. Altman DG, Bland JM. Measurement in Medicine: The Analysis of Method Comparison Studies. Journal of the Royal Statistical Society Series D (The Statistician). 1983;32(3):307–17.
- 22. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989 Mar;45(1):255–68. pmid:2720055
- 23. Trindade PHE, Lopez-Soriano M, Merenda VR, Tomacheuski RM, Pairis-Garcia MD. Effects of assessment method (real-time versus video-recorded) on a validated pain-altered behavior scale used in castrated piglets. Sci Rep. 2023 Oct 31;13(1):18680. pmid:37907564
- 24. Ludbrook J. Confidence in Altman-Bland plots: a critical review of the method of differences. Clin Exp Pharmacol Physiol. 2010 Feb;37(2):143–9. pmid:19719745
- 25. da Silva GV, Pivato GM, Peres BG, Luna SPL, Pairis-Garcia MD, Trindade PHE. Simplified assessment of castration-induced pain in pigs using lower complexity algorithms. Sci Rep. 2023 Dec 1;13(1):21237. pmid:38040949
- 26. Anil L, Anil SS, Deen J. Pain detection and amelioration in animals on the farm: issues and options. J Appl Anim Welf Sci. 2005;8(4):261–78. pmid:16436030
- 27. Evangelista MC, Watanabe R, Leung VSY, Monteiro BP, O’Toole E, Pang DSJ, et al. Facial expressions of pain in cats: the development and validation of a Feline Grimace Scale. Sci Rep. 2019 Dec 13;9(1):19128. pmid:31836868
- 28. Evangelista MC, Steagall PV. Agreement and reliability of the Feline Grimace Scale among cat owners, veterinarians, veterinary students and nurses. Sci Rep. 2021 Mar 4;11(1):5262. pmid:33664387
- 29. Monteiro BP, Lee NH, Steagall PV. Can cat caregivers reliably assess acute pain in cats using the Feline Grimace Scale? A large bilingual global survey. Journal of Feline Medicine and Surgery. 2023 Jan 1;25(1):1098612X221145499. pmid:36649089
- 30. Liljequist D, Elfving B, Roaldsen KS. Intraclass correlation–A discussion and demonstration of basic features. PLOS ONE. 2019 Jul 22;14(7):e0219854. pmid:31329615
- 31. Leung V, Zhang E, Pang DS. Real-time application of the Rat Grimace Scale as a welfare refinement in laboratory rats. Sci Rep. 2016 Aug 17;6:31667. pmid:27530823
- 32. Evangelista MC, Benito J, Monteiro BP, Watanabe R, Doodnaught GM, Pang DSJ, et al. Clinical applicability of the Feline Grimace Scale: real-time versus image scoring and the influence of sedation and surgery. PeerJ. 2020 Apr 14;8:e8967. pmid:32322445
- 33. Raekallio M, Heinonen KM, Kuussaari J, Vainio O. Pain Alleviation in Animals: Attitudes and Practices of Finnish Veterinarians. The Veterinary Journal. 2003 Mar 1;165(2):131–5. pmid:12573601
- 34. Huxley JN, Whay HR. Current attitudes of cattle practitioners to pain and the use of analgesics in cattle. Veterinary Record. 2006;159(20):662–8. pmid:17099174
- 35. Laven R, Huxley J, Whay H, Stafford K. Results of a survey of attitudes of dairy veterinarians in New Zealand regarding painful procedures and conditions in cattle. New Zealand Veterinary Journal. 2009 Aug 1;57(4):215–20. pmid:19649015
- 36. Thomsen PT, Gidekull M, Herskin MS, Huxley JN, Pedersen AR, Ranheim B, et al. Scandinavian bovine practitioners’ attitudes to the use of analgesics in cattle. Veterinary Record. 2010 Aug;167(7):256–8. pmid:20710034
- 37. Thomsen PT, Anneberg I, Herskin MS. Differences in attitudes of farmers and veterinarians towards pain in dairy cows. The Veterinary Journal. 2012 Oct 1;194(1):94–7. pmid:22516921
- 38. Moody CM, Niel L, Pang DJ. Is training necessary for efficacious use of the Glasgow Feline Composite Measure Pain Scale? Can Vet J. 2022 Jun;63(6):609–16. pmid:35656525
- 39. Barletta M, Young CN, Quandt JE, Hofmeister EH. Agreement between veterinary students and anesthesiologists regarding postoperative pain assessment in dogs. Veterinary Anaesthesia and Analgesia. 2016 Jan 1;43(1):91–8. pmid:25920011
- 40. Doodnaught GM, Benito J, Monteiro BP, Beauchamp G, Grasso SC, Steagall PV. Agreement among undergraduate and graduate veterinary students and veterinary anesthesiologists on pain assessment in cats and dogs: A preliminary study. Can Vet J. 2017 Aug;58(8):805–8. pmid:28761184
- 41. Zhang EQ, Leung VS, Pang DS. Influence of Rater Training on Inter- and Intrarater Reliability When Using the Rat Grimace Scale. J Am Assoc Lab Anim Sci. 2019 Mar;58(2):178–83. pmid:30755291
- 42. Krosnick JA. Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology. 1991;5(3):213–36.
- 43.
Dawkins R. Hierarchical organisation: A candidate principle for ethology. In: Growing points in ethology. Oxford, England: Cambridge U Press; 1976.
- 44.
Simon HA. Models of man; social and rational. Oxford, England: Wiley; 1957. xiv, 287 p. (Models of man; social and rational).
- 45. Trindade PHE, de Araújo AL, Luna SPL. Weighted pain-related behaviors in pigs undergoing castration based on multilevel logistic regression algorithm. Applied Animal Behaviour Science. 2023 Jul 18;106002.