Introduction

Lung sounds provide critical insights into pulmonary health and underlying disease. Traditionally, auscultation with a stethoscope has been the gold standard for bedside respiratory assessment [1–3]. However, advancements in technology have now enabled the use of electronic devices, such as digital stethoscopes and audio recorders, to capture and analyze respiratory sounds. These tools can be used to facilitate longitudinal monitoring, remote sharing, and integration of audio data into research pipelines for model development [4,5].

In addition, recent efforts have focused on developing machine learning (ML) models for automated lung sound classification, particularly in detecting adventitious sounds such as crackles and wheezes [6,7]. A common approach involves converting audio recordings into spectrograms and applying deep learning architectures, such as convolutional neural networks (CNNs), to build binary classifiers or detection models [8–10]. Despite promising performance, these models typically require substantial computational resources and extensive dataset-specific optimization, constraining their generalizability and limiting their use beyond specialized centres.

A potential alternative to CNNs for automated lung sound analysis is using general-purpose multimodal large language models (LLMs) to classify lung sounds [11]. Unlike traditional deep learning models, which rely on large, labeled datasets and require dedicated training for specific tasks, multimodal LLMs leverage pre-trained knowledge across diverse data formats, such as images, videos, and code. This flexibility may reduce the need for building task-specific architectures from scratch and instead shift the focus toward designing effective prompts. In this context, prompting strategies, which refer to structured instructions that shape how the model interprets and responds to an input, serve as a mechanism to optimize performance without explicit retraining. Such approaches may offer a scalable pathway for applying multimodal LLMs to defined tasks, like respiratory sound classification [12].

One such strategy is few-shot prompting, where a model is given a small set of labeled examples within the prompt to guide its output. Unlike zero-shot prompting, which provides no prior examples, few-shot prompting aims for models to infer patterns from limited contextual information. This strategy has been shown to improve LLM output accuracy for well-defined text-based tasks [12–14]. For example, prior research has demonstrated that few-shot prompting enhances performance in tasks, such as differential diagnosis generation and text classification [15]. However, whether this in-context learning extends to multimodal tasks, particularly in the classification of lung sound spectrograms, remains unexplored.

To our knowledge, no prior studies have evaluated the performance of a general-purpose multimodal LLM for lung sound analysis or investigated the impact of few-shot prompting in this context. Therefore, this study aims to address these gaps by assessing the ability of a multimodal LLM to classify lung sounds from mel-spectrograms and by comparing the baseline performance of few-shot versus zero-shot prompting.

Materials and methods

Prompt engineering and development

A standardized prompt structure was developed for both zero-shot and few-shot strategies. For the zero-shot strategy, GPT-4o was presented with the spectrogram and a predefined task question regarding lung sound classification, without any prior exposure to labeled examples (Fig 1). In contrast, the few-shot prompts incorporated examples of labeled reference spectrograms, providing the model with input-output pairs before classification of the spectrogram under evaluation (Fig 2). The reference spectrograms were derived from previously validated audio samples, representing crackles, wheezes, both, and normal sounds, obtained from an external dataset [23], and generated using the same preprocessing pipeline as the ICBHI database. This approach ensured a strict separation from the ICBHI testing set to prevent data leakage [24,25]. Few-shot exemplars were manually curated based on their spectral clarity and characteristic representation of the intended auscultatory category. The few-shot prompt design was informed by multimodal prompting strategies from prior studies, which have been shown to improve classification accuracy when contextual examples are provided [21,26].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Zero-Shot Prompt Structure for Multimodal Model Lung Sound Classification.

https://doi.org/10.1371/journal.pdig.0001179.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Few-Shot Prompt Structure with Labeled Reference Example Spectrograms for Multimodal Model Lung Sound Classification.

https://doi.org/10.1371/journal.pdig.0001179.g002

Results

Overall classification performance

Overall classification accuracy was 0.320 (95% confidence interval [CI]: 0.309–0.331) for zero-shot prompting and 0.363 (95% CI: 0.352–0.374) for few-shot prompting. For the zero-shot prompting, the model achieved a true positive (TP) count of 935, true negative (TN) of 1271, false positive (FP) of 2371, and false negative (FN) of 2321. These counts corresponded to a precision of 0.283 (95% CI: 0.267–0.298), recall of 0.287 (95% CI: 0.272–0.303), F1-score of 0.285, specificity of 0.349 (95% CI: 0.334–0.364), and NPV of 0.354 (95% CI: 0.338–0.369).

Using few-shot prompting, the model produced counts of 976 TP, 1529 TN, 2113 FP, and 2280 FN. Corresponding performance metrics included a precision of 0.316 (95% CI: 0.300–0.332), recall of 0.300 (95% CI: 0.284–0.315), F1-score of 0.308, specificity of 0.420 (95% CI: 0.404–0.436), and NPV of 0.401 (95% CI: 0.386–0.417). McNemar’s test revealed a statistically significant difference in performance (χ² = 86.47, p < 0.001), indicating that the few-shot strategy significantly outperformed the zero-shot strategy.

For zero-shot prompting, balanced accuracy was 0.292, and few-shot prompting achieved a balanced accuracy of 0.299. Confusion matrices illustrating the distribution of correct and incorrect predictions across all four lung sound categories are provided for zero-shot (Fig 3) and few-shot (Fig 4) prompting strategies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Confusion Matrix for Zero-Shot Prompting Strategy.

The matrix displays the distribution of predicted versus ground truth labels across the four lung sound categories (normal, crackles, wheezes, both) for the zero-shot prompting strategy. Values represent the number of respiratory cycles classified into each category. Rows indicate ground truth labels, and columns indicate predicted labels.

https://doi.org/10.1371/journal.pdig.0001179.g003

Per-label classification performance

In the multiclass analysis, accuracy for the “only crackles” label was 0.603 (95% CI: 0.592–0.615) for zero-shot prompting and 0.582 (CI: 0.571–0.594) for few-shot prompting. For “only wheezes,” accuracy improved from 0.634 (95% CI: 0.623–0.645) in the zero-shot strategy to 0.737 (95% CI: 0.726–0.747) in the few-shot strategy. The “both” category showed high accuracy for both strategies, with 0.924 (95% CI: 0.918–0.931) for zero-shot and 0.915 (95% CI: 0.909–0.922) for few-shot prompting. Accuracy for the “normal” label was 0.478 (95% CI: 0.466–0.490) for zero-shot and 0.492 (95% CI: 0.480–0.504) for few-shot prompting.

Using the binary classification analysis, accuracy for detecting crackles was 0.563 (95% CI: 0.552–0.575) in the zero-shot strategy and 0.545 (95% CI: 0.533–0.557) in the few-shot strategy. For wheezes, accuracy improved from 0.628 (95% CI: 0.617–0.640) in the zero-shot strategy to 0.710 (95% CI: 0.700–0.721) with few-shot prompting. The remaining per-label performance metrics are reported in Table 1 for the four classes and Table 2 for the two binary classes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Multiclass performance metrics for each lung sound label (only crackles, only wheezes, both, normal) using zero-shot and few-shot prompting.

https://doi.org/10.1371/journal.pdig.0001179.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Binary classification performance metrics for crackles and wheezes using zero-shot and few-shot prompting.

https://doi.org/10.1371/journal.pdig.0001179.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Confusion Matrix for Few-Shot Prompting Strategy.

The matrix displays the distribution of predicted versus ground truth labels across the four lung sound categories (normal, crackles, wheezes, both) for the few-shot prompting strategy. Values represent the number of respiratory cycles classified into each category. Rows indicate ground truth labels, and columns indicate predicted labels.

https://doi.org/10.1371/journal.pdig.0001179.g004

On sub-analysis of the classification performance for the “both” label (where both crackles and wheezes were present in the ground truth), the model identified crackles in 126 zero-shot and 170 few-shot cases, and wheezes in 244 zero-shot and 186 few-shot cases. Only 8 instances had both crackles and wheezes correctly predicted in the zero-shot strategy, compared to 18 in the few-shot strategy. McNemar’s test revealed statistically significant differences in performance between zero- and few-shot strategies for both crackles (χ² = 27.191, p < 0.001) and wheezes (χ² = 38.679, p < 0.001).

For the “normal” label (where both crackles and wheezes were absent), the model correctly predicted absence of crackles in 2,643 zero-shot and 2,383 few-shot cases, and absence of wheezes in 2,256 and 2,737 cases, respectively. Both crackles and wheezes were jointly absent in 1,271 zero-shot and 1,529 few-shot cases. Again, McNemar’s test showed significant differences between zero- and few-shot prompting for both crackles (χ² = 140.337, p < 0.001) and wheezes (χ² = 344.395, p < 0.001).

Discussion

To our knowledge, this study is the first to evaluate the classification of lung sound spectrograms using a general-purpose multimodal LLM and to compare few-shot and zero-shot prompting strategies in a respiratory sound context. Few-shot prompting produced marginal improvements in spectrogram-based classification performance, yielding statistically significant gains over zero-shot prompting across several metrics, including accuracy, specificity, and F1-score for the majority of lung sound labels. These findings align with recent work demonstrating that few-shot prompting can enhance performance on text-based medical tasks [25,26,29]. Despite these improvements, overall model performance remains insufficient for clinical deployment, positioning this work as an exploratory benchmark for visual in-context learning in this domain.

Although GPT-4o was evaluated on a large and clinically diverse dataset using a standardized preprocessing pipeline, the overall performance of both prompting strategies fell below clinically acceptable thresholds, with overall accuracy for zero-shot and few-shot strategies at 32.0% and 36.3%, respectively [8]. These values are lower than those reported for task-specific deep learning models trained on the ICBHI dataset [16], including CNNs, audio spectrogram transformer models, and other architectures, which typically achieve accuracy ranges of approximately 50 to 90% depending on preprocessing, model design, and class balancing techniques [8,27,30–32]. Human benchmark studies similarly report diagnostic accuracy between 55 and 90%, with crackles and wheezes consistently posing greater interpretive difficulty than normal breath sounds [31,33]. For comparison, in the multiclass setting, a random-chance classifier would achieve an accuracy of 25%; thus, the observed 32.0% (zero-shot) and 36.3% (few-shot) performance, while above random chance, underscores the limited discriminative ability of both strategies on this task. Consistent with these findings, multiclass balanced accuracy also remained low (0.292 for zero-shot vs. 0.299 for few-shot), indicating that explicitly adjusting for class imbalance does not materially change the overall performance profile. At the same time, the slight improvement with few-shot prompting suggests a small but measurable benefit in more evenly distributing predictions across classes, though this remains far from closing the gap with task-specific models.

Both prompting strategies showed notably poor performance in identifying lung cycles containing simultaneous crackles and wheezes, indicating that the multimodal LLM may struggle to integrate compound acoustic features. In the multiclass setting, recall for this category remained below 4%, and F1-scores were less than 10%. These findings suggest that when acoustic features overlap or co-occur with considerable variability, GPT-4o may be unable to separate individual sounds without task-specific training [34]. Furthermore, the inconsistent class-wise trends, such as the drop in crackles accuracy (from 60.3 to 58.2%) alongside a rise in wheezes accuracy (from 63.4 to 73.7%) with few-shot prompting, suggest that the effect of few-shot learning is not uniformly beneficial across all classes. This may be due to the more continuous and tonal nature of wheezes, which manifest in spectrograms as sustained horizontal patterns, features that may be more visually distinctive to a multimodal LLM. Crackles, on the other hand, are brief, discontinuous events whose sporadic nature may be less readily interpretable by a vision model without prior domain training [35]. In contrast, the “normal” class showed comparatively strong and consistent performance, likely due to the relative visual uniformity of spectrograms lacking adventitious sounds and the absence of unpredictable crackle or wheeze patterns within a respiratory cycle [35].

Taken together, our results suggest that GPT-4o shows limited but measurable responsiveness to few-shot prompting for spectrogram-based classification. Although its performance remained substantially lower than that of task-specific deep learning models–and well below the gains typically reported for text-based tasks [26]–the modest gains observed with contextual examples indicate a small degree of visual in-context learning without domain-specific training [26]. Considering these findings, exploration of other prompt-based approaches—for example, varying the number or visual characteristics of exemplars, or considering techniques such as chain-of-thought or retrieval-augmented prompting—could help clarify the extent to which prompting strategies influence model performance [36–39]. In parallel, broader evaluation across diverse patient populations, real-world or noisy recordings, and additional adventitious lung sounds such as rhonchi and stridor may help clarify how these models behave across a fuller spectrum of pulmonary acoustics [40–42].

Beyond technical performance, translation of multimodal LLMs into clinical respiratory assessment tools requires addressing significant practical, regulatory, and ethical hurdles. As no LLMs are currently cleared for standalone diagnostic use, robust “human-in-the-loop” oversight is critical for ensuring safety and accountability as these technologies advance toward clinical evaluation and potential deployment [25,43]. This approach aligns with existing regulatory precedents for AI-augmented digital auscultation tools, where authorization is limited to assistive interpretation requiring clinician oversight [44–46]. Additional concerns include liability arising from model errors or “hallucinations”, particularly regarding who bears responsibility if incorrect or misleading outputs contribute to diagnostic delay, inappropriate management, or missed pathology. These issues are further compounded by uncertainty about sustaining reliable performance across diverse patient populations and variable recording conditions, as well as the need for mechanisms to monitor and detect performance drift over time [12,47,48]. The susceptibility of LLMs to adversarial inputs, subtle prompt manipulations, and context-dependent errors adds another layer of complexity that must be carefully examined before clinical adoption [49,50]. Advancing toward regulated medical use will also require clear pathways for accountability and validation. While formal regulatory structures for generative AI and multimodal models are limited, emerging frameworks, such as Health Canada’s pre-market guidance for ML-enabled medical devices [51] and the FDA’s draft guidance for AI-enabled device software function [52], provide an early structure for safety documentation and risk mitigation. Ultimately, meaningful clinical integration requires large-scale real-world validation studies, standardized evaluation protocols, and a focus on equitable access to ensure that AI-augmented respiratory assessment tools are usable across both high- and low-resource environments [53–55].

This study was not without its limitations. First, although spectrograms were generated from the ICBHI 2017 dataset using a standardized preprocessing pipeline to ensure visual consistency, this transformation may not preserve all acoustic nuances present in the original recordings, particularly subtle transient features or sounds masked by background noise [16]. Nonetheless, such noise and variability reflect real-world conditions and support generalizability. Second, while the zero- and few-shot prompts followed previously validated approaches and were applied consistently across all 6,898 cases, model outputs remain inherently sensitive to prompt formulation, and alternative prompt structures could yield different results [37]. Lastly, this study evaluated the performance of a single multimodal LLM, which, although GPT-4o is a popular and well-studied foundation model, may not reflect the performance of smaller, domain-specific, or medically fine-tuned models [56]. Future work should benchmark performance across emerging multimodal architectures to contextualize these findings within the rapidly evolving LLM landscape.

Conclusion

Our study demonstrates that few-shot prompting generally achieves marginal improvements over zero-shot prompting for mel-spectrogram-based lung sound classification using a general-purpose multimodal LLM. Although overall performance remains far below that of task-specific deep learning models and insufficient for clinical use, this approach provides a baseline for visual in-context learning and foundation for subsequent refinement. Further work should focus on optimizing prompting strategies and evaluating performance across more diverse, clinically representative datasets.