Observables and coordination in piano practice

Music-facing observables obtained from Musical Instrument Digital Interface (MIDI) or sensor systems make it possible to condition practice on measured coordination changes. Studies on effectors, modality, and tempo show that synchronization stability depends on hand, presentation, and rate rather than being uniform across conditions, which motivates interfaces that operate directly on these music-specific measurements [3,4]. Recent piano-specific modeling work shows that skilled piano control cannot be reduced to simple timing accuracy alone. Upper-limb coordination during octave playing depends on biomechanical constraints [20]. Expressive piano output also reflects nonlinear mappings between dynamic control parameters and sound [21]. Brink et al. and Maarup et al. further reported that variability structure and a bodily hierarchy across voice, hands, and feet shape coordination transitions. These findings in turn motivate explicit noise modeling and the use of standardized observables for practice controllers [9,22]. Longitudinal neuroimaging on piano training further indicates context-dependent, non-monotonic plasticity, so progress cannot be treated as stationary during practice [7,23,24].

Fatigue studies on pianists show that repetitive or demanding segments induce local muscular fatigue and degrade timing and key velocity. They also show that subjective rest does not always match objective indicators [1,2]. These findings support returning a noisy online fatigue estimate for immediate action filtering together with a delayed true fatigue value for long timescale statistics. Interpersonal and duet studies report that joint action and perturbations modulate interbrain synchronization and the balance of self other integration [11,25]. Such effects are outside the scope of the single-learner, post-action interface considered here and are therefore not modeled.

Safety in sequential decision making

Safe and constrained reinforcement learning encodes performance under cost limits through Lagrangian or proximal updates, which yields policies that track average constraints during learning [14,26]. Complementary lines of work enforce safety through certified sets and predictive shielding that override unsafe actions near constraint boundaries and provide forward-invariance style guarantees [15,27]. More recent studies combine runtime enforcement with learned policies in safety-critical systems, where an external enforcer corrects an agent whenever formal safety rules would be violated [12,28–30].

A large fraction of these approaches assumes fully observed states or synchronous cost feedback and operates on a single time scale, as in representative value-based reinforcement learning under full observability [31,32]. Shielding under partial observability is closely related because it uses a model-based safety filter when the agent state is incomplete [33]. In fatigue-aware piano rhythm training the controller receives post-action music metrics and two distinct fatigue signals. Safety must therefore be split into a slow Lagrangian adaptation that uses revealed true fatigue and a fast predictive guard that uses the online estimate. Experiments in PianoGym are designed with a shared environment-side guard so that different decision rules can be compared under the same fatigue budget.

Planning and model predictive controller for practice

Optimization based sequencing on music platforms shows that exploiting position aware and locally sequential preferences increases within session engagement. This supports ordered practice rather than independent ranking of exercises [18]. Model based real time accompaniment further demonstrates that temporal alignment with human performers improves when the controller adapts online [19]. These studies validate goal aware, model based lookahead in musical interaction. However, they generally do not model learner side skill transfer, interference across tasks, or fatigue limited practice budgets, and the control inputs are typically not drawn from a finite library.

Model predictive controllers have been adapted for rhythmic or periodic motor tasks. Examples include basis function parameterization for fast gait generation and combinations of central pattern generators with MPC [34,35]. Gain scheduled and bio rhythm informed MPC maintains performance under time varying operating points, which indicates that predictive controllers can respect physiological cycles without losing responsiveness [36,37]. The piano practice setting considered here differs because actions come from a finite, structured library with tagged difficulty, skill targets, and positive fatigue increments, and planning is restricted to guard-filtered safe action sets. The combination of post-action observability, dual-timescale safety, and library-based MPC is not addressed in the above MPC and sequencing literature.

Problem statement and interaction protocol

Fatigue-aware piano practice is modeled as a constrained, post-action, partially observable Markov decision process (CPOMDP) with music-specific observables and two fatigue signals. This common interface makes it possible to analyze both sample efficiency and safety.

Post-action partially observable process

At decision step t, the learner occupies an unobserved latent state

(1)

where x_t contains the proficiencies of K rhythm-related skills, f_t is the accumulated cognitive and motor fatigue, and m_t is a memory or retention trace of the same dimension.

The controller maintains an internal estimate and chooses a structured practice action according to

(2)

The environment then evolves according to the true state

(3)

that is, the transition kernel depends on (s_t, a_t) and not on the estimate. After the transition, the environment reveals three post-action signals

(4)

where

(5)

collects the music-facing measurements, is a noisy online fatigue estimate intended for immediate safety screening, and is the revealed true fatigue intended for long-run constraint adaptation. The scalar r_t denotes the immediate reward revealed after executing a_t, and is the reward map from the post-action observation and executed action to this scalar reward. The two fatigue signals are consumed by different safety components in Section Dual-timescale fatigue safety. Policies in Section PianoMPC controller are defined on the filtered state estimate , as shown in Fig 1.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Post-action interaction protocol.

https://doi.org/10.1371/journal.pone.0351141.g001

Objective and constraints

Rhythm mastery is declared once

(6)

hold for W = 3 consecutive decision steps, where W denotes the consecutive-mastery window. The main performance indicator is the time to mastery (TTM), defined as the smallest index t at which Eq. (6) is satisfied, with if the condition is not met within the episode.

Because every practice segment induces fatigue, the process is constrained by the long-run average of the true fatigue:

(7)

with task-dependent threshold , where denotes the episode length in the PianoGym environment. The control objective is therefore

(8)

Here denotes the expected long-run fatigue cost under policy , is expectation over trajectories induced by and the simulator dynamics. In the PianoGym instantiation used for experimentation, the cumulative reward in Eq. (8) is monotone in TTM, so reporting TTM is sufficient to compare controllers. The reward-based formulation is nevertheless retained because the model-predictive controller in Section PianoMPC controller optimizes a finite-horizon surrogate of Eq. (8).

Environment and predictive model

The controller performs model-predictive rollouts and therefore requires a deterministic approximation of the environment dynamics. This section makes the abstract CPOMDP in Section Problem statement and interaction protocol concrete by specifying the latent variables, the structured action library, the skill and fatigue updates, and the stochastic observation layer from which rewards are computed. All parameters below are treated as known constants of the PianoGym environment and are not learned by the controller. This makes PianoMPC a certainty-equivalent planner: it plans from the current filtered estimate as if that estimate were the rollout state, rather than maintaining a full posterior belief over latent skills.

Parameterization and calibration

Parameterization and calibration PianoGym uses a fixed parameterization so that all controllers are compared in the same simulator rather than learning a different environment model. The action-library structure, skill targets, transfer signs, and fatigue-load ordering were manually specified from the musical roles of the exercises. The numerical learning rates, forgetting rate, fatigue gain, rest recovery rate, observation scales, and noise levels were then set once on an internal calibration suite of simulated pilot profiles spanning balanced, mild-left-weak, and severe-left-weak learners. The calibration target was not to fit a human cohort, but to place asynchrony, dominance gap, fidelity, fatigue, and TTM in interpretable ranges aligned with the evaluation thresholds. After this calibration step, the parameters were frozen for every baseline, ablation, and robustness experiment. Sensitivity to these choices is assessed in the dynamics-mismatch, scoped-mismatch, threshold-window, and guard-dependence analyses in Section Experiments. Table 1 summarizes these parameter groups and their sources.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Parameter sources in PianoGym.

https://doi.org/10.1371/journal.pone.0351141.t001

State and action space instantiation

The latent skill vector contains K = 5 rhythm-relevant components

(9)

describing left-hand control, right-hand control, 2:3 polyrhythm coordination, 3:4 polyrhythm coordination, and fast switching. The memory trace has the same dimension and plays the role of a smoothed retention level.

The action set is a finite library of predefined structured practice actions and is exposed to the controller as its discrete action space. Each action is encoded as

(10)

where is a difficulty tag, with specifies which skills the segment targets, specifies learning rates per skill, is a sparse transfer or interference matrix, is the positive fatigue load, and is the execution time cost. This representation separates the musical target, the learning intensity, and the physiological load. The library also contains an explicit REST action,

(11)

which is kept available at every decision step.

Skill dynamics

Skill adaptation results from diminishing returns, difficulty matching, and fatigue attenuation. For an executed action a_t,

(12)

where reduces gains near mastery. The difficulty matching gate, as shown in Fig 2, is used to filter candidate actions so that their tagged difficulty stays close to the current skill estimate.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Difficulty-matching gate mechanism.

https://doi.org/10.1371/journal.pone.0351141.g002

(13)

which becomes active when the projected skill level approaches the declared difficulty d.

Fatigue attenuates learning through

(14)

with rate and lower bound .

Transfer, interference, and process noise are then added:

(15)

The next skill vector is clipped element-wise to the admissible range

(16)

where, for a vector v, applies to every coordinate v_i. This clipping is part of the environment step and is mirrored by the controller.

Fatigue and memory dynamics

For practice actions , fatigue increases with the action load:

(17)

For the REST action, fatigue recovers at rate :

(18)

Retention is maintained through an exponential smoother:

(19)

and forgetting is represented as a small decay proportional to missing retention:

(20)

with decay rate . This decay is also part of the dynamics mirrored by the controller.

Observation and reward generation

Given and the executed action a_t, the environment samples

from explicit action-dependent observation equations. Let w_A be the fixed skill weights used for asynchrony, let and be action-specific scale parameters, and let be an action-specific dominance-gap offset.

(21)

where . The dominance gap is generated as

(22)

where . The fidelity signal is a clipped logistic-normal proxy

(23)

These equations make the quantitative link explicit: higher relevant skill lowers asynchrony and improves fidelity, left-right imbalance changes dominance gap, and higher fatigue increases timing error and lowers fidelity. They are simulator observation equations, not fitted physiological measurement laws. The online fatigue estimate is obtained by

(24)

To aggregate heterogeneous observables into a single learning signal, each measurement v is standardized using fixed statistics :

(25)

Here z(v)⁺ denotes the positive part of the standardized residual. Asynchrony and dominance gap are standardized in this way. Fidelity already lies in [0,1] and is used in raw form. The immediate reward is

(26)

where are fixed reward weights, is the duration cost of the executed action, and is its coefficient. The reward penalizes poor musical coordination and long practice segments and keeps duration-related effects comparable across actions.

Dual-timescale fatigue safety

Fatigue must remain within specified limits both on average and at the next decision instant. Safety is therefore enforced on two coupled time scales, as shown in Fig 3. A slow Lagrangian adaptation tracks the long-run constraint in Eq. (7) using the revealed true fatigue. A fast predictive guard filters unsafe actions using the online estimate. In this paper, the safety layer refers to the simulator-level fatigue-feasibility interface used to screen practice actions. During experimentation, the same one-step guard is implemented as an environment-side guard for all agents in order to provide a common safety interface. The environment-side guard takes precedence and can overwrite any action proposed by a controller according to a fixed fallback rule.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Dual-timescale safety layer.

https://doi.org/10.1371/journal.pone.0351141.g003

Slow timescale: lagrangian adaptation

At the beginning of step t, the controller has access to the true fatigue f_t that was revealed at the end of the previous transition. An exponential moving average of revealed fatigue is updated as

(27)

and the Lagrange multiplier for fatigue is advanced by projected gradient ascent

(28)

The maximum operator keeps the Lagrange multiplier nonnegative, as required for the fatigue inequality constraint. The scalar converts the average constraint into an instantaneous penalty and is kept fixed for the entire decision step t. During planning in Section PianoMPC controller, this multiplier penalizes predicted trajectories whose fatigue rises above .

Fast timescale: predictive guard

The fast component restricts actions that would produce an immediate overload. At step t, the internal fatigue estimate (computed in Section PianoMPC controller) is combined with the deterministic dynamics Eq. (17)–Eq. (18) and the action-specific loads in order to predict the next-step fatigue for every . Two margins are used for two different purposes. The conservative one-step margin defines a primary step cap, whereas the relaxed guard slack defines a peak diagnostic threshold used by the environment-side guard and by guard-consistency experiments. The primary one-step set is

(29)

For a guard horizon H_g, the relaxed peak condition is

(30)

The executed guard uses Eq. (29) as the ordinary action filter, and REST is always available. The relaxed band in Eq. (30) is logged for calibration and can affect execution only in the fallback case where no non-rest primary action survives; otherwise it does not enlarge the ordinary safe set. Accordingly, is used as the conservative execution cap, whereas is used only as a relaxed peak-diagnostic threshold. When the environment-side guard is present, the environment applies the same interface to the finally proposed action.

PianoMPC controller

This section introduces PianoMPC, a finite-horizon, certainty-equivalent controller that plans only over actions certified as safe by the dual-timescale mechanism. The controller uses the deterministic model in Section Environment and predictive model, the multipliers in Section Dual-timescale fatigue safety, and the same structured action library as the environment.

Certainty-equivalent state tracking

At the start of step t, immediately after receiving generated by action , the controller forms an updated estimate

The update is carried out in two stages.

First, a prediction step propagates the previous estimate through the deterministic model:

(31)

where applies the noise-free dynamics Eq. (12)–Eq. (20) and Eq. (17)–Eq. (18). This produces a prior for step t.

Second, a correction step aligns the fatigue component with the revealed true fatigue while keeping the skill and memory components unchanged:

(32)

(33)

(34)

This certainty-equivalent design is motivated by two characteristics of the setting. Musical observables (Async, DomGap, Fid) are noisy and task dependent, so back-propagating them into all skill coordinates at every step would increase variance and computational load. In contrast, the environment reveals the true fatigue without noise. Using it to anchor yields an accurate fatigue signal for both the slow Lagrangian update and the fast guard. The observables are therefore used through the deterministic observation and reward model in Section Environment and predictive model and are not used for state correction.

Finite-horizon planning

Given and the current multiplier , the controller solves a receding-horizon problem of length H, as shown in Fig 4. Let a candidate action sequence be . The deterministic model is rolled forward as

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Model predictive control with receding horizon.

https://doi.org/10.1371/journal.pone.0351141.g004

(35)

starting from . At each lookahead step t + h, the predictive guard in Section Dual-timescale fatigue safety is applied to produce the future safe set using the predicted fatigue contained in .

The immediate reward at lookahead index h is obtained by applying the deterministic observation-and-reward model to the predicted state and to action . For compactness, this predicted quantity is denoted by . The planning problem is

(36)

(37)

(38)

The multiplier computed in Eq. (28) is held constant over the horizon, so that the optimization in Eq. (36) depends only on the predicted trajectories.

The discrete optimization in Eq. (36) is solved by a beam search of width b. At each depth, only the best b partial sequences consistent with the safe sets are kept, which yields a computational cost proportional to and to the size of the filtered action sets. After evaluating all depth-H sequences retained by the beam, the first action of the best sequence is executed. At the next decision step, the state is re-estimated and the planning procedure is repeated. This produces a receding horizon controller that respects both long-run and short-horizon fatigue constraints and that is used in Section Experiments.

Experiments

This section evaluates whether model-predictive planning improves learning under fatigue-constrained practice. All experiments use the same PianoGym action library and the same environment-side short-horizon safety guard. Two safety regimes are reported. In the main comparison reported in Section Main comparison under the safety guard, the environment fatigue hard constraint is enabled for the three learner profiles, so that agents differ only in their decision rules under a common safety interface. In the task suite study reported in Section Robustness across task families, each profile follows its default configuration. The balanced and mild-left-weak profiles keep only the guard, whereas the severe-left-weak profile enables both the guard and the hard constraint.

Task composition

A 3 × 3 task suite is constructed in PianoGymEnv (see Table 2). The rows are the three learner profiles used throughout the experiments, and for each profile three transfer levels (weak, medium, strong) are used. This gives 9 tasks in total.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Profiles in the PianoGym task suite.

https://doi.org/10.1371/journal.pone.0351141.t002

Scale and execution protocol

Unless otherwise stated, S = 10 independent random seeds are run for each task, and each seed generates one trajectory. A trajectory runs for at most steps and terminates early once the mastery condition is satisfied. This produces

records of states, actions, observations, and fatigue. Under this configuration the total sample size is on the order of 10⁵, which is sufficient for reporting means and 95% confidence intervals.

Symmetry between left-weak and right-weak profiles

Only the left-weak variants (mild-left-weak, severe-left-weak) are reported in the main text. The data generator can swap left and right skill dimensions and action targets to obtain the right-weak counterparts. Because the action-library fatigue parameters, the short-horizon safety guard, and the PianoMPC state estimator are symmetric in the two dimensions, the mirrored tasks yield the same rankings. Their safety statistics are also almost identical. The corresponding results can be reproduced from the released code or from the supplementary material.

Observation and action interface

The environment uses a post-action interface. After executing action a_t at time t, it returns

together with the online fatigue estimate and the true fatigue . The action set contains several structured practice actions and an explicit rest action REST. For comparability, a common environment-side short-horizon safety guard is applied to every agent. Each agent first proposes a candidate action. The guard then filters actions using the current online fatigue estimate (denoted in the experiments) and produces

with in the default experiments. The environment-side guard also logs whether the proposed action would pass the relaxed diagnostic band , with . Only actions in this primary set are executed, except for the relaxed fallback case defined in Section Dual-timescale fatigue safety. Some agents, such as PianoMPC and the fatigue-aware LinUCB variant, forecast future fatigue and therefore tend to propose actions that already satisfy the guard. Other agents do not anticipate the guard, but their final executed actions are still filtered by the same environment-side rule. This design keeps the safety mechanism identical across methods and lets the comparison focus on the quality of the decision rules.

Experimental setup

One simulation step corresponds to a practice segment of about 30–60 seconds. This correspondence fixes the scale of the fatigue load , the recovery rate , and the typical time to mastery (TTM). All observation scalings, such as , , , and associated noise levels are estimated on an internal calibration set and then frozen for all experiments. The calibration set contains simulated pilot runs from the three learner profiles and was used only to set observation ranges and noise levels before the reported experiments were run.

Unless specified otherwise, the experiments use K = 5 skill dimensions and an action library of size , which includes the REST action. The mastery condition requires a window of W = 3 consecutive successful steps. Key hyperparameters are set as follows: the profile-specific fatigue threshold is for balanced, mild-left-weak, and severe-left-weak profiles; the one-step guard margin is ; the relaxed diagnostic slack is ; and the guard horizon is . For the main comparison, the MPC planning horizon is H = 3; the horizon sweep separately evaluates . The beam search width is b = 20. Dynamics parameters include a memory smoothing factor and a forgetting rate . The reward weights are set to , and the time-cost coefficient is .

The per-step computational complexity of PianoMPC is given by , where T_F is the cost of one forward prediction . In practice, the finite action library combined with the predictive guard ensures that the safe action set remains small, keeping the controller computationally efficient.

Evaluation metrics

All results are computed by the evaluation module in PianoGym and are averaged across tasks and seeds. Unless otherwise stated, means and 95% confidence intervals are computed across independent seeds within the stated profile or task aggregation unit. For rate metrics, figure displays are clipped to the valid interval [0,1] or [0,100]% without changing the underlying means. Consider a trajectory

where denotes fatigue, denotes the environment fatigue threshold, W denotes the mastery window, and denotes the maximum episode length.

Time to mastery (TTM) quantifies sample efficiency by measuring how many steps are required to achieve stable mastery for a window of length W. Formally,

(39)

with if the mastery condition is not satisfied within . TTM is the primary performance metric used to rank agents in Section Main comparison under the safety guard to Section Effect of fatigue-threshold and mastery-window choices (lower is better). The counter records consecutive successful steps up to and including step t and resets to zero whenever any criterion in Eq. (6) is not satisfied.

Task return is computed by recording the instantaneous raw reward and aggregating

(40)

(41)

Safety metrics follow the dual-timescale safety design, consisting of a short-horizon guard and a long-horizon fatigue limit. Each fatigue sample is thresholded as

Following standard practice in constrained and safe reinforcement learning [14], the following quantities are reported:

(42)

(43)

(44)

(45)

FeasibleRate and OverloadRate are complementary by definition, while AvgViolation measures the magnitude of fatigue violations rather than their frequency. Guard replacement rate is reported as an intervention diagnostic for proposed-versus-executed actions and is not treated as a safety success metric. For this reason, the results are interpreted as fatigue-feasibility outcomes under the simulator budget rather than as absolute safe or unsafe behavior.

A progress metric called rhythm independence gain is also reported:

(46)

This metric uses the same thresholds as the mastery condition defined by the environment and therefore supports cross task comparison. Additional diagnostics, including memory retention and offline evaluation coverage, appear in the public implementation.

Baselines

All baselines observe the same post-action signals, use the same action set, and are filtered by the same environment-side guard. Our PianoMPC agent and the fatigue-aware LinUCB variant also run the same short-horizon guard inside the policy, whereas the other baselines rely only on the environment-side guard. Thus differences in Section Main comparison under the safety guard come mainly from the decision rule. Thompson [38] is a lightweight posterior-sampling bandit under delayed feedback. LinUCB [39] is a contextual bandit on PianoGym features but does not plan future fatigue. Bayesian MAB [40] adds discounting and change-point handling for non-stationary profiles. CCB-DF [41] learns from delayed counterfactual rewards. DQN [31] is a value-based agent that relies on the external guard for safety. Safe-AC [32] treats fatigue above as cost in a Lagrangian actor-critic. Auto-Curriculum [42] orders exercises and inserts REST, mimicking human teaching without MPC-style planning.

Results on three profiles

This section reports the main comparison under the common safety guard. Table 3 summarizes three learner profiles and eight agents. FeasibleRate values range from 0.89 to 0.97 in the main comparison, with nonzero overload rates still possible. For this reason, the primary comparison is mean TTM interpreted together with FeasibleRate and violation diagnostics, rather than a binary safe/unsafe label.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Overall comparison of time to mastery (TTM) and FeasibleRate (Feas.). Values are mean ± 95% CI over 10 runs. Best values are highlighted in bold and second best are underlined.

https://doi.org/10.1371/journal.pone.0351141.t003

PianoMPC achieves mean TTM values of 24.4, 26.0, and 28.2 steps across the balanced, mild-left-weak, and severe-left-weak profiles, respectively. This range is clearly below the second group, formed by BayesianMAB with 35–40 steps. All other methods require between the low 40s and about 60 steps, with LinUCB, DQN, CCB-DF and AutoCurriculum often around 50–60 steps.

Fig 5 panel (a) shows the same three-level structure. LinUCB keeps the highest FeasibleRate, about two points above PianoMPC, yet this improvement is marginal compared with the extra training time. The large gap appears because LinUCB reacts myopically when fatigue is near the limit and spends many steps on low-value rest choices. The controller plans the fatigue budget over the horizon, so it converts the same fatigue allowance into progress. Fig 5 panel (b) follows the time pattern, since shorter runs accumulate less negative reward. Panel (c) reports that some slower methods reach slightly higher coordination, which reflects longer exposure rather than stronger learning rules. Fig 6 shows that PianoMPC improves reward earliest and keeps the advantage through the early practice steps. This behaviour is consistent with planning under post-action observations and with the dual safety design of the environment. A guard-dependence ablation compares the default environment-side guard, a weaker guard, and no guard. PianoMPC remains the fastest method in all three settings, with macro mean TTM values of 25.67, 24.50, and 25.77 steps and FeasibleRate values of 0.921, 0.920, and 0.927, respectively. Under the default environment-side guard, the guard replacement rate for PianoMPC is 0.000, whereas replacement rates for several reactive baselines are much higher. This indicates that the guard is a common execution interface and diagnostic, rather than the sole driver of PianoMPC’s efficiency.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Quantitative results for the main experiment.

Panel labels show mean TTM, reward, and coordination gain; error bars are 95% seed CIs.

https://doi.org/10.1371/journal.pone.0351141.g005

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Learning curves for the main experiment.

Curves show sliding-averaged raw reward; shaded bands are 95% seed CIs.

https://doi.org/10.1371/journal.pone.0351141.g006

Effect of planning horizon

To assess how much lookahead is needed, the planning horizon H was varied. This test used the balanced profile while all other settings were fixed. The sweep uses the same balanced-profile seed schedule and agent-update pipeline as the main comparison, so the H = 3 point matches the PianoMPC entry in Table 3; H = 10 is an additional sweep setting rather than the table configuration. Fig 7 shows the resulting trade-off between training speed and fatigue feasibility. A myopic policy (H = 1) achieves the lowest balanced-profile mean TTM at 24.1 steps, but its FeasibleRate is 0.902 and its macro FeasibleRate across profiles is 0.850. The H = 1 setting is therefore treated as a low-feasibility setting rather than categorically unsafe.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Effect of PianoMPC planning horizon on the balanced profile.

Left axis shows TTM; right axis shows FeasibleRate on a zoomed scale with visual headroom above 100%; symmetric 95% CIs are clipped to the valid 0–100% range for display.

https://doi.org/10.1371/journal.pone.0351141.g007

At the main-comparison setting H = 3, the balanced-profile mean TTM is 24.4 steps and FeasibleRate is 0.945. Longer horizons raise FeasibleRate to 1.000 for H = 5 and H = 10, with balanced-profile mean TTM values of 27.3 and 25.8 steps. This pattern indicates that short lookahead already captures useful near-term fatigue structure, while longer lookahead mainly changes the feasibility-speed trade-off. Thus planning depth, not only the shared guard, explains the efficiency gap.

Effect of agent-level safety components

This experiment evaluates the safety modules inside the agent policy. The short-horizon guard and the soft fatigue penalty are enabled or removed. Fig 8 reports a profile-aware bar plot of mean TTM and realized fatigue-overload rate, computed as , across all three learner profiles. Note that the environment guard remains enabled for every variant and the ablation removes only agent-side components such as the internal guard or the soft penalty.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 8. Agent-side safety-module ablation across three learner profiles.

Bars show macro means; open markers show profile-specific values. Overload rate denotes the realized trajectory overload rate, , after the shared environment-side guard.

https://doi.org/10.1371/journal.pone.0351141.g008

With both PianoMPC safety components active, the macro mean realized overload rate is 8.2% and mean TTM is 26.2 steps. Removing the soft penalty gives a similar realized overload rate of 8.3% and a mean TTM of 26.7 steps. Thus, under the shared environment-side guard, PianoMPC’s internal guard and replanning dominate this ablation, while the soft penalty has only a small effect on the realized feasibility-speed trade-off.

The LinUCB variants remain slower, with mean TTM values between 49.2 and 53.1 steps. Their realized overload rates stay between 3.5% and 8.0%, indicating a clearer guard-dependent feasibility-speed trade-off for the reactive bandit variants. Safe-AC appears between these groups, with mean TTM of 42.5 steps and an 8.2% realized overload rate, so it improves over the bandit variants but does not match PianoMPC’s efficiency.

Consistency between windowed and peak-guard safety metrics

This experiment checks whether the online guard diagnostics behave consistently as the guard slack changes. Fig 9 reports online peak-violation rate and guard false-negative rate for PianoMPC on the balanced profile. The diagnostic window is fixed at in this plot so that the figure focuses on the guard horizon H_g and the relaxed slack .

Download:

• PNG
  larger image
• TIFF
  original image

Fig 9. Guard-calibration diagnostics for PianoMPC on the balanced profile.

Panels show peak-violation and false-negative rates across guard horizons at fixed diagnostic window .

https://doi.org/10.1371/journal.pone.0351141.g009

The online peak-violation rate decreases as increases, which is expected because the diagnostic threshold becomes less conservative. The false-negative rate remains low across the displayed settings and approaches zero at . These rates are diagnostics of the guard calibration; they do not mean that violations are allowed as a goal of the method. The online guard is used for action filtering, whereas windowed violation metrics are retained as offline checks of the realized trajectory.

Robustness across task families

This experiment tests whether the advantage of planning holds under different task structures. The suite combines three categories (Balanced, Mild-left-weak, Severe-left-weakness) with three transfer levels (weak, medium, strong). Table 4 reports time to mastery and FeasibleRate for the four strongest agents. The suite includes nonzero overload rates in several severe-left-weakness settings, so these values should be read as fatigue-feasibility diagnostics rather than as binary safety claims. We keep these defaults to test robustness under realistic per-task settings rather than under a single globally tuned constraint.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Robustness of top agents across the 3 × 3 task suite. Cells report mean TTM and FeasibleRate with 95% seed CIs.

https://doi.org/10.1371/journal.pone.0351141.t004

PianoMPC achieves the lowest mean TTM in every cell of the table. On weak transfer tasks it needs about 43–47 steps, while CCB-DF needs around 78 steps and Thompson stays above 70 steps. BayesianMAB lies between them, at 49–54 steps, and is therefore the second tier. This shows that, when transfer is scarce and fatigue is enforced, planning still converts practice into progress more efficiently than reactive bandits.

When transfer becomes medium or strong, every agent improves, but the gains are not equal. PianoMPC drops to roughly 25 steps for medium transfer and to 14–17 steps for strong transfer across all categories. BayesianMAB and CCB-DF also speed up, yet they remain 10–20 steps slower than the planner. In the severe-left-weakness tasks, PianoMPC is the only agent that stays below 30 mean steps at medium transfer while keeping FeasibleRate above 0.9. This indicates that the planning controller remains effective across this task suite under the evaluated fatigue limits.

Robustness to dynamics misspecification

This experiment studies how sensitive the agents are to errors in the task dynamics. Three environment parameters are scaled around the nominal value one. The parameters control memory decay, fatigue accumulation, and gating sharpness. Fig 10 reports the change in four metrics with respect to the baseline run. Values close to zero indicate stable behaviour under misspecification.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 10. Sensitivity of agents to dynamics misspecification.

Panels show metric changes from nominal; AutoCurriculum reaches the run limit in scanned settings.

https://doi.org/10.1371/journal.pone.0351141.g010

Across all settings, PianoMPC stays close to the zero line. For memory errors, its time to mastery changes by at most about four steps, and FeasibleRate varies within roughly three tenths of a percentage point. Average fatigue and learning slope also show only small shifts. This pattern suggests that online replanning absorbs moderate model errors without extra tuning.

Non-planning agents react much more strongly. For misspecified forgetting rates, LinUCB is affected the most and can require more than 50 additional steps, while AutoCurriculum always hits the run limit in terms of TTM, so its sensitivity in time to mastery cannot be observed. Several bandit baselines also lose around one percentage point of feasibility when the fatigue gain is inaccurate. Misspecification of the gating parameter pushes CCB-DF and LinUCB even further from the origin. These results indicate that methods with fixed exploration schedules are more sensitive to the tested dynamics errors, whereas the predictive controller changes less under the same simulator changes. The zero sensitivity of AutoCurriculum in some panels is therefore a ceiling artifact: it reaches the run limit in the scanned settings, rather than showing greater robustness. Tests with scoped simulator changes also keep the calibrated fatigue model fixed while increasing observation noise and varying transfer/interference strength. In additional scoped simulator-change tests, PianoMPC retained the lowest overall mean TTM and a competitive FeasibleRate when observation noise was increased by two to three times and transfer strength was scaled by 1.2. These results support robustness within the tested simulator changes, but they do not imply robustness to arbitrary model error.

Effect of fatigue-threshold and mastery-window choices

The experiment studies how evaluation hyperparameters affect reported performance. Only the fatigue threshold and the mastery window change, agent policies remain identical. Fig 11 reports both TTM and FeasibleRate, directly addressing whether the ranking depends on a single operational mastery or fatigue definition. It also compares PianoMPC, BayesianMAB, and CCB-DF on the balanced profile.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 11. Effect of changing evaluation hyperparameters.

Panels show TTM and FeasibleRate under threshold scaling and mastery windows.

https://doi.org/10.1371/journal.pone.0351141.g011

Panel (a) scales the fatigue limit from 0.8 to 1.2 of the nominal value. The PianoMPC curve stays between 22.5 and 25.5 steps. CCB-DF varies from about 35–45 steps and peaks exactly at the unscaled threshold. BayesianMAB moves inside the 31–36 step band and never overtakes PianoMPC. The corresponding FeasibleRate panel shows that the threshold sweep changes feasibility levels but does not remove PianoMPC’s TTM advantage. Thus the relative ordering in Section Main comparison under the safety guard is not caused by a single threshold choice.

The mastery-window sweep is limited to because longer windows act mainly as stress tests and can obscure the operational definition of TTM. The default W = 3 is marked in both mastery-window panels. PianoMPC remains the fastest method over the displayed window range, while FeasibleRate varies because changing W changes episode termination times and the fatigue samples included in the trajectory average. This behaviour shows that TTM is an operational PianoGym metric, not a universal definition of musical mastery.