Maintaining Homeostasis by Decision-Making

Living organisms need to maintain energetic homeostasis. For many species, this implies taking actions with delayed consequences. For example, humans may have to decide between foraging for high-calorie but hard-to-get, and low-calorie but easy-to-get food, under threat of starvation. Homeostatic principles prescribe decisions that maximize the probability of sustaining appropriate energy levels across the entire foraging trajectory. Here, predictions from biological principles contrast with predictions from economic decision-making models based on maximizing the utility of the endpoint outcome of a choice. To empirically arbitrate between the predictions of biological and economic models for individual human decision-making, we devised a virtual foraging task in which players chose repeatedly between two foraging environments, lost energy by the passage of time, and gained energy probabilistically according to the statistics of the environment they chose. Reaching zero energy was framed as starvation. We used the mathematics of random walks to derive endpoint outcome distributions of the choices. This also furnished equivalent lotteries, presented in a purely economic, casino-like frame, in which starvation corresponded to winning nothing. Bayesian model comparison showed that—in both the foraging and the casino frames—participants’ choices depended jointly on the probability of starvation and the expected endpoint value of the outcome, but could not be explained by economic models based on combinations of statistical moments or on rank-dependent utility. This implies that under precisely defined constraints biological principles are better suited to explain human decision-making than economic models based on endpoint utility maximization.

1 Random walk to generate gambles for virtual foraging task

General idea
We propose that decision making aimed at maintaining homeostasis (i.e., aimed at avoiding to die from hunger) can be investigated by using gambles that are derived within the mathematical framework of random walks.
• To maintain homeostasis, a biological agent has to keep its internal energy resources or energy points x above zero at any (discrete) time point i (with x, i ∈ N 0 ).
• In each trial (i.e., at each new foraging decision), the agent starts (the random walk) with internal resources x 0 at time point i = 0. Within each trial, the agent passes through n time steps (i.e., days).
• At each time step n = 1, the agent's internal resources incur a sure cost−c (with c ∈ N 0 ), which mirrors the consumption of energy (e.g., in terms of calories).
• To replenish the internal resources, the agent choses a risky foraging option and probailistically receives its outcomes at each time step n. That is, within a given time step n = 1 the agent can gain an amount g (with g ∈ N 0 ) with probability p. This would, for example, correspond to collecting berries or to hunting deer, where berries could provide a lower gain (less calories) but have a higher probability than deer. The probabilities of nding berries or hunting down deer could, for example, vary according to dierent seasons or environments.
• Alternatively, if the agent does not gain anything, the internal resources only incur the sure cost−c with probability q (where q = 1 − p).
• The gain is assumed to be equal or larger than the cost, otherwise the agent would not strive for it, therefore g ≥ c.
This situation represents a random walk starting at x 0 with • a step size of g − c and a probability p of going right and • a step size of−c and a probability q of going left.
The random walk has a lower absorbing barrier at x b = 0, which mirrors dying from hunger. Within this framework, gambles can be constructed as a function of p (and thus q), n, x 0 , c, and g.

Assumptions
A number of simplications are made: 1. The agent makes one single decision at the beginning of each trial (e.g., the agent decides whether to collect berries or to hunt deer and sticks to that decision throughout the number of days). That is, the gambles consist of compound lotteries, which comprise n sequential lotteries.
2. The agent does not deplete the food sources and does not get more procient at obtaining food. That is, the probabilities p and q are constant within each trial.
3. Similarly, cost c and gain g are constant within each round.
4. Costs do not dier between foraging options. For example, collecting berries and hunting deer are assumed to have the same costs (in terms of calories spent).

5.
Here, only a lower absorbing barrier is considered. An upper absorbing barrier (e.g., death due to overeating) is not included.
6. Dying from hunger represents the only threat to homeostasis (e.g., there are no predators).
7. Only a single variable (the amount of internal energy resources) has to be kept within a homeostatic range (e.g., there is no need to obtain specic nutrients and no conicts or opportunity costs with respect to other activities such as sleep or reproduction).

Outcome distributions of the random walk
In the following, the probability distributions of random walks will be described.
The description of random walks will increase in complexity until all features outlined above can be incorporated.

Simple random walk starting at zero
A random walk is called simple if steps to the right have the step size +1 and steps to the left have the step size −1 (i.e., if c = 1 and g = 2; since g − c is the step size to the right and since−c is the step size to the left). Let's assume the agent starts the random walk at zero. Let X n denote the random variable which indicates the position of the agent after n time steps. Then, Let W n denote the number of steps to the right within the rst n steps. Then W n has a binomial distribution.
If there are a steps to the right (+1) and therefore n − a steps to the left (−1) If n is even then X n is also even and if n is odd then X n is also odd. Therefore, if n and x are not either both even or both odd then P (X n = x) = 0. Put dierently, the range of X n is After each number of steps n there are n + 1 possible positions. That is, the range of X n contains n + 1 elements. If the agent has visited a certain position at a certain time step n, the agent can visit it again at n + 2, n + 4, . . . time steps. Another way of saying this is that if x and n are both even or odd P (X n = x) 0, otherwise P (X n = x) = 0. W n = a if and only if X n = 2a − n. Writing x = 2a − n, so that a = n+x 2 and n − a = n−x 2 , the probability distribution of X n is 1.3.2 Simple random walk starting at position x 0 Now, let's assume that the agent does not start the random walk at zero but at x 0 . Let X n (x 0 ) denote the random variable which indicates the position of the agent after n time steps in a simple random walk starting from x 0 . Then,

Random walk with unequal step sizes
Now, let's assume that step sizes to the right and step sizes to the left can take values = ±1.
Steps to the right have the size g − c and steps to the left have the size −c. Then, If the agent has visited a certain position at a certain time step n, the agent can visit it again at n + g, n + 2g, . . . time steps. Writing x = ag − nc + x 0 , so that a = nc+x−x0 g and n − a = n(g−c)−x+x0 g , the probability distribution of

Absorbing barrier at zero
So far the random walk has been unrestricted. That is, it had no (absorbing) barrier (and thus dying was not possible). In a random walk with an absorbing barrier at zero x b = 0 the range of X n (x 0 ) only includes elements ≥ 0. Conceptually, if one imagines the random walk as a tree, in which new branches are added at each time step n, one has to rst calculate the probability distribution within the full tree (i.e., without considering the barrier). Then, to calculate the probability distribution of a random walk with an absorbing barrier at zero, all downstream branches starting from visits at zero (or values below zero) have to be pruned (i.e., subtracted) from the full tree, which is given by the corresponding random walk without absorbing barrier.
To calculate the probabilities of starvation, i.e. the probabilities of reaching zero (or values below zero) within n time steps, one has to perform the following three steps.
1. Determine all time steps h i when hits of the barrier can occur (i.e., time steps n in which the agent in an unrestricted walk can be at x b = 0; with h ≤ n; i < n).
2. For all those hits of the barrier h i , calculate the probabilities of the agent being at x b = 0 for the rst time.
3. Add up the probabilities of the agent being at x b = 0 for the rst time at all time steps h i when hits of the barrier occur.
(Note that for step sizes to the left unequal to = −1, analogous calculations have to be made for all cases in which the agent can be at a position x < 0 without passing through zero. E.g. if c = 2, the random walk can go directly from +1 to −1. For simplicity these cases are not explicitly described below but the rationale is the same.) Hits of the barrier Determine the hits of the barrier h i within n time steps for all i (i.e., the rst time that the range of X n (x 0 ) includes zero). The agent can visit the barrier x b at h 1 , h 1 + g, h 1 + 2g, h 1 + 3g, . . . time steps until h i ≤ n.

Probabilities of hitting the barrier for the rst time The probability
for the agent in a random walk (starting from x 0 ) to be at x b = 0 for the rst time after n steps is dened as To calculate f n (x 0 ), one needs to calculate the probability for the agent in a random walk (starting from x 0 ) to be at x b = 0 not necessarily for the rst time after n steps, which is dened as There is no explicit formula for f n (x 0 ) for random walks with unequal step sizes and an absorbing barrier. But f n (x 0 ) can be found as a function of u n (x 0 ). At the rst hit of the barrier h 1 To nd the probability of hitting the barrier for the rst time at h 2 , let's consider the example of a simple random walk starting at one (i.e., x 0 = 1). The agent can hit x b = 0 after 1, 3, 5, . . . time steps (i.e., h 1 , h 1 + g, h 1 + 2g, . . . time steps; and thus h 1 = 1, h 2 = 1 + g, h 3 = 1 + 2g, . . . ). There are two mutually exclusive ways in which the agent can be at x b = 0 at the second hit of the barrier h 2 = 3.
• First, the agent visits zero for the rst time at h 2 (i.e., at h 2 = 3 with probability f h2 (x 0 )).
• Second, the agent repeatedly visits zero. That is, the agent visits zero for the rst time at h 1 (i.e., at h 1 = 1 with probability f h1 (x 0 ) = u h1 (x 0 )) and returns to zero after g = 2 further time steps. That is, the probability of going from the rst hit of the barrier h 1 to the second hit of the barrier h 2 within g = 2 time steps is u g (x b ). Therefore, the probability of having repeatedly visited zero at h 2 is Since the rst and the second way of reaching zero are mutually exclusive, they can be added up. Therefore, at the second hit of the barrier h 2 More generally, there are i mutually exclusive ways in which the agent can hit the barrier x b = 0 at h i (with probability u hi (x 0 )).
• First, the agent visits the barrier x b for the rst time at time step h i (with probability f hi (x 0 )).
That is, the probabilities of going from the hits of the barrier h i−1 , h i−2 , h i−3 , . . . , h 1 to the hit of the barrier h i are u g ( Therefore, the probability of having repeatedly visited the barrier Note that the values of the indices within a product have to add up to h i . Note also that the range of X n contains n + 2 − i elements for i 1 (where i are the number of hits of the barrier).
Adding up probabilities of hitting the barrier for the rst time To nd the probability of having reached the absorbing barrier at x b = 0 within n time steps (i.e., the probability of starvation p starve ), add up the probabilities of being at x b = 0 for the rst time at all possible hits h i of the barrier.
2 Calculating statistical moments 2.1 Expected value To calculate the rst statistical moment, i.e., the expected value of a random walk with an absorbing barrier x b (at zero) at time step n, one has to rst calculate the probabilities of all elements P (X n (x 0 ) = x) within the range of X n (x 0 ) and then take the sum of all those probabilities multiplied by their respective values P (X n (x 0 ) = x) x.
The rationale is similar as above for nding the probabilities of hitting the absorbing barrier. To nd the probabilities of being at a certain position x in a random walk with an absorbing barrier (at zero), one has to rst calculate the probability distribution within the corresponding full tree (i.e., without considering the barrier). Then, all downstream branches starting from visits at zero (or values below zero) have to be pruned (i.e., subtracted) from the full tree. The time steps when the random walk can hit zero h i and the probabilities when the random walk reaches zero f hi (x 0 ) have already been determined above.
• First, calculate the probability of being in x in the corresponding full tree without absorbing barrier P f ull . (This probability corresponds to u hi (x 0 ) above.) P f ull (X n (x 0 ) = x) • Second, calculate the downstream branches. That is, the probabilities of going from the barrier x b to x (at all time steps h i when the barrier was hit). (These probabilities correspond to u g (x b ), u 2g (x b ), u 3g (x b ), . . . , u (i−1)g (x b ) above.) P (X n−hi (x b ) = x), P (X n−hi−1 (x b ) = x), P (X n−hi−2 (x b ) = x), . . . , P (X n−h1 (x b ) = x) The probability of being in position x in a tree with an absorbing barrier at The expected value (EV ) is the weighted sum over all J elements x 1 , x 2 , . . . , x J within the range of X n (x 0 ). The number of elements J is n + 2 − i for i 1 (where i are the number of hits of the barrier; for i = 0, i.e., no hits of the barrier, J equals n + 1).
Variance and skewness The second and third statistical moments, i.e., variance (V ar) and skewness (Skw) are calculated as follows