1.1 Related Work

In the last decade, the number of APTs [1] increased rapidly and numerous related security incidents were reported all over the world. One major reason therefore is that APTs are not focusing on a single vulnerability in a system (which could be detected and eliminated easily), but are using a chain of vulnerabilities in different systems to reach high-security areas within a company network. In this context, adversaries often exploit the fact that most of the protection efforts go into perimeter protection, so that moving inside the infrastructure is much easier and the attacker has a good chance to go unnoticed once being inside. Overcoming the perimeter protection by social engineering or malware (even unknowingly) carried inside by legitimate persons (bring-your-own-device problem) are only two ways to penetrate the perimeter security. Once the perimeter has been overcome, insider attacks are considered as an even bigger threat [2]. Extensive guidelines and recommendations exist to secure this internal area [3], e.g., the demilitarized zone (DMZ) but the intensity of the surveillance is limited. Specialized tools for intrusion detection or intrusion prevention require a large amount of administration and human resources to monitor the output of these systems.

APTs are characterized by a combination of several different attack methods (social engineering, technical hacks, malware, etc.) that is being tailored to and optimized for the specific organization, its IT network infrastructure and the existing security measures therein. Often, even yet not officially reported weaknesses, known as zero-day vulnerabilities, of the network infrastructure are in additional use. Especially the application of social engineering in the beginning stages of an APT lets the attacker bypass many technical measures like intrusion detection and prevention systems, so as to efficiently (and economically) get through the outer protection (perimeter) of the IT network. A prominent APT attack was the application of the Stuxnet malware in 2008 [4–6], which was introduced into Iran’s nuclear plants sabotaging the nuclear centrifuges. In the following years, other APT attacks, like Operation Aurora, Shady Rat, Red October or MiniDuke [1, 7, 8] have become public. Additionally, the Mandiant Report [9] explicitly states how APTs are used on a global scale for industrial espionage and that the attackers are often closely connected to governmental organizations.

The detection of APT attacks has therefore become an object of extensive research over the past years. As perimeter protection tools are occasionally failing to prevent intrusions, anomaly detection methods have been inspected to provide additional protection [10, 11]. The main idea is to detect the presence of an adversary inside an organization’s network, based on the adversary’s actions when it moves from one spot to another, or tries to access sensitive data (honeypots). Often, the detection rests on log file analysis, with data collected from all over the network and applications therein. Designated logging engines (e.g., syslog (http://tools.ietf.org/html/rfc5424) or logging management solutions (e.g., Graylog (http://graylog2.org/) are usually in charge here. Nevertheless, the detection of exceptional events in these log files alone is insufficient, since anomalies are often exposed not before events are correlated with each other [12, 13]. Since today’s systems are heavily connected and interchange a large amount of data on a regular basis, the size of the logging information increases drastically, making an evaluation quite difficult.

An example for a tool realizing this approach is AECID (Automatic Event Correlation for Incident Detection) [14, 15]. AECID enforces white-lists and monitors system events, their occurrences as well as the interdependencies between different systems. In the course of this, the system is able to get an overview on the “normal” behavior of the infrastructure. If some systems start to act differently from this normal behavior, an attack is suspected and an alert is raised.

Whereas AECID (and similar tools) are detective measures (as they trigger alerts based on specific events that have happened already), our approach in the following is preventive in the sense of estimating and minimizing the risk of a successful APT from the beginning (cf. 1.2). Game theory is here applied to optimize the defense against a stealthy invader, who attempts to sneak into the system on a set of known paths, while the defender does its best to guard all these ways simultaneously. This is the abstract version of the situation that is normally summarized under the term APT.

Game theory appears as a natural tool to analyze conflicts of interest, such as obviously arise between the defender and the attacker mounting an APT. Powerful techniques to defend against stealthy takeover have been defined (partially originating from [16, 17] but also based on a variety of precursor and independent approaches, such as collected in [18]), but a method that fits into established risk management processes and can be instantiated with vague, fuzzy and qualitative risk assessments (such as uttered by domain experts) is demanding yet missing. Particularly intricate are matters of social risk response, say, if an enterprise seeks to minimize losses of reputation besides direct costs; assessing the public community’s response to certain actions being taken is a vague and difficult issue, to which sophisticated game theoretic [19–22] and agent based models [23] can be applied for an analysis and risk quantification. A recognized feature of any game-theoretic treatment of APT and in general every cyber-security scenario is the lack and asymmetry of information (say, the absence of knowledge about the attacker’s strategy spaces or payoffs, cf. [24, 25], while the attacker may have full information about the target system). This asymmetry is even stronger than what can be captured by many game-theoretic models, since organizational constraints may enforce the defender to act only at certain points in time, while the attacker is free to become active at any time. That is, the game is discrete time for one player, but continuous time for the other player—a setting that is hardly considered in game-theoretic literature related to security, and as such a central novelty in this work.

As we will show later (cf. Section 10.1 and Lemma 9), matrix games are nonetheless a proper model to account for what the defender can do against an APT, if we confine ourselves with the goal of playing the game to the best of our own protection and allow the outcomes to be random and unpredictable. Under this relaxation over the conventional game theoretic modelling, we can account for the outcome to be dependent on an action that is taken at different points in time, and especially also for actions that were interrupted before they could carry to completion. This addresses the issue identified by [26], who pointed out that moves may take a variable amount of time rather than being instantaneous (and thus atomic).

Ultimately, a significant obstacle for practitioners in the application of any game theoretic model is the lack of understanding of the ingredients to the game. That is, no matter how sophisticated the model may be, it nevertheless needs to be instantiated with whatever data is available. In many cases, this data is either qualitative (fuzzy) expert knowledge (formulated in some taxonomy, e.g., [27]) or obtained from simulation (see [28] for one example). Either may not be suitable to instantiate the proper APT model, even though the APT-game model would be quite sophisticated and powerful (such as [29]) in its capabilities for risk mitigation. In any case, this takes us to empirical game-theoretic models; a category into which this work falls.

1.2 Our Contribution

We present a novel form of capturing payoff uncertainty in game theoretic models. We deviate from standard games in the conceptual way of measuring the outcome of a gameplay not in crisp terms, but by an entire probability distribution object. That is, we play game theory on the abstract space of distributions for the following reasons:

1. The specification of losses and payoffs in a game is often difficult: how would we accurately quantify the results of a defense in light of an attack? Do we count the number of infected machines (such as done in [16])? Shall we work with monetary loss (causing difficulties in how to “price-tag” loss of reputation or consumer’s trust)? Conversely, can we play games over a categorical scale of payoffs, such as risk is being quantified in many standards like ISO 31000 [30] or similar?
   We can elegantly avoid any such issue by letting the game being defined by any outcome that can be ordered (as in conventional game theory), but in addition, allowing an action to have many different random outcomes (this is usually not possible in standard games). In doing so, we gain a considerable flexibility and degree of freedom to tackle a variety of issues, which we will discuss later on.
2. There is a strong asymmetry in the player’s information in many senses: first, the game structure itself is not common knowledge, since the defender knows only little about the opponent, while the opponent knows very much about the defender’s infrastructure (as lies in the nature of APTs, since these typically include an a-priori phase of investigation and espionage). Second, the game play is different for both players, since moves are not mutually observable, nor must happen instantaneously or even at the same times.
   Again, this can be captured by letting the effects of action be nondeterministic and random even if both, the attacker’s and defender’s action were both known.
3. Any game-theoretic model for security may itself be only part of an outer risk management process, and as such must be “compatible” with the surrounding workflows, which cover APT mitigation among other aspects. That is, the game theoretic model’s input and output must be useful with what the risk management process can deliver and requires. Our APT-games will be designed to fulfil this need.
4. Conventional stochastic models like Bayesian games indeed also capture uncertainty, but do so by letting the modeler describe a variety of different possible game structures, among which nature chooses at random in the actual gameplay. While different such structures can embody different outcomes, and the likelihood for these can be specified as a distribution (similar to what we do), each of these possible game structures must be specified in the classical way, thus effectively “multiplying” the problems of practitioners (if one game is difficult to specify, the specification of several ones does not appear to ease matters). Our approach avoids these issues by working with empirical data directly, and keeping the game models simple at the same time.

In light of the last point in particular, we will restrict our attention in the following to the problem of how to define games over qualitatively assessed outcomes that may be random. That is, the central question that this work discusses is essentially a form of reasoning under uncertainty:

Given some possibilities to act, what would be the best choice if the consequences of an action are intrinsically random?

We will show how to answer this question if the randomness can be modeled in the most general form by specifying probability distributions for the outcome. However, unlike normal optimization that maximizes some numeric quantity derived from the distribution of a random variable X, our games will optimize the shape of X’s distribution itself.

The presentation will heavily use examples for illustration, yet the concepts themselves will be described and also defined in a general form. To get started, consider the following example of decision making under the setting that we consider. Example 1 is about the protection of intellectual property rights (IPR), whose theft can be a reason to mount an APT.

Example 1 (Assigning IPR Responsibilities). Assume that an enterprise runs a project and is worried about protection of IPR. To mitigate this issue, one or more persons shall be put in charge of IPR protection. For this, say, three options are available:

1. Assign IPR responsibility to one person: This will increase the workload of the employee, and must be made w.r.t. available resources and skills. Neither is precisely quantifiable nor may be sufficient at all times. Thus, even assuming a strong commitment of the person to its role, some residual risk of damage occurring remains (human error of subordinates cannot be ultimately ruled out despite any strong supervision).
2. Assign IPR responsibility to a team of two or three persons: Resources and skills may be much richer in this setting, but there is a danger of mutual reliance on one another, such that in the worst case, no-one really does the job (as a result of social coordination failure). Chances for this worst case to occur may be even higher than for option 1.
3. Do security/ IPR training sessions: here, we would completely rely the joint behavior of the employee’s and their commitment to the confidentiality of project content and adherence to the training’s messages. Nevertheless, chances for IPR loss due to human error (e.g., an unencrypted email leaking confidential information, or similar) may be lowered only temporarily, so that the training would have to be repeated from time to time.

The optimal choice in Example 1 is not obvious, since consequences are not all entirely guaranteed for always foreseeable. Intuitively, we would go with the setting under which loss of intellectual property is least likely. Later, in Section 4, we will construct an ordering relation ⪯ that does exactly this if the loss distributions are defined on a scale of damage whose maximum is the loss of intellectual property (e.g., quantified by the business value attached to it). So, if we are somehow able to model possible random outcomes in each of the three scenarios, we can (algorithmically) compute the “best” (i.e., ⪯-minimal) loss distribution to be the best choice among the three above. This is a matter of loss distribution specification, which we will discuss in Section 5.1.

Finally, we remark that all of the theory sketched here has been implemented in R (including full fledged support of multi-criteria game theory based on distributions as discussed in section 8.1), to validate the method and to compute the results for the examples (such as in Section 9) shown here.

1.3 Organization of the paper

Section 2 briefly introduces the tools and concepts that our models are built upon. We will strongly rely on TVA (see Section 2.1) and human expertise in our model building, which we believe to be a viable approximation of how security risk management works in practice. Section 3 presents an example, which we will carry through the article to illustrate the concepts and approach as a whole. Section 4 introduces the theory of how decisions can be made if their outcome is rated through an entire probability distribution object (rather than a number), and Section 5 takes this basis to define games, equilibria and to highlight similarities but also an important qualitative difference between the so-generalized games and their classical counterparts. Sections 6 and 7 apply the framework to APTs by picking up the example from Section 3, and give algorithmic details on how to practically work out results (the aforementioned differences between our and classical games call for various mathematical tricks here). Section 8 briefly discusses generalizations towards multi-criteria decision making. Section 9 finishes the example from Section 3 by presenting results and security protection advices obtained from our game-theoretic APT mitigation game. Section 10 presents a critical discussion in terms of answering direct questions that were collected from practical experience with the proposed method in a research project (see the acknowledgment section at the end of the paper). Conclusions are drawn in Section 11.

2.1 Topological Vulnerability Analysis

Topological vulnerability analysis [31] is the systematic identification of attacks to a system, based on the system’s structure and especially its network topology. The process usually consists of creating a complete picture of the infrastructure augmented by all available details about the components. Modeling the system’s topology as an (undirected) graph G(V, E) with a designated target node v₀ ∈ V, we can use standard path searching algorithms to identify paths from the exterior of G towards the target node v₀. Whatever structure is dug up by the TVA, an immediate question concerns the applicability of known attack patterns to the infrastructure model (similar to virus patterns being looked up in software). Graph matching techniques (see [32, 33] for example) appear as an interesting tool to apply here. The known (or suspected) vulnerabilities/exploits related to the nodes in V then determines which paths are theoretically open to v₀ to successfully attack the system. These attack paths are thus sequences of vulnerabilities (augmented with the respective preconditions to exploit a vulnerability), and are the main output of a TVA. An APT can then (in a slightly simplified perspective) be viewed as the entirety of attack paths, and is physically mounted by sequentially working along a chosen attack path in a way that avoids detection of the attack at all stages (stealthy). Particular practical risk arises from exploits of not yet known vulnerabilities, which are commonly called zero-day exploits. Uncertainty about these partially roots in the complexity of the network, so that graph entropy measures (see, e.g., [34]) may be considered as a measure to help quantifying the chances of attacks coming over paths that were missed during the analysis. The practical handling of this residual risk is often a matter of using domain knowledge, collecting expert opinions, experience and information mining, combined with suitable mathematical models (e.g., [35, 36]). Our model immanently includes a zero-day vulnerability measure, as will be discussed in Section 7.2.

2.2 Attack Graphs and Attack Trees

The entirety of ways into a system, including intersections and alternative routes on attack paths makes up the attack graph [37]. It is essentially a representation based on the system topology G, in which outgoing links of a node v are retained only if some exploit on v enables reaching v’s neighbor (see Figs 1 and 2 for an example). In terms of representation, an attack graph is to be distinguished from an attack tree, which is usually an AND/OR tree representing the possible exploit chains in a different way. Regardless of which is available, the main object of interest for our purposes is the set of attack paths, which directly corresponds to the action set of player 2 in our APT-games.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Infrastructure from [38] to illustrate game-theoretic APT modeling.

https://doi.org/10.1371/journal.pone.0168675.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Example Attack Graph [38].

https://doi.org/10.1371/journal.pone.0168675.g002

2.3 Extensive Form Games

Towards a game theoretic model of APT, we will use extensive form games (EFGs). While a full fledged formal definition of EFG is lengthy and complex, it will suffice to give a description of it to highlight the similarities to APTs. Formally, EFG are described by a tree T(V_T, E_T), with a designated root node that represents the starting stage in the game. Consequently, T has edges directed outwards from the root. For an APT, the root corresponds to the (hypothetical) point representing the exterior of the network graph G. The EFG is played between a set of (for our purposes two) players, including a hypothetical player “chance” that represents random moves in the game. Each node v ∈ V_T in the game tree T(V_T, E_T) carries an information on which player is currently at move (including the “chance” player). Furthermore, moves that are indistinguishable by other players are collected in a player’s information set. This, from the opponent’s perspective, represents the uncertainty about what a player has currently done in the game. In an APT model, the information set would correspond to possible locations where the attacker could currently be (again, recalling that an APT is stealthy). The EFG description is completed by assigning a vector of outcomes to the leaf nodes in the game tree. Normally, the outcomes are real values and specified for all players. Viewing an APT as an EFG, we would thus require to specify our own damage when the APT has been carried to the end (i.e., the target node v₀ has been reached), but also the payoff to the adversary would needed to be known. The latter is a practical issue, since the uncertainty in the game is not only due to the attacker’s moves themselves, but also caused by external influences outside any of the player’s influences. Shifting all this uncertainty induced exteriorly to the chance-node in the EFG description appears infeasible, since much of it may depend on the particular action and current (even past) moves of both players (defender being player 1 and the attacker being player 2 in an APT game). However, given that the two players have different information on the current stage of the game play (only the attacker knows its precise position, the defender knows nothing; not even the presence of the attacker is assured), defining the information sets appears hardly doable.

Since the concept of EFG as being games with imperfect information essentially rests on information sets, any best behavior in the game play will inevitably rest on hypotheses on the player’s moves. For an EFG, we would describe these hypotheses as probability distributions on the information sets. In lack of these, we can only model the outcome based on the known defender’s actions and assuming the attacker to be possibly everywhere in the system. Practically, these hypotheses will rarely be available as hard figures and mostly come in qualitative terms like “low”, “medium” or “high” risk. Classical game theory is not naturally designed to work in such fuzzy terms.

Finally, an assumption of frequent criticism concerns the game model to be common knowledge to both players. This is certainly questionable in APT scenarios. In fact, a defender may in practice only have limited and widely uncertain information about the attacker’s incentives, current moves, current location or even its presence in the game. Thus, the information set would in the worst case cover the entirety of the game graph, and neither is the payoff to the attacker precisely quantifiable in most cases.

To avoid all these issues, we propose to replace the attacker’s payoffs by our own losses (in an implicit assumption of a zero-sum competition), in which an equilibrium behavior is a provable bound (see Lemma 9) to the payoff for the player having modeled the game. Second, we avoid difficulties of uncertain payoffs by defining the game play itself as a one-shot event, in which both players choose their strategies for the round of the game and the payoff is determined by that choice (thus, shifting all matters of uncertainty about where a player is in the game entirely to the payoffs). To this end, we will model an APT as a game with complete information but uncertain payoffs. In fact, the payoffs will be entire probability distributions rather than numbers.

4.1 Optimal Decisions if Consequences are Uncertain

Definition 2 assures that the density function f_L of any random loss L admits moments E(Lⁿ) of all orders , so that we get a well-defined condition for an ordering based on moment sequences:

Definition 3 (⪯-Preference between Loss Distributions). Let L₁ ∼ F₁, L₂ ∼ F₂ be losses with . We prefer L₁ over L₂, written as L₁ ⪯ L₂, if there is an index so that for all k ≥ k₀. We synonymously write F₁ ⪯ F₂ whenever we explicitly refer to distributions rather than RVs.

A minor technical difficulty arises from the yet unsettled issue of whether or not there are non-isomorphic instances of , in which case we could get ambiguities in the ⪯-ordering. The next result, however, rules out this danger.

Proposition 4 (cf. [44]). The set with as in definition 2 and ⪯ as in definition 3 is a totally ordered set, where F₁ ⪯ F₂ implies ϕ(F₁) ≤ ϕ(F₂), with the embedding , and the ⪯-ordering on is invariant w.r.t. how is constructed.

While the theoretical definition is easy, important practical questions about this preference demand an answer, in particular:

1. What is the practical meaning of the ⪯-ordering for risk management?
2. If ⪯ is practically meaningful, how can we (efficiently) decide it?

Let us postpone the answer to the first question until Section 4.3, and come to the algorithmic matters of deciding ⪯ first. The answer to the second question will then also deliver the answer to the first one.

4.2 Practical Decision of ⪯-Preferences

Let us first discuss the case where the loss distribution is continuous. Common examples in risk management (cf. [45]) are extreme value distribution or stable distribution (with fat tails). Although such distributions may not necessarily have a bounded support (thus not corresponding to a random loss in the sense of definition 2), we can approximate the distributions by random losses via defining a risk acceptance threshold , and truncating the distribution outside the range [1, a]. The concrete value a can be chosen upon a desired accuracy ε > 0, for which we can choose a large enough to have the residual likelihood of damage > a is smaller than ε, or formally, Pr(L > a) < ε (we will come back to the choice of a in section 10.2).

Practically, the risk acceptance threshold is the value above which risks are simply “being taken” or are covered by proper insurance contracts. Thus, specifying the value a and truncating the loss distributions accordingly makes distributions with fat and/or unbounded tails fit as approximate versions into definition 2.

If L₁, L₂ have the same compact support , and since the respective density functions f_L1, f_L2 are assumed continuous, both admit limits b₁ = lim_{x → a} f_L1(x) and b₂ = lim_{x → a} f_L2(x). For the moment, assume b₁ ≠ b₂, i.e., f_L1(a) ≠ f_L2(a) (the case of equality is treated later). The continuity of both functions implies that f_L1(x) ≠ f_L2(x) holds in an entire left neighborhood (a − ε, a] of a for some ε > 0. It is then a simple matter of calculus to verify that (since both L₁, L₂ ≥ 1), the speed of divergence of the respective moment sequences and is determined by which density function takes larger values in the region (a − ε, a], recalling that both densities vanish at x > a. That is, we have (2) and the condition of Definition 3 is ultimately satisfied (in either way).

Lemma 5. Let L₁, L₂ be two random loss variables with continuous distribution functions , and let f₁, f₂ denote the respective densities. If both RVs are supported in an interval [1, a] for , and there is some ε > 0 such that f₁(x) < f₂(x) for all x ∈ (a − ε, a], then L₁ ⪯ L₂.

Lemma 5 (see [44] for a proof) offers an easy way to decide preferences based on the RV’s density functions only. The procedure is the following: Call [1, a] the common support of both loss variables L₁, L₂, and consider the density functions f₁, f₂:

• If f₁(a) < f₂(a), then L₁ ⪯ L₂,
• Otherwise, if f₁(a) > f₂(a), then L₂ ⪯ L₁.

Upon a tie, i.e., f₁(a) = f₂(a), we need to either decrease a, truncate the distributions properly, and repeat the analysis, or we may look at derivatives at a to tell us which density takes larger values locally near a. The latter approach is further expanded in Section 7.1.

If the distribution is discrete, say, if the available data is not continuous but qualitative (e.g., categorical), then things are even simpler: if L₁, L₂ are both distributions over the same categories, then L₁ ⪯ L₂, if L₁ puts less likelihood to categories of large damage than L₂ (see Fig 5 for an example).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Example of ⪯-choosing among two empirical distributions (inconsistent expert opinions).

https://doi.org/10.1371/journal.pone.0168675.g005

Formally, ⪯ thus boils down to a humble lexicographic ordering whenever the losses have categorical distributions.

Definition 6 (lexicographic ordering). For two vectors and of not necessarily the same length, we define if and only if there is an index i₀ so that x_i0 < y_i0 and x_i = y_i whenever i < i₀.

For two categorical distributions given in matrix notation and letting the support be given in descending order of risk levels r_n > r_n−1 > … > r₁, we observe that , when the distributions are:

That is, the action with the higher likelihood of extreme damage is less favorable, and upon a tie (equal chances of large damages), the likelihood for the next smaller risk level tips the scale, etc.

4.3 Practical Meaning of ⪯-Preferences

Summarizing the previous discussion in concise form directly takes us to the practical meaning of ⪯-preferences:

We have L₁ ⪯ L₂, if large damages (near the maximum a) are more likely to occur under L₂ than under L₁.

This is just an intuitive re-statement of Lemma 5. However, and remarkably, the converse to it is also true, which is the following:

Theorem 7 (Thm 2.14 in [44]). Let L₁, L₂ be two RVs with distribution functions . If L₁ ⪯ L₂, then a threshold x₀ exists such that Pr(L₁ > x) < Pr(L₂ > x) for every x ≥ x₀.

Restating this intuitively again, Theorem 7 tells that:

a ⪯-minimal decision among two choices with respective consequences L₁ and L₂ minimizes the chances for large damages to occur.

This is exactly what we are looking for: Risk management is in many cases focusing on extreme events rather than small distortions (which the system’s “natural” resilience is expected to handle anyway), and the focus of the ⪯-relation to prefer distributions with lighter tails perfectly accounts for this. Having ⪯ as a total ordering with a practical interpretation as being “risk-averse”, this already addresses the simple case of decision making among finitely many choices (as discussed in the next Section 5).

5.1 Constructing Loss Distributions

The simplest approach to construct loss distributions that satisfy definition 2 is to either:

• collect as much data as is available, and compile an empirical distribution from it,
• or define the loss distribution directly based on expertise (say, if the action’s incurred loss has a known distribution), if this is possible.

The latter case may occur seldom in practice, unless the particular threat has been studied specifically (such as disaster management or value at risk calls for extreme value distributions, etc.), and the “adversary” is nature itself. Against a rational adversary such as business competitors, hackers, etc., threat intelligence and expertise is the fundament upon which loss may be measured. Often, this assessment is made in qualitative terms for several (good) reasons, such as:

• Human reasoning is neither numeric nor crisp, i.e., experts may find it simpler to give assessments like “high risk” instead of having to specify a hard figure.
• Numerical precision can create the illusion of accuracy where there is none. There are only few types of incidents on which reliable statistical data is available, and having huge amounts of data on APTs attacks on general cybersecurity incidents may be unrealistic (and also undesirable if the incidents concern oneself).

In practice, rating actions w.r.t. their outcomes is naturally a matter of expert surveys, with answers possibly looking like shown in Table 2. Collecting many such opinions and putting them together in an empirical distribution about a precaution’s performance may give distributions whose shape is unimodal (if a consensus among opinions is found), or multimodal, if disagreeing opinions are reported. Whatever happens or whether or not the outcome looks like illustrated in Fig 3, the ⪯-preference relation now allows for an elegant deal with this kind of uncertainty.

5.2 Games and Equilibria

With the uncertain outcome in a scenario of defense i vs. attack j being captured by a (perhaps empirical) probability distribution L_ij, and the complete set of distributions being totally ordered w.r.t. ⪯, it is a simple and straightforward manner to define matrix games and equilibria in the well-known way. For convenience of the reader, we give the necessary concepts and definitions here.

Let AS₁, AS₂ be the action spaces for player 1 and 2, respectively, with cardinalities n and m. Let be a matrix of RVs that are all supported on the same compact set . Let F_ij be the distribution function of the random loss L_ij. In each round of the game, the random outcome R is conditional on the chosen actions of player 1 and player 2, and has the distribution R ∼ L_ij if player 1 chooses action i ∈ AS₁ and player 2 chooses action j ∈ AS₂.

We consider randomized choice rules and , i.e., the vectors p, q describe the likelihoods of actions being taken by either player. In that case, the random outcome has a distribution R ∼ F (p, q) computed from the law of total probability, which is (3) assuming a stochastically independent choice of actions by both players. This is the utility function in case of random outcomes (note that Eq (3) is exactly the same formula as is familiar from matrix game theory). So, the actual gameplay is not about maximizing the average revenue (as usual in game theory), but towards optimally “shaping” the outcome distribution F (p, q) towards ⪯-minimality. That is, in a zero-sum competition, player 1 and player 2 seek to choose their actions in order to minimize/maximize the likelihood of extreme events. Speaking differently again, player 1 attempts to shift the mass allocated by the respective density f (p, q) towards lowest damages, whereas player 2 tries his best to shape the density f towards putting more likelihood on larger damages. This is the essential technical process of our game-theoretic APT risk mitigation strategies, whose optimality is that of (standard) game-theoretic equilibria in zero-sum matrix games (see [46] for a formal treatment).

As for standard games, it can be shown that the saddle-point value F (p, q)is invariant w.r.t. different equilibria, and that equilibria defined w.r.t. ⪯ exist (and can be generalized to Nash-equilibria in n-person games in the canonic way). The way of proving it makes use of the embedding of distributions into the hyperreal space , where all the known results necessary to re-establish the fundament of game theory are available (yet further substantiating our loss representation by a moment sequence). Unfortunately, however, not all properties are directly inherited, such as a central computational feature of zero-sum games is absent in our setting:

Proposition 8. There exist zero-sum matrix games for which fictitious play (according to [47, 48]) does not converge.

Proposition 8 is proved by constructing a concrete example (see [49]) of a game that cannot be solved using fictitious play. Thus, it is an unfortunate obstacle in applying well-known game theory to our new setting. The formal fix relies on yet another representation of the loss densities, which admits converting a matrix game over into a set of standard matrix games over , which can be solved by fictitious play again. The details of this are postponed until Section 7, culminating in the main Theorem 14 that assures that we can ultimately escape the situation that proposition 8 warns us about. Interestingly, this fix has a useful side-effect, whose physical meaning is a heuristic account for zero-day exploits. We will revisit this aspect in more detail later in Section 7.2.

6.1 Identifying Mitigation Strategies

Having the attack graph and once attack vectors have been derived from it, all of which are collected in the adversary’s action space AS₂, the next step is the identification of mitigation strategies. This process is a standard phase in many risk management practices, and often based on known countermeasures against the identified threats (accounting for unexpected events is a matter of zero-day exploit handling, which we will revisit shortly in Section 7.2). Since the process of defining countermeasures is a task that highly depends on the attack vectors AS₂, we cannot define a general purpose procedure to identify AS₁ here (it is individual and different for various infrastructures). For the sake of generality and conciseness of presentation in this work, let us therefore assume that all relevant defense actions are available and constitute the action set AS₁ for the defender. It may well be the case that not all actions are effective against all threats, and neither may a designated countermeasure i be necessarily effective against threat j. In that case, we may pessimistically assume maximal damage to be likely in scenario (i, j). Matching all defenses in AS₁ against all attack vectors in AS₂ is a matter of defining the game’s loss distributions, which is the next step towards completing the game model in Section 6.2. To simplify the notation in the following, let us abstractly denote the action spaces as AS₁ = {1, 2, …, n} and AS₂ = {1, 2, …, m}, with the specific details of the i-th defense and the j-th attack for all i, j being available in the risk management documentation (in the background of our modeling).

6.2 Defining the APT Game

Towards a matrix game model of APTs, it remains to specify the outcome of each attack/defense scenario. To capture the intrinsic uncertainty here, we will resort to a qualitative assessment like sketched above and/or arising from vague opinions like Table 2 illustrates. The APT game is then defined upon loss distributions according to definition 2 to describe the potential loss in each scenario (i, j) ∈ AS₁ × AS₂. In cases where we are unable to come up with a reasonable guess on the distributions, a pessimistic approach towards a worst-case assessment could work as follows (we will later revisit the issue in the discussion Section 10.2):

1. Fix a particular position in the network, i.e., a certain point where an attack is considered. Let the hypothesized attack be the one with index j₀ ∈ AS₂.
2. Fix a defense action i ∈ AS₁.
3. In lack of better knowledge, assume a uniform distribution of all possible j, including j₀, and rate the success probability p_j0 of the defense conditional on j = j₀ (i.e., if your guess was right). This rating can also be made conditional on the expected “doability” of the current defense (e.g., if the defense means patching, the patch may not be available at all times, or it may be ineffective). Fig 6 displays the process as a decision tree, in which the worst case outcome (highlighted in gray) is taken if either the countermeasure is considered as possibly effective but may still fail (with a certain likelihood), or if the countermeasure is not applicable at all. In any case, the expert—based on the assumed attacker’s behavior—is not bound to confine her/himself to a single answer, and may rate all the possibilities with different likelihoods
4. If possible, collect many such assessments (say, from surveys, simulations, etc.), and compile an empirical distribution from the available data. This empirical distribution is then nothing else than a histogram recording the number of uttered opinions (shown as the bar chart in Fig 6) Note that the uniformity assumption on the attacker’s location can be replaced by a more informed guess, if one is available. For example, the adversarial risk analysis (ARA) framework [50–52] addresses exactly this issue.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Loss Assessment of Counteraction vs. Threat.

https://doi.org/10.1371/journal.pone.0168675.g006

This procedure is repeated for the entirety of scenarios in AS₁ × AS₂, i.e., until the full game matrix has been specified. Fig 7 illustrates the three sub-steps per entry of this procedure in the game matrix (note that the right-most decision path (bold-printed in Fig 6) is reflected as the fourth choice option in the example survey shown in Fig 7). We stress that the full lot of |AS₁| ⋅ |AS₂| may hardly be necessary to put to a survey, since not all actions are effective against one another (defenses may work against specific threats, so only a few combinations in |AS₁| ⋅ |AS₂| need to be polled explicitly, and others can rely on default settings; cf. Fig 6). Furthermore, some loss distributions can be equally well computed from simulations (cf., e.g., [53]).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Specification of an APT Game (Example Workflow Snapshot).

https://doi.org/10.1371/journal.pone.0168675.g007

By construction of the total ordering, the game that we define to minimize the loss would then be played towards minimizing the likelihood for large damages (by Theorem 7). Returning to our example sketched in Section 3, the uncertainty in a risk level quantification based on distance in the graph would thus mean that the gameplay is such as to keep the adversary “as far away as possible” from its target. This is indeed what we would naturally expect, and the ⪯-relation acting on loss distributions that are based on distance achieve precisely this kind of defense.

Although this modeling of APTs is heavily based on (subjective) expertise and manual labour, it fits quite well into standard risk management processes (such as ISO 31000 [30] or ISO 27005 [54]), and nevertheless greatly simplifies matters of modelling over the classical approach, as a variety of issues are elegantly solved. A selection is summarized in Table 6, with a complementary discussion given in Section 10.2. Additional help in the specification of risk assessments is also offered by thinking about costs of an exploit or known ratings of vulnerabilities such as by common vulnerability scoring system (CVSS) [55]. Such ratings are commonly delivered along with the TVA (e.g., by tools like OpenVAS).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Benefits of Distribution-Valued Game-Modeling over Classical Game-Modeling.

https://doi.org/10.1371/journal.pone.0168675.t006

7.1 Computing Optimal Defenses (Equilibria)

To avoid proposition 8 to apply for our APT-games, we need to assure that all loss distributions share the same support (the counterexample used to prove proposition 8 relies on different losses whose supports that are strictly contained in one another). To this end, we will introduce another representation of a loss distribution as a sequence, so that the lexicographic ordering <_lex on the new sequence equals the ⪯-ordering of definition 3.

To make things precise, let us assume that a particular empirical loss distribution has been compiled from the data x₁, x₂, …, x_N, as obtained from simulations or expert questionnaires. Moreover, assume that is categorical, so that the underlying data points (answers) can be ranked within a finite range (in the example in Fig 7, we have three categories, e.g., {“low”, “medium”, “high”}, which correspond to the ordered ranks {1, 2, 3}). The case when the loss distributions is continuous is in fact even simpler, and discussed later in remark 10. For now, let us stick with the expectedly more common practical case where risk assessments are made in categories rather than hard figures.

First, the empirical distribution (normalized histogram) is replaced by a kernel density estimator (KDE) using Gaussian kernels of a fixed bandwidth to define (5) where

• is the standard Gaussian function,
• and h > 0 is a bandwidth parameter that can be estimated using (any) standard statistical rule (of thumb, e.g., Silverman’s formula [59]).

Although any such nonparametric estimation can be quite inaccurate, yet as the data on which it is based is subjective anyway, the additional approximation error may not be as significant (nor in any sense quantifiable; still, Nadaraja’s theorem (see [60]) would assure that a continuous unknown underlying loss distribution would be approximated arbitrarily well in probability, as N → ∞, provided that h is chosen as h(N) = c ⋅ N^−α for any two constants c > 0 and 0 < α < 1/2.)

Using the KDE Eq (5), we can cast the distribution-valued game back into a regular matrix-game over the reals. Note that in choosing Gaussian kernels, we naturally extend all density functions to the entirety of . Such distributions would not be losses in the sense of definition 2, as the supports are no longer bounded.

Using the aforementioned risk acceptance threshold a > 1 to truncate all loss distributions at a, casts the loss distributions into the proper form. We can then expand the (truncated) loss density into a Taylor series for every scenario (i, j) ∈ AS₁ × AS₂. To ease notation in the following, let us drop the double index and simply write to mean the kernel density approximation of the empirical distribution in the given scenario, based on N data samples. Then, its Taylor series expansion at point a is (6) which converges everywhere on [1, a] by our choice of the Gaussian kernel. The k-th inner derivative is obtained from the kernel density definition Eq (5) as, (7)

Here, our use of Gaussian density pays a second time, since the k-th derivative of the exponential term can be expressed in closed form using Hermite polynomials by exploiting the relation (8) in which H_k(x) is the k-th Hermite polynomial, defined recursively as H_k+1(x) ≔ 2xH_k(x) − 2H_k−1(x) upon H₀(x) = 1 and H₁(x) = 2x. Plugging Eqs (8) into (7), and after rearranging terms, we find (9)

Evaluating the derivatives up to some order and substituting the values back into Eq (6), we could numerically construct the kernel density estimator. Fortunately, there is a shortcut here to avoid this, if we use the vector of derivatives with alternating signs to represent the Taylor-series expansion, and in turn the KDE, by (10) where the entries of the sequence can be computed from Eq (9).

Interestingly, under the assumptions made (i.e., truncation at a point and approximating the empirical distribution by a Gaussian KDE), the lexicographic order on the series representation Eq (9) equals the preference order ⪯ on the hyperreal representation of the loss distribution (see Lemma 18 in the appendix for a proof). That is, we can decide ⪯ between two sequence representations and of the form Eq (10) as follows:

• If y₀ < z₀, then L₁ ⪯ L₂. If y₀ > z₀, then L₂ ⪯ L₁. Otherwise, y₀ = z₀, and we check
• if y₁ < z₁, then L₁ ⪯ L₂. If y₁ > z₁, then L₂ ⪯ L₁. Otherwise, y₁ = z₁, and we check
• if y₂ < z₂, then L₁ ⪯ L₂, etc.

Remark 10 (Continuous loss models). If a continuous loss model is specified, differentiability may be not be an issue if the density f has derivatives of all orders. Otherwise, we can convolve f by a Gaussian density k_h with small variance h > 0 to get an approximation at any desired precision. Kernel density estimates are exactly such convolutions and thus provide convenient differentiability properties here.

Experimentally, we observed that in many cases the preference decision ⪯ can be made already using the first value f(a) in the sequence Eq (10). If the decision cannot be made (upon a tie f_L1(a) = f_L2(a)), then we can move on to the first order derivative, and so on. Thus, we technically do fictitious play in parallel on a “stack” of matrix games , where the k-th matrix is constructed (only on demand) with the k-th entry of the sequence representation Eq (10). The selection of strategies is herein always made on the first game matrix A₀, looking at the others only in cases where the decision cannot be made directly within A₀ (see Fig 8 for an illustration). Since we are now back at a regular matrix game, the usual convergence properties of fictitious play are restored.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Applying Fictitious Play.

https://doi.org/10.1371/journal.pone.0168675.g008

Of course, we cannot run fictitious play on an infinite stack of matrix games, so we are necessarily forced to restrict attention to a finite “sub-stack”. However, depending on how “deep” the stack is made, we can reach an equilibrium at arbitrary precision. This is made rigorous in the following definition:

Definition 11 (Approximate Equilibrium). Let ε > 0, δ > 0 be given, and let be a zero-sum matrix game with distribution-valued payoffs. We call a strategy profile an (ε, δ)-approximate equilibrium, if there is an equilibrium (in the zero-sum game A) such that both of the following conditions hold:

1. , and
2. ,

where the equilibrium payoffs and are defined by Eq (3) upon the approximate and the (regular) equilibrium, respectively.

Remark 12. By the equivalence of norms on , the ∞-norm in condition 1 can be replaced by any other norm upon, as long as the value of ε is chanced accordingly.

Remark 13. The continuous dependence of F(p, q) on (p, q) (in terms of the topologies on and L¹; the latter space being the one where the payoff distributions live in) as implied by Eq (3) leads to thinking that letting ε → 0 would also cause δ → 0. This impression is not necessarily true, since the ε-condition in definition 11 can be taken as a convergence criterion when iteratively computing equilibria, while the payoff distributions can be approximated at a precision that is independent of this value ε. Indeed, the goodness of approximation is controlled by the amount of data and the parameter h chosen for the KDE or the mollifier (cf. remark 10), and as such is independent of ε. Hence, asking for a δ-deviation for the payoff distribution accounts for a nontrivial degree of freedom here.

While existence of equilibria in mixed strategies for all finite games is assured (see [58]), the existence of an approximate equilibrium is not as obvious. In fact, the actual use of an approximate equilibrium is to find it within the set of regular matrix games, so that games with payoffs from can be solved for equilibria just like a standard matrix game would be treated. Clearly, since and by virtue of the embedding ϕ, the set of matrix games forms a subclass of games of the form , which itself covers games of the form by virtue of ϕ. A practical method to solve games over is obtained by approximating the solution from the inner set of equilibria in matrix games with real-valued payoffs. This is the main theorem of this work, whose proof is delegated to the appendix.

Theorem 14 (Approximation Theorem). For every ε > 0, δ > 0 and every zero-sum matrix game with distribution-valued payoffs, there is another zero-sum matrix game so that an equilibrium in Γ₂ is an (ε, δ)-approximate equilibrium in Γ₁.

7.2 Zero-Day Exploits

As a matter of consequence from the KDE approximation of empirical densities using Gaussian kernels, the approximate density in any case is supported on the entire real line. That is, the density function (5) assigns positive likelihood to the entire range (a, ∞), where a is again our risk acceptance threshold. For the specific scenario (i, j), this also means that positive likelihood is assigned to losses in the range Z = (max_k {x_k} , ∞), where x₁, …, x_N are the observations upon which our empirical loss distribution is based, and Z is the range of losses that were never observed. These are, by definition, exactly the events of zero-day exploits. More importantly, losses in Z have—by construction—a positive likelihood to occur in scenario (i, j) under the approximate RV with density function (5).

In other words, no matter of whether or not we explicitly sought to model zero-day exploits, they are automatically (implicitly) taken into account by the loss density approximation technique laid out in Section 7. The specific likelihood for a zero-day exploit, as based on the information available, is simply the mass assigned to Z under (as defined in Eq (5)). This value increases, the more observations on higher losses are available, say, if more experts expect higher damages to occur. In that case, the kernel density estimate will put more mass on this area, thus fattening the tails of the KDE approximation Eq (5). Fig 9 will later display the equilibrium outcome of an example APT-game model, showing the “zero-day area” in gray.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Equilibrium loss distribution for the example APT mitigation game.

https://doi.org/10.1371/journal.pone.0168675.g009

In connection with the design of our games to yield optimal behavior that minimizes the chances for high losses, the likelihood for zero-day exploits is then automatically minimized, since the ⪯-optimal decisions are those that have all their masses shifted towards lowest damages as much as possible. Therefore, practically, we can adjust our modeling account for zero-day exploits by adding more pessimistic observations to the data sets from which we construct the loss distributions. But from that point onwards, the construction automatically considers extreme events in the way as we want it without further explicit considerations.

8.1 Multiple Goals and Optimal Tradeoffs

Suppose for simplicity that only a choice between finitely many options is to be made, where risk is decreased upon increasing investments.

Example 15 (Uninterruptible power supplies). Imagine that a company ponders about installing additional power-supplies to cover for outages. Depending on how many such systems are available (in different subsidiaries of the company), the risk for an outage will obviously decrease. Equally clear is the increase of costs per additional system, so putting both of these outcome measures into a graph, we end up with finding the optimal tradeoff somewhere in the middle. The question is now how to find the optimum algorithmically.

Formally and generally, let be different measures of loss that all need to be accounted for in the scenario (i, j) ∈ AS₁ × AS₂, and let α₁, α₂, …, αd ∈ (0, 1) be weights assigned to these losses (identically for all i, j). Practically, such losses could concern (among others):

• confidentiality of information (loss distributions ),
• availability of systems (loss distributions ),
• security investments/costs to run defense actions (loss distributions ),
• etc.

Then, [62] has shown that a vector-valued loss referring to multiple interdependent criteria can be converted into a simple (single-criteria) loss by taking (11) into the optimization. Technically, the convex combination takes us to the Pareto-front of admissible actions, and the resulting optima and Nash-equilibria are understood in terms of Pareto-optimality. In Eq (11), the addition is understood pointwise on the distribution functions, which is mathematically justified since the expectation operator is linear w.r.t. the distributions over which it is computed (hence, the moment sequences arise in the proper form). Consequently, multicriteria game theory as studied in [61–63] applies without change here.

The only technical constraint that applies here is that all for all i, j, k must have the same support . That is, we must measure the loss in a common scale, for otherwise, the above scalarization does not make sense.

Combining Losses of Different Nature.

Different goals may be measured in individual scales (such as monetary loss being expressed as a number, loss of public reputation expressed in a nominal scale like “low/medium/high confidence” or loss of customers being an integer count). To harmonize these towards making the convex combination Eq (11) meaningful, we need to cast all these scales into a common scale Ω over the reals. While there is no general method to do this, practical heuristics to achieve this mainly do two steps:

1. Define an ordered set of fixed loss categories that shall apply for all goals of interest, and define the understanding of each category individually per goal.
2. Map all concrete (e.g., numeric) losses into the so-defined categories.

This approach is, for example, followed in many national risk management standards, such as [66–68]

Picking up example 15, let us assume that we seek to decide between installing between zero and five auxiliary power supply systems. Letting the priorities to be “security:costs = 60:40” (i.e., α₁ = 0.6, α₂ = 0.4), we find the optimal tradeoff to be between two and three additional power supplies. Finding the actual optimum is then a simple matter of comparing calculations for 2 and 3 power supplies since all other options have been eliminated; see Fig 10 (the computation on loss distributions would be identical, only taking a pointwise weighted sum of the distributions; only the resulting plot would be much less illustrative than the shown picture of real-valued functions).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Optimal Tradeoffs (simple case).

https://doi.org/10.1371/journal.pone.0168675.g010

8.2 Optimizing a (Permanent) Security Design

Since the concept of Nash equilibrium on which optimal choices are based assumes randomized (and thus repeated) actions, the question on how to apply the optimization to “permanent” countermeasures (that are not repeated actions in the classical sense) naturally appears. Indeed, this case boils down to a special case of a game in which the action space of player 2 is singleton, |AS₂| = 1, and we only look at the performance of different actions, all of which may be static. Let us give two practical examples:

Example 16 (installing anti malware systems). Installations of anti-virus software is a standard precaution, however, given inconsistencies between reported performances and the diversity of threats, it is often advisable to install several anti malware precautions. No system ships with a guaranteed detection rate, and different systems may be differently fast in identifying and blocking threats from the outside. The decision must again be based on empirical data (reported recognition rates) and just installing the full palette of available software will not necessarily increase the protection. Thus, there is an eventual tradeoff between security and cost, for which an optimum has to be found (see Section 8.1).

Example 17 (hiring security guards). As with example 15 and as illustrated in Fig 10, hiring more security guards will increase likelihoods to catch an intruder, but also will increase costs. This is a case where one performance indicator is random (the chances to keep intruders outside), while the other is quite deterministic (the guard’s salaries). As before, the sought decision is a mere ⪯-optimal selection among finitely many choices.

Generally, in both examples, the problem is to find an optimal choice among our options in AS₁, while the effects of a choice are not determined by an adversarial move in the game, but rather up to our own assessment (formally, we can simulate this by defining AS₂ to be singleton and contain an abstract (not further specified) action). The technical issue illustrated in both examples is the problem of ⪯-comparing deterministic to random outcomes. This is indeed easily possible and has been formally proven in [49] to work as follows: let a be a real value and let B be a real-valued RV, whose distribution is f_B and is supported on [1, b].

• If a ≤ b, then we ⪯-prefer a (intuitively, this is because having a constant outcome a is more secure to rely on than on a random effect B).
• If a > b, then we ⪯-prefer B (intuitively, since it in any case admits less damage than a, whose damage is guaranteed).
• If a = b, then we prefer B (intuitively, this is because B admits damages less than a, while the other action entails a guaranteed loss ≥ anything that can happen under B).

This procedure works on single criteria decisions only. If multiple criteria are to be taken into account, then we must again resort to a kernel density approximation to uniformly represent all values as RVs (thus, adding a controllable/reasonable degree of uncertainty) to the variable a, to be able to use the lexicographic comparisons on the convex combination of utility functions as sketched in Section 8.1.

10.1 Actions in Continuous Time

Typically, cyber warefare and APTs are not games that take rounds, but a process of actions in continuous time with no defined “stages” in the game. More precisely, we have the situation of one player likely being forced to take actions at discrete times, while the other player is free act continuously over time. For example, the security officer may be unable to become active at any (prescribed optimal) time of the day, since s/he must adhere to organizational constraints of the daily business. On the contrary, the adversary is only bound to the business organizational matters as far as it concerns the mounting of an attack. In any case, the attacker can act at any point in time (during night, during peak hours of work, etc.), while the defender must use the proper time slots to become active to minimize distortions in the actual enterprise business.

This is a remarkable qualitative difference to both, discrete and continuous time game models, covering matrix games (discrete in time) and others (e.g., [16] as a continuous time model). Our specific APT model is discrete in time (as being a matrix game), but can account for randomness in the outcome caused by a continuously acting opponent. This is a mere change to the outcome (loss) distributions to cater for an action to be interrupted, distorted or to cause more damage the longer it has happened in the past.

Theoretically, the embedding of the payoff distributions into equips us with the full spectrum of mathematical tools as are known and applied to construct continuous time games (this is a direction of future research from this work). Practically, we believe the application of discrete time games to more properly match the possibilities of a real-life security officer, who may take actions only at particular times, i.e., outside peak-hours of work, when devices are temporarily idle, or similar. Invoking Lemma 9 here tells us that if the defense is optimized for discrete time actions of the defender, a “continuous time attacker” may nevertheless only deviate and as such cause less damage than what the matrix game predicts.

10.2 On Some Modeling Issues

This section is similar to many well-known collections of “frequently asked questions”, and indeed can be taken as a guideline on how to overcome a set of usual modeling obstacles when the model shall be applied practically:

How to deal with large attack graphs?

As a matter of fact, attack graphs can get huge even for small infrastructures. Enumerating all strategies by finding all paths can thus become an infeasible task (as there is usually an exponential number of them). Instead, we may either simulate attacks on the nodes directly, thus defining the attack strategies not as paths in the attack graph but rather as the set of possible exploits in our infrastructure, and ask for a qualitative risk assessment based on the exploits only. Even if the infrastructure is large, having the freedom to work with qualitative assessments in our game-theoretic model eases matters of risk assessment essentially up to the same complexity as a normal risk and vulnerability assessment would require. That is, there is no conceptual obligation in our APT game model definition to work with paths in the attack graph (this is only one option among many), and the APT game can be defined on aggregated parts of an infrastructure, or using any other condensed or high-level view on the system.

Where to get the losses/probabilities from?

Probability—in general—is a notoriously opaque concept for many practitioners, and specifying probability in our model, as for any probabilistic model, is a crucial initial task. However, unlike many other techniques, our requirements are different in an important way: we do not ask an expert for a numeric estimate of a probability, but instead it suffices to ask several experts for a qualitative rating of likelihood concerning certain attacks. That is, the expert is not challenged to tell a precise number to quantify how likely an attack is, but can rather resort to saying that the success for an attack or mitigation strategy may be “low”, “medium” or “high”. This is even in alignment with recommendations of the German Federal Office for Information Security (BSI), which explicitly warns about “precise probabilities” which can misleadingly take estimates as being objectively accurate. Asking several experts about their opinions and re-normalizing the absolute frequencies into probabilities avoids the common problems of calibrating probabilistic models and naturally delivers the necessary loss distributions as we need them (see Fig 7).

Moreover, the criticism uttered against probabilities (the difficulty to specify them and the illusion of accuracy created by them) applies much more generally to many statistical models, but not as such in the model proposed here.

An explicit alternative to surveys or mere expert opinions is the use of simulation. Models like [28] explicitly define a continuous time simulation for a moving target defense that can be adapted. Similar models from disease spreading analysis based on percolation theory can also be used to probabilistically assess the outcome of a malware infection (say via a bring-your-own-device scenario) [53] may also directly deliver the sought loss distributions. The appeal of any such simulation is the automatism that they provide to reduce the modeling labour.

What if a defense fails?

Normal game theory assumes defenses to be effective in general, for otherwise, an ineffective defense could be deleted from the game on grounds of strategic dominance (cf. [70]). In admitting defense strategies to have random effects, the failure of a defense is yet merely another form of failure of an action. Thus, upon proper modeling of the chances for an action to fail, such events are naturally covered by our random loss RVs admitting multiple different outcomes, including a complete failure. For example, if patches or updates are not available with known frequencies (say, if the vendor has a “patch day”), we can assign this likelihood as the probability of high damage despite the action. Note, however, that this is not the freedom of choosing not to do the defense action at the prescribed time. Doing so would mean to deviate from the equilibrium, which results in a worse protection. Adhering to the equilibrium behavior strategy yet failing in the defense action itself is, however, covered by the allowed likelihoods for an action to fail at random.

How to handle one-shot situations?

The existence of optimal decisions is often based on randomized actions, which means that actions and the entire game can be repeated. Practically, we may not be able to reset the infrastructure to a defined initial condition after a damage happened. That is, the game actually changes (at least temporarily) upon past actions. Such situations are covered by the notion of dynamic (stochastic) games, which allow future instances of the game play to depend on past rounds. So, an action may be one-shot and upon failure, may create a completely new situation. Assuming that the entirety of possibilities admits a finite number of game-theoretic descriptions, we technically have a stochastic game in Shapley’s sense [56]. The usual notion of equilibrium (optimal defense) in such competitions is, however, more intricate and its existence is often tied to additional assumptions or modifications to the game (e.g., by resorting to dynamic games, or similar). The practical issue here is the concrete choice of equilibrium outcomes (discounted, averaged, etc.) to retain a practical meaning in the APT context. We avoid such difficulties here by allowing the outcome to be different in each round and determined by past iterations of the game, as long as the outcome remains identically distributed between rounds. If we think of the game structure itself following a stochastic process, then the loss distributions constituting the game structure may be taken as the stationary distribution of the process, under any known condition of convergence (e.g., see [71]). We will leave this as a route for future work, and close this discussion with the statement that one-shot situations are conceptually equivalent to dealing with repeatable situations, as we do not optimize the cumulative long-run average (which would assume a repeated gameplay), but rather optimally shape the distribution of the outcome for every repetition and thus also for “one-shots”.

Is Knowledge about the Adversary’s Incentives or Intentions Required?

Adversary modeling is often perceived necessary or at least useful in defending assets, especially in APT scenarios. However, defenses tailored to a specific guess about the adversary’s intentions or incentives may perform only suboptimal depending on the accuracy of the guess. Although a game-theoretic model can be designed to take into account adversarial payoffs if they are known, we do not actually require an accurate understanding of the adversary’s intentions or incentives.

It is important, however, to understand who the adversary is, because this is what determines its action set AS₂. The more we know about the attacker, the more accurate we can model its actions, and thus reduce the possibilities for unexpected incidents. Therefore, we must stress that Lemma 9 only spares us to understand the adversary’s intentions, but we can in no case ignore its capabilities. Thus, we do require an adversary model here, yet being accurate only on the adversaries possible actions and on our own loss upon these.

How to set the risk acceptance threshold a?

The risk acceptance threshold a > 1 is here required primarily for technical reasons, i.e., to assure the boundedness of supports to ease deciding preferences among actions. Thus, as long as any such value is being defined, the theory and results remain intact. Physically, this parameter corresponds to the maximal damage that we expect to occur ever, or otherwise said, cases of damage that are covered by insurances or considered so unlikely that the risk is simply taken.

Quite obviously, the concrete choice of the threshold a has a substantial influence on the decisions and computed equilibria. Indeed, cutting off tails at different locations can even reverse the preference under ⪯. Therefore, this value should be chosen based on trial comparisons between actions to look for a paradoxical/counter-intuitive outcome of ⪯ (see [58] for an illustration), so that the value a can be set accordingly. Technically, it determines the region of loss that we would relate to zero-day exploits (as was discussed in the context of Fig 9). In any case, the particular choice of a is up to expertise, experience, and is seemingly out of the scope of any default procedure to choose it.

Several rule of thumbs may be defined, such as choosing a as a maximum quantile over all loss distributions (similar to a value at risk (VaR) approach outlined in [72]), or directly taking a as the worst risk assessment made or possible. Our implementation in R takes the maximum observation or most pessimistic loss assessment as the cutoff point, assuming that no more than the worst expected outcome may be expected. However, if losses are quantified in monetary terms or general business value, the lot of insurance coverage may determine the acceptable risks that can be taken.