Deterrence theory
Rational deterrence theory: One approach to theorizing about deterrence has entailed the use of rational choice and game-theoretic models of decision making (see game theory).  Rational deterrence theory entails:

Rationality: actors are rational
Unitary actor assumption: actors are understood as unitary
Dyads: interactions tend to be between dyads (or triads) of states
Strategic interactions: actors consider the choices of other actors
Cost-benefit calculations: outcomes reflect actors' cost-benefit calculations
Deterrence theorists have consistently argued that deterrence success is more likely if a defending state's deterrent threat is credible to an attacking state. Huth outlines that a threat is considered credible if the defending state possesses both the military capabilities to inflict substantial costs on an attacking state in an armed conflict, and the attacking state believes that the defending state is resolved to use its available military forces. Huth goes on to explain the four key factors for consideration under rational deterrence theory: the military balance, signaling and bargaining power, reputations for resolve, interests at stake.
The American economist Thomas Schelling brought his background in game theory to the subject of studying international deterrence. Schelling's (1966) classic work on deterrence presents the concept that military strategy can no longer be defined as the science of military victory. Instead, it is argued that military strategy was now equally, if not more, the art of coercion, intimidation and deterrence. Schelling says the capacity to harm another state is now used as a motivating factor for other states to avoid it and influence another state's behavior. To be coercive or deter another state, violence must be anticipated and avoidable by accommodation. It can therefore be summarized that the use of the power to hurt as bargaining power is the foundation of deterrence theory and is most successful when it is held in reserve.
In an article celebrating Schelling's Nobel Memorial Prize for Economics, Michael Kinsley, Washington Post op‑ed columnist and one of Schelling's former students, anecdotally summarizes Schelling's reorientation of game theory thus: "[Y]ou're standing at the edge of a cliff, chained by the ankle to someone else. You'll be released, and one of you will get a large prize, as soon as the other gives in. How do you persuade the other guy to give in, when the only method at your disposal—threatening to push him off the cliff—would doom you both? Answer: You start dancing, closer and closer to the edge. That way, you don't have to convince him that you would do something totally irrational: plunge him and yourself off the cliff. You just have to convince him that you are prepared to take a higher risk than he is of accidentally falling off the cliff. If you can do that, you win."

Military balance: Deterrence is often directed against state leaders who have specific territorial goals that they seek to attain either by seizing disputed territory in a limited military attack or by occupying disputed territory after the decisive defeat of the adversary's armed forces. In either case, the strategic orientation of potential attacking states generally is for the short term and is driven by concerns about military cost and effectiveness. For successful deterrence, defending states need the military capacity to respond quickly and strongly to a range of contingencies. Deterrence often fails if either a defending state or an attacking state underestimates or overestimates the other's ability to undertake a particular course of action.

Signaling and bargaining power: The central problem for a state that seeks to communicate a credible deterrent threat by diplomatic or military actions is that all defending states have an incentive to act as if they are determined to resist an attack in the hope that the attacking state will back away from military conflict with a seemingly resolved adversary. If all defending states have such incentives, potential attacking states may discount statements made by defending states along with any movement of military forces as merely bluffs. In that regard, rational deterrence theorists have argued that costly signals are required to communicate the credibility of a defending state's resolve. Those are actions and statements that clearly increase the risk of a military conflict and also increase the costs of backing down from a deterrent threat. States that bluff are unwilling to cross a certain threshold of threat and military action for fear of committing themselves to an armed conflict.

Reputations for resolve: There are three different arguments that have been developed in relation to the role of reputations in influencing deterrence outcomes. The first argument focuses on a defending state's past behavior in international disputes and crises, which creates strong beliefs in a potential attacking state about the defending state's expected behaviour in future conflicts. The credibilities of a defending state's policies are arguably linked over time, and reputations for resolve have a powerful causal impact on an attacking state's decision whether to challenge either general or immediate deterrence. The second approach argues that reputations have a limited impact on deterrence outcomes because the credibility of deterrence is heavily determined by the specific configuration of military capabilities, interests at stake, and political constraints faced by a defending state in a given situation of attempted deterrence. The argument of that school of thought is that potential attacking states are not likely to draw strong inferences about a defending states resolve from prior conflicts because potential attacking states do not believe that a defending state's past behaviour is a reliable predictor of future behavior. The third approach is a middle ground between the first two approaches and argues that potential attacking states are likely to draw reputational inferences about resolve from the past behaviour of defending states only under certain conditions. The insight is the expectation that decisionmakers use only certain types of information when drawing inferences about reputations, and an attacking state updates and revises its beliefs when a defending state's unanticipated behavior cannot be explained by case-specific variables.
An example shows that the problem extends to the perception of the third parties as well as main adversaries and underlies the way in which attempts at deterrence can fail and even backfire if the assumptions about the others' perceptions are incorrect.

Interests at stake: Although costly signaling and bargaining power are more well established arguments in rational deterrence theory, the interests of defending states are not as well known. Attacking states may look beyond the short-term bargaining tactics of a defending state and seek to determine what interests are at stake for the defending state that would justify the risks of a military conflict. The argument is that defending states that have greater interests at stake in a dispute are more resolved to use force and more willing to endure military losses to secure those interests. Even less well-established arguments are the specific interests that are more salient to state leaders such as military interests and economic interests.
Furthermore, Huth argues that both supporters and critics of rational deterrence theory agree that an unfavorable assessment of the domestic and international status quo by state leaders can undermine or severely test the success of deterrence. In a rational choice approach, if the expected utility of not using force is reduced by a declining status quo position, deterrence failure is more likely since the alternative option of using force becomes relatively more attractive.

Tripwires: International relations scholars Dan Reiter and Paul Poast have argued that so-called "tripwires" do not deter aggression. Tripwires entail that small forces are deployed abroad with the assumption that an attack on them will trigger a greater deployment of forces. Dan Altman has argued that tripwires do work to deter aggression, citing the Western deployment of forces to Berlin in 1948–1949 to deter Soviet aggression as a successful example.
A 2022 study by Brian Blankenship and Erik Lin-Greenberg found that high-resolve, low-capability signals (such as tripwires) were not viewed as more reassuring to allies than low-resolve, high-capability alternatives (such as forces stationed offshore). Their study cast doubt on the reassuring value of tripwires.

Nuclear deterrence theory: In 1966, Schelling is prescriptive in outlining the impact of the development of nuclear weapons in the analysis of military power and deterrence. In his analysis, before the widespread use of assured second strike capability, or immediate reprisal, in the form of SSBN submarines, Schelling argues that nuclear weapons give nations the potential to destroy their enemies but also the rest of humanity without drawing immediate reprisal because of the lack of a conceivable defense system and the speed with which nuclear weapons can be deployed. A nation's credible threat of such severe damage empowers their deterrence policies and fuels political coercion and military deadlock, which can produce proxy warfare.
According to Kenneth Waltz, there are three requirements for successful nuclear deterrence:

Part of a state's nuclear arsenal must appear to be able to survive an attack by the adversary and be used for a retaliatory second strike
The state must not respond to false alarms of a strike by the adversary
The state must maintain command and control
The stability–instability paradox is a key concept in rational deterrence theory. It states that when two countries each have nuclear weapons, the probability of a direct war between them greatly decreases, but the probability of minor or indirect conflicts between them increases. This occurs because rational actors want to avoid nuclear wars, and thus they neither start major conflicts nor allow minor conflicts to escalate into major conflicts—thus making it safe to engage in minor conflicts. For instance, during the Cold War the United States and the Soviet Union never engaged each other in warfare, but fought proxy wars in Korea, Vietnam, Angola, the Middle East, Nicaragua and Afghanistan and spent substantial amounts of money and manpower on gaining relative influence over the third world.
Bernard Brodie wrote in 1959 that a credible nuclear deterrent must be always ready but never used.