Optimization

KKT Condition

KKT condition is the necessary & sufficient conditions for inequality constrained problems. Reference

1. Unconstraint Problem

Suppose the objective $f(x)$ is a convex function, then the necessary and sufficient condition is $$ \nabla f(x^*) =0 $$ In a more general setting,

2. Equality Constrained Problem

$$ \begin{aligned} &\min_{x \in R^{n}} f(x) \\ &\text{s.t. } c_{i}(x)=0, i=1, \ldots, m, \quad m \leq n \end{aligned} $$

3. Inequality Constrainted Problem

$$ \begin{aligned} &\min_{x \in R^{n}} f(x) \\ &\text{s.t. } c_{i}(x) \geq 0, i=1, \ldots, m, \quad m \leq n \end{aligned} $$

$$ \min_{x \in R^{n}} f(x) \\ \text{s.t. } c_{i}(x) \geq 0, i=1, \ldots, m, \quad m \leq n $$