picos.expressions.exp_entropy¶

Implements Entropy and NegativeEntropy.

Classes

class picos.expressions.exp_entropy.Entropy(x, y=None)[source]¶

Bases: Expression

Entropy or negative relative entropy of an affine expression.

Negative relative entropy is also known as the perspective of the logarithm.

Definition

Let $x$ be an $n$ -dimensional real affine expression.

If no additional expression $y$ is given, this is the entropy

$-\sum_{i = 1}^n \operatorname{vec}(x)_i \log(\operatorname{vec}(x)_i).$
If an additional affine expression $y$ of same shape as $x$ is given, this is the negative relative entropy (or logarithmic perspective)

$-\sum_{i = 1}^n \operatorname{vec}(x)_i \log\left( \frac{\operatorname{vec}(x)_i}{\operatorname{vec}(y)_i} \right) &= -\sum_{i = 1}^n \operatorname{vec}(x)_i \left[ \log(\operatorname{vec}(x)_i) - \log(\operatorname{vec}(y)_i) \right] \\ &= \sum_{i = 1}^n \operatorname{vec}(x)_i \left[ \log(\operatorname{vec}(y)_i) - \log(\operatorname{vec}(x)_i \right] \\ &= \sum_{i = 1}^n \operatorname{vec}(x)_i \log\left( \frac{\operatorname{vec}(y)_i}{\operatorname{vec}(x)_i} \right).$

Warning

When you pose a lower bound on this expression, then PICOS enforces $x \geq 0$ through an auxiliary constraint during solution search. When an additional expression $y$ is given, PICOS enforces $y \geq 0$ as well.

__add__(other)[source]¶: Denote addition with another expression on the right-hand side.

__ge__(other)[source]¶: Return a constraint that the expression is lower-bounded.

__init__(x, y=None)[source]¶

Construct an Entropy.

Parameters

x (AffineExpression) – The affine expression $x$ .
y (AffineExpression) – An additional affine expression $y$ . If necessary, PICOS will attempt to reshape or broadcast it to the shape of $x$ .

__neg__()[source]¶: Denote the negation of the expression.

__radd__(other)[source]¶: Denote addition with another expression on the left-hand side.

__sub__(other)[source]¶: Denote subtraction of another expression from the expression.

property n¶: Length of x.

property x¶: The expression $x$ .

property y¶: The additional expression $y$ , or None.

class picos.expressions.exp_entropy.NegativeEntropy(x, y=None)[source]¶

Bases: Expression

Negative or relative entropy of an affine expression.

Relative entropy is also known as the Kullback-Leibler divergence.

Definition

Let $x$ be an $n$ -dimensional real affine expression.

If no additional expression $y$ is given, this is the negative entropy

$\sum_{i = 1}^n \operatorname{vec}(x)_i \log(\operatorname{vec}(x)_i).$
If an additional affine expression $y$ of same shape as $x$ is given, this is the relative entropy (or Kullback-Leibler divergence)

$\sum_{i = 1}^n \operatorname{vec}(x)_i \log\left( \frac{\operatorname{vec}(x)_i}{\operatorname{vec}(y)_i} \right) = \sum_{i = 1}^n \operatorname{vec}(x)_i \left[ \log(\operatorname{vec}(x)_i) - \log(\operatorname{vec}(y)_i) \right].$

Warning

When you pose an upper bound on this expression, then PICOS enforces $x \geq 0$ through an auxiliary constraint during solution search. When an additional expression $y$ is given, PICOS enforces $y \geq 0$ as well.

__add__(other)[source]¶: Denote addition with another expression on the right-hand side.

__init__(x, y=None)[source]¶

Construct a NegativeEntropy.

Parameters

x (AffineExpression) – The affine expression $x$ .
y (AffineExpression) – An additional affine expression $y$ . If necessary, PICOS will attempt to reshape or broadcast it to the shape of $x$ .

__le__(other)[source]¶: Return a constraint that the expression is upper-bounded.

__neg__()[source]¶: Denote the negation of the expression.

__radd__(other)[source]¶: Denote addition with another expression on the left-hand side.

__sub__(other)[source]¶: Denote subtraction of another expression from the expression.

property n¶: Length of x.

property x¶: The expression $x$ .

property y¶: The additional expression $y$ , or None.

Fork me on GitLab