picos.expressions.exp_entropy

Implements Entropy and NegativeEntropy.

Outline

Classes

Entropy

Entropy or negative relative entropy of an affine expression.

NegativeEntropy

Negative or relative entropy of an affine expression.

Classes

Entropy

class picos.expressions.exp_entropy.Entropy(x, y=None)[source]

Bases: picos.expressions.expression.Expression

Entropy or negative relative entropy of an affine expression.

Negative relative entropy is also known as the perspective of the logarithm.

Definition

Let x be an n-dimensional real affine expression.

  1. If no additional expression y is given, this is the entropy

    -\sum_{i = 1}^n \operatorname{vec}(x)_i
\log(\operatorname{vec}(x)_i).

  2. If an additional affine expression y of same shape as x is given, this is the negative relative entropy (or logarithmic perspective)

    -\sum_{i = 1}^n \operatorname{vec}(x)_i
\log\left(
    \frac{\operatorname{vec}(x)_i}{\operatorname{vec}(y)_i}
\right)
&= -\sum_{i = 1}^n \operatorname{vec}(x)_i
\left[
    \log(\operatorname{vec}(x)_i) - \log(\operatorname{vec}(y)_i)
\right] \\
&= \sum_{i = 1}^n \operatorname{vec}(x)_i
\left[
    \log(\operatorname{vec}(y)_i) - \log(\operatorname{vec}(x)_i
\right] \\
&= \sum_{i = 1}^n \operatorname{vec}(x)_i
\log\left(
    \frac{\operatorname{vec}(y)_i}{\operatorname{vec}(x)_i}
\right).

Warning

When you pose a lower bound on this expression, then PICOS enforces x \geq 0 through an auxiliary constraint during solution search. When an additional expression y is given, PICOS enforces y \geq 0 as well.

__init__(x, y=None)[source]

Construct an Entropy.

Parameters
  • x (AffineExpression) – The affine expression x.

  • y (AffineExpression) – An additional affine expression y. If necessary, PICOS will attempt to reshape or broadcast it to the shape of x.

property n

Length of x.

property x

The expression x.

property y

The additional expression y, or None.

NegativeEntropy

class picos.expressions.exp_entropy.NegativeEntropy(x, y=None)[source]

Bases: picos.expressions.expression.Expression

Negative or relative entropy of an affine expression.

Relative entropy is also known as the Kullback-Leibler divergence.

Definition

Let x be an n-dimensional real affine expression.

  1. If no additional expression y is given, this is the negative entropy

    \sum_{i = 1}^n \operatorname{vec}(x)_i
\log(\operatorname{vec}(x)_i).

  2. If an additional affine expression y of same shape as x is given, this is the relative entropy (or Kullback-Leibler divergence)

    \sum_{i = 1}^n \operatorname{vec}(x)_i
\log\left(
    \frac{\operatorname{vec}(x)_i}{\operatorname{vec}(y)_i}
\right)
= \sum_{i = 1}^n \operatorname{vec}(x)_i
\left[
    \log(\operatorname{vec}(x)_i) - \log(\operatorname{vec}(y)_i)
\right].

Warning

When you pose an upper bound on this expression, then PICOS enforces x \geq 0 through an auxiliary constraint during solution search. When an additional expression y is given, PICOS enforces y \geq 0 as well.

__init__(x, y=None)[source]

Construct a NegativeEntropy.

Parameters
  • x (AffineExpression) – The affine expression x.

  • y (AffineExpression) – An additional affine expression y. If necessary, PICOS will attempt to reshape or broadcast it to the shape of x.

property n

Length of x.

property x

The expression x.

property y

The additional expression y, or None.