NS-Gym Update Functions Module¶

class ns_gym.update_functions.BudgetBoundedIncrement(scheduler, k, B)[source]¶

Bases: UpdateDistributionFn

Increment the parameters so that the total amount of change is bounded by some budget.

Overview:: This function contrains the total amount of change in the parameter by some max budget. This formulation is outlined in Cheung et al. 2020.

Parameters:

scheduler (base.Scheduler) – scheduler that determines when the update function fires.
k (float) – The amount which the parameter is updated.
B (Union[int,float]) – The maximum total amount of change allowed in the parameter.

class ns_gym.update_functions.DistributionDecrementUpdate(scheduler, k)[source]¶

Bases: UpdateDistributionFn

Decrement the probability of going in the intended direction by some k.

Overview:: This function is used to decrement the probability distribution by some k. The probability distribution is represented as a list of probabilities. The intended direction is the first element in the probability distribution.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
k (float) – The amount which the parameter is updated.

class ns_gym.update_functions.DistributionIncrementUpdate(scheduler, k)[source]¶

Bases: UpdateDistributionFn

Increment the the parameter by k.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
k (float) – The amount which the parameter is updated.

Note

This update function is useful for testing the robustness of the agent to changes in the environment. If the parameter is a probability, k would update the probability of going in the intended direction. Otherwise, k would be added to the parameter’s value.

class ns_gym.update_functions.DistributionNoUpdate(scheduler)[source]¶

Bases: UpdateDistributionFn

Does not update the parameter but return correct ns_bench interface.

Overview:: This function does not update the parameter.

class ns_gym.update_functions.DistributionStepWiseUpdate(scheduler, update_values)[source]¶

Bases: UpdateDistributionFn

Update the parameter to values to a set of predefined values at specific time steps.

Parameters:

scheduler (base.Scheduler) – scheduler that determines when the update function fires.
update_values (list) – A list of values that the parameter is updated to at specific time steps.

class ns_gym.update_functions.LCBoundedDistrubutionUpdate(scheduler, L, update_fn=None)[source]¶

Bases: UpdateDistributionFn

Decrement the parameters so that the change is Lipshitz continuous.

Overview:

This function would call the decrement update function and check if the change is Lipshitz continuous. If not it would recall the decrement update function until the change is Lipshitz continuous.

The Lipshitz continuous constraint between to probability distributions is defined as:

\[W_1(p_t(.|s,a),p_{t'}(.|s,a)) <= L * |t - t'|\]

Where \(W_1\) is the Wasserstein distance between two probability distributions.

Parameters:

update_fn (Type[base.UpdateDistributionFn]) – The update function that updates the parameter.
L (float) – The Lipshitz constant.

Note

This update function is an implementation of transition fucntion in Lecarpentier and Rechelson et al. 2019

class ns_gym.update_functions.RandomCategorical(scheduler, seed=None)[source]¶

Bases: UpdateDistributionFn

Update the distirbution as a random categorical distribution.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
seed (Optional[int], optional) – Seed for the random number generator. Defaults to None.

Note

This update function would return a new random categorical distribution. The new categorical distribution is sampled from a Dirichlet distribution with all parameters equal to 1.

class ns_gym.update_functions.DeterministicTrend(scheduler, slope)[source]¶

Bases: UpdateFn

Update the parameter with a deterministic trend.

Overview:: \[Y_t = Y_{t-1} + slope * t\]

where \(Y_t\) is the parameter value at time step \(t\) and slope is the slope of the trend.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
slope (float) – The slope of the trend.

class ns_gym.update_functions.ExponentialDecay(scheduler, decay_rate)[source]¶

Bases: UpdateFn

Exponential decay of the parameter.

Overview:

\[Y_t = Y_0 * exp(-\lambda * t)\]

where \(Y_t\) is the parameter value at time step \(t\), \(Y_0\) is the initial parameter value, and \(\lambda\) is the rate of decay.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
decay_rate (float) – The rate of decay. i.e. \(\lambda\)

class ns_gym.update_functions.GeometricProgression(scheduler, r)[source]¶

Bases: UpdateFn

Apply a geometric progression to the parameter.

Overview:

\[Y_t = Y_0 * r^t\]

where \(Y_t\) is the parameter value at time step \(t\), \(Y_0\) is the initial parameter value, and \(r\) is the common ratio.

class ns_gym.update_functions.IncrementUpdate(scheduler, k)[source]¶

Bases: UpdateFn

Increment the the parameter by k.

Overview:: \[Y_t = Y_{t-1} + k\]

where \(Y_t\) is the parameter value at time step \(t\) and \(k\) is the amount to increment the parameter by.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
k (float) – The amount which the parameter is updated.

class ns_gym.update_functions.NoUpdate(scheduler)[source]¶

Bases: UpdateFn

Do not update the parameter but return correct interface

Overview:: This function does not update the parameter when called. It is useful for testing and debugging.

Parameters:: scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.

class ns_gym.update_functions.OscillatingUpdate(scheduler, delta)[source]¶

Bases: UpdateFn

Update the parameter with an oscillating function.

Overview:

\[Y_t = Y_{t-1} + \delta * sin(t)\]

where \(Y_t\) is the parameter value at time step \(t\) and \(\delta\) is the amplitude of the sine wave.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
delta (float) – The amplitude of the sine wave.

class ns_gym.update_functions.RandomWalk(scheduler, mu=0, sigma=1, seed=None)[source]¶

Bases: UpdateFn

Parameter update function that updates the parameter with white noise.

Overview:: A pure random walk : \(Y_t = Y_{t-1} + \epsilon_t\) where \(Y_t\) is the parameter value at time step \(t\) and \(\epsilon\) is white noise.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
mu (Union[float,int], optional) – The mean of the white noise. Defaults to 0.
sigma (Union[float,int], optional) – The standard deviation of the white noise. Defaults to 1.
seed (Union[int,None], optional) – Seed for the random number generator. Defaults to None.

class ns_gym.update_functions.RandomWalkWithDrift(scheduler, alpha, mu, sigma, seed=None)[source]¶

Bases: UpdateFn

A parameter update function that updates the parameter with white noise and a drift term.

Overview:

\[Y_t = \alpha + Y_{t-1} + \epsilon_t\]

where \(Y_t\) is the parameter value at time step \(t\), \(\alpha\) is the drift term, and \(\epsilon\) is white noise.

Parameters:

alpha (float) – The drift term.
mu (float) – The mean of the white noise.
sigma (float) – The standard deviation of the white noise.
seed (int) – Seed for the random number generator. Defaults to None.

class ns_gym.update_functions.RandomWalkWithDriftAndTrend(scheduler, alpha, mu, sigma, slope, seed=None)[source]¶

Bases: UpdateFn

Parameter update function that updates the parameter with white noise and a deterministic trend.

Overview:

\[Y_t = \alpha + Y_{t-1} + \text{slope} * t + \epsilon_t\]

where \(Y_t\) is the parameter value at time step \(t\), \(\alpha\) is the drift term, \(\text{slope}\) is the slope of the trend, and \(\epsilon\) is white noise.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
alpha (float) – The drift term.
mu (float) – The mean of the white noise.
sigma (float) – The standard deviation of the white noise.
slope (float) – The slope of the trend.
seed (Union[int, None], optional) – Seed for the random number generator. Defaults to None.

class ns_gym.update_functions.StepWiseUpdate(scheduler, param_list)[source]¶

Bases: UpdateFn

Update the parameter at specific time steps.

Overview:: This function updates the parameter to the next value in the param_list when called. If the param_list is empty, the parameter is not updated.

Parameters:

scheduler (Type[base.Scheduler]) – scheduler that determines when the update function fires.
param_list (list) – A list of parameters to update to.