NS-Gym Context Switching Utilities

ns_gym.context_switching.make_env_with_context(env_name, context_value, context_parameter_name='masscart', seed=None)

Creates a ns_gym environment with a specified context parameter value.

Parameters:

• env_name (str) – Gymnasium environement name.

• context_value (float) – The value for the context parameter.

• context_parameter_name (str) – The name of the parameter to tune.

• seed (Optional[int]) – Seed for the environment.

Returns:

The configured environment.

Return type:

gym.Env

Caution

Not implemented for all environments. Currently supports:

• Classic Control: CartPole-v1, Acrobot-v1, MountainCarContinuous-v0, MountainCar-v0, Pendulum-v1

• Gridworlds: CliffWalking-v0, FrozenLake-v1, ns_gym/Bridge-v0

ns_gym.context_switching.run_context_episode(agent, ns_env_instance, num_episodes)

Runs an StableBaselines3 policy in a given ns_gym environment for a number of episodes.

Parameters:

• agent (StableBaselines3 Policy) – The trained agent/policy to evaluate.

• ns_env_instance (gym.Env) – The ns_gym environment instance.

• num_episodes (int) – Number of episodes to run.

ns_gym.context_switching.eval_target_contexts(policy, make_env_func_partial, num_episodes_per_context, target_context_range)

Evaluates a given policy across a range of target contexts. :type policy: :param policy: The trained StableBaselines3 agent/policy to evaluate. Should be compatible with StableBaselines3. :type make_env_func_partial: :param make_env_func_partial: A partial function of make_env_with_context (with context_parameter_name fixed). :type num_episodes_per_context: :param num_episodes_per_context: How many episodes to run for each target context. :type target_context_range: :param target_context_range: Array of target context values to evaluate on.

Returns:

Array of mean rewards for each target context, Array of std deviations.

ns_gym.context_switching.normalize_rewards_matrix(U_matrix_raw)

Normalizes a reward matrix using min-max scaling (0-1 range).

ns_gym.context_switching.calculate_sem(data_array)

Calculates the standard error of the mean for a 1D array.

ns_gym.context_switching.calculate_generalized_performance(agent_list, trained_agent_source_contexts, make_env_func_partial, num_episodes_per_eval_context, evaluation_target_context_range, normalize=True)

Calculates the generalized performance matrix U (raw and normalized), related metrics, and their standard errors.

ns_gym.context_switching.plot_performance_curves(U_matrix_to_plot, eval_context_range, train_source_contexts, overall_perf_to_display, context_param_name, is_normalized, output_filename=None)

Plots performance curves (either raw or normalized) and saves the plot.

ns_gym.context_switching.save_metrics_to_file(filename, U_matrix_raw, peak_performances_raw, overall_system_performance_raw, sem_overall_raw, U_matrix_normalized, peak_performances_normalized, overall_system_performance_normalized, sem_overall_norm, normalization_params, source_contexts, context_range, args_dict)

Saves both raw and normalized metrics, including SEM, to a text file.