Actions

Similarly to observations, several types of actions can be used in every environment. They are defined in the action module. Each environment comes with a default action type, which can be changed or customised using environment configurations. For instance,

import gymnasium as gym

env = gym.make('highway-v0', config={
    "action": {
        "type": "ContinuousAction"
    }
})

Continuous Actions

The ContinuousAction type allows the agent to directly set the low-level controls of the vehicle kinematics, namely the throttle \(a\) and steering angle \(\delta\).

import gymnasium as gym

env = gym.make('highway-v0', config={
    "action": {
        "type": "ContinuousAction"
    }
})

Note

The control of throttle and steering can be enabled or disabled through the longitudinal and lateral configurations, respectively. Thus, the action space can be either 1D or 2D.

Discrete Actions

The DiscreteAction is a uniform quantization of the ContinuousAction above.

import gymnasium as gym

env = gym.make('highway-v0', config={
    "action": {
        "type": "DiscreteAction"
    }
})

The actions_per_axis parameter allows to set the quantization step. Similarly to continuous actions, the longitudinal and lateral axis can be enabled or disabled separately.

Discrete Meta-Actions

The DiscreteMetaAction type adds a layer of speed and steering controllers on top of the continuous low-level control, so that the ego-vehicle can automatically follow the road at a desired velocity. Then, the available meta-actions consist in changing the target lane and speed that are used as setpoints for the low-level controllers.

import gymnasium as gym

env = gym.make('highway-v0', config={
    "action": {
        "type": "DiscreteMetaAction"
    }
})

The full corresponding action space is defined in ACTIONS_ALL

ACTIONS_ALL = {
        0: 'LANE_LEFT',
        1: 'IDLE',
        2: 'LANE_RIGHT',
        3: 'FASTER',
        4: 'SLOWER'
    }

Some of these actions might not be always available (lane changes at the edges of the roads, or accelerating/decelrating beyond the maximum/minimum velocity), and the list of available actions can be accessed with get_available_actions() method. Taking an unavailable action is equivalent to taking the IDLE action.

Similarly to continuous actions, the longitudinal (speed changes) and lateral (lane changes) actions can be disabled separately through the longitudinal and lateral parameters. For instance, in the default configuration of the intersection environment, only the speed is controlled by the agent, while the lateral control of the vehicle is automatically performed by a steering controller to track a desired lane.

Manual control

The environments can be used as a simulation:

env = gym.make("highway-v0", config={
    "manual_control": True
})
env.reset()
done = False
while not done:
    env.step(env.action_space.sample())  # with manual control, these actions are ignored

The ego-vehicle is controlled by directional arrows keys, as defined in EventHandler

API

class highway_env.envs.common.action.ActionType(env: AbstractEnv, **kwargs)

A type of action specifies its definition space, and how actions are executed in the environment

space() → Space

The action space.

property vehicle_class: Callable

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: int | ndarray) → None

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters:

action – the action to execute

get_available_actions()

For discrete action space, return the list of available actions.

property controlled_vehicle

The vehicle acted upon.

If not set, the first controlled vehicle is used by default.

class highway_env.envs.common.action.ContinuousAction(env: AbstractEnv, acceleration_range: tuple[float, float] | None = None, steering_range: tuple[float, float] | None = None, speed_range: tuple[float, float] | None = None, longitudinal: bool = True, lateral: bool = True, dynamical: bool = False, clip: bool = True, **kwargs)

An continuous action space for throttle and/or steering angle.

If both throttle and steering are enabled, they are set in this order: [throttle, steering]

The space intervals are always [-1, 1], but are mapped to throttle/steering intervals through configurations.

Create a continuous action space.

Parameters:

• env – the environment

• acceleration_range – the range of acceleration values [m/s²]

• steering_range – the range of steering values [rad]

• speed_range – the range of reachable speeds [m/s]

• longitudinal – enable throttle control

• lateral – enable steering control

• dynamical – whether to simulate dynamics (i.e. friction) rather than kinematics

• clip – clip action to the defined range

ACCELERATION_RANGE = (-5, 5.0)

[-x, x], in m/s².

Type:

Acceleration range

STEERING_RANGE = (-0.7853981633974483, 0.7853981633974483)

[-x, x], in rad.

Type:

Steering angle range

space() → Box

The action space.

property vehicle_class: Callable

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: ndarray) → None

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters:

action – the action to execute

class highway_env.envs.common.action.DiscreteAction(env: AbstractEnv, acceleration_range: tuple[float, float] | None = None, steering_range: tuple[float, float] | None = None, longitudinal: bool = True, lateral: bool = True, dynamical: bool = False, clip: bool = True, actions_per_axis: int = 3, **kwargs)

Create a continuous action space.

Parameters:

• env – the environment

• acceleration_range – the range of acceleration values [m/s²]

• steering_range – the range of steering values [rad]

• speed_range – the range of reachable speeds [m/s]

• longitudinal – enable throttle control

• lateral – enable steering control

• dynamical – whether to simulate dynamics (i.e. friction) rather than kinematics

• clip – clip action to the defined range

space() → Discrete

The action space.

act(action: int) → None

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters:

action – the action to execute

class highway_env.envs.common.action.DiscreteMetaAction(env: AbstractEnv, longitudinal: bool = True, lateral: bool = True, target_speeds: Vector | None = None, **kwargs)

An discrete action space of meta-actions: lane changes, and cruise control set-point.

Create a discrete action space of meta-actions.

Parameters:

• env – the environment

• longitudinal – include longitudinal actions

• lateral – include lateral actions

• target_speeds – the list of speeds the vehicle is able to track

ACTIONS_ALL = {0: 'LANE_LEFT', 1: 'IDLE', 2: 'LANE_RIGHT', 3: 'FASTER', 4: 'SLOWER'}

A mapping of action indexes to labels.

ACTIONS_LONGI = {0: 'SLOWER', 1: 'IDLE', 2: 'FASTER'}

A mapping of longitudinal action indexes to labels.

ACTIONS_LAT = {0: 'LANE_LEFT', 1: 'IDLE', 2: 'LANE_RIGHT'}

A mapping of lateral action indexes to labels.

space() → Space

The action space.

property vehicle_class: Callable

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: int | np.ndarray) → None

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters:

action – the action to execute

get_available_actions() → list[int]

Get the list of currently available actions.

Lane changes are not available on the boundary of the road, and speed changes are not available at maximal or minimal speed.

Returns:

the list of available actions

class highway_env.envs.common.action.MultiAgentAction(env: AbstractEnv, action_config: dict, **kwargs)

space() → Space

The action space.

property vehicle_class: Callable

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: int | ndarray) → None

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters:

action – the action to execute

get_available_actions()

For discrete action space, return the list of available actions.