Actions#

Similarly to observations, several types of actions can be used in every environment. They are defined in the action module. Each environment comes with a default action type, which can be changed or customised using environment configurations. For instance,

import gymnasium as gym

env = gym.make('highway-v0')
env.configure({
    "action": {
        "type": "ContinuousAction"
    }
})
env.reset()

Continuous Actions#

The ContinuousAction type allows the agent to directly set the low-level controls of the vehicle kinematics, namely the throttle \(a\) and steering angle \(\delta\).

Note

The control of throttle and steering can be enabled or disabled through the longitudinal and lateral configurations, respectively. Thus, the action space can be either 1D or 2D.

Discrete Actions#

The DiscreteAction is a uniform quantization of the ContinuousAction above.

The actions_per_axis parameter allows to set the quantization step. Similarly to continuous actions, the longitudinal and lateral axis can be enabled or disabled separately.

Discrete Meta-Actions#

The DiscreteMetaAction type adds a layer of speed and steering controllers on top of the continuous low-level control, so that the ego-vehicle can automatically follow the road at a desired velocity. Then, the available meta-actions consist in changing the target lane and speed that are used as setpoints for the low-level controllers.

The full corresponding action space is defined in ACTIONS_ALL

ACTIONS_ALL = {
        0: 'LANE_LEFT',
        1: 'IDLE',
        2: 'LANE_RIGHT',
        3: 'FASTER',
        4: 'SLOWER'
    }

Some of these actions might not be always available (lane changes at the edges of the roads, or accelerating/decelrating beyond the maximum/minimum velocity), and the list of available actions can be accessed with get_available_actions() method. Taking an unavailable action is equivalent to taking the IDLE action.

Similarly to continuous actions, the longitudinal (speed changes) and lateral (lane changes) actions can be disabled separately through the longitudinal and lateral parameters. For instance, in the default configuration of the intersection environment, only the speed is controlled by the agent, while the lateral control of the vehicle is automatically performed by a steering controller to track a desired lane.

Manual control#

The environments can be used as a simulation:

env = gym.make("highway-v0")
env.configure({
    "manual_control": True
})
env.reset()
done = False
while not done:
    env.step(env.action_space.sample())  # with manual control, these actions are ignored

The ego-vehicle is controlled by directional arrows keys, as defined in EventHandler

API#

class highway_env.envs.common.action.ActionType(env: AbstractEnv, **kwargs)[source]#

A type of action specifies its definition space, and how actions are executed in the environment

space() → Space[source]#: The action space.

property vehicle_class: Callable#

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: int | ndarray) → None[source]#

Execute the action on the ego-vehicle.

Most of the action mechanics are actually implemented in vehicle.act(action), where vehicle is an instance of the specified highway_env.envs.common.action.ActionType.vehicle_class. Must some pre-processing can be applied to the action based on the ActionType configurations.

Parameters:: action – the action to execute

get_available_actions()[source]#: For discrete action space, return the list of available actions.

property controlled_vehicle#

The vehicle acted upon.

If not set, the first controlled vehicle is used by default.

class highway_env.envs.common.action.ContinuousAction(env: AbstractEnv, acceleration_range: Tuple[float, float] | None = None, steering_range: Tuple[float, float] | None = None, speed_range: Tuple[float, float] | None = None, longitudinal: bool = True, lateral: bool = True, dynamical: bool = False, clip: bool = True, **kwargs)[source]#

An continuous action space for throttle and/or steering angle.

If both throttle and steering are enabled, they are set in this order: [throttle, steering]

The space intervals are always [-1, 1], but are mapped to throttle/steering intervals through configurations.

Create a continuous action space.

Parameters:

env – the environment
acceleration_range – the range of acceleration values [m/s²]
steering_range – the range of steering values [rad]
speed_range – the range of reachable speeds [m/s]
longitudinal – enable throttle control
lateral – enable steering control
dynamical – whether to simulate dynamics (i.e. friction) rather than kinematics
clip – clip action to the defined range

ACCELERATION_RANGE = (-5, 5.0)#

[-x, x], in m/s².

Type:: Acceleration range

STEERING_RANGE = (-0.7853981633974483, 0.7853981633974483)#

[-x, x], in rad.

Type:: Steering angle range

space() → Box[source]#: The action space.

property vehicle_class: Callable#

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: ndarray) → None[source]#

Execute the action on the ego-vehicle.

Parameters:: action – the action to execute

class highway_env.envs.common.action.DiscreteAction(env: AbstractEnv, acceleration_range: Tuple[float, float] | None = None, steering_range: Tuple[float, float] | None = None, longitudinal: bool = True, lateral: bool = True, dynamical: bool = False, clip: bool = True, actions_per_axis: int = 3, **kwargs)[source]#

Create a continuous action space.

Parameters:

env – the environment
acceleration_range – the range of acceleration values [m/s²]
steering_range – the range of steering values [rad]
speed_range – the range of reachable speeds [m/s]
longitudinal – enable throttle control
lateral – enable steering control
dynamical – whether to simulate dynamics (i.e. friction) rather than kinematics
clip – clip action to the defined range

space() → Discrete[source]#: The action space.

act(action: int) → None[source]#

Execute the action on the ego-vehicle.

Parameters:: action – the action to execute

class highway_env.envs.common.action.DiscreteMetaAction(env: AbstractEnv, longitudinal: bool = True, lateral: bool = True, target_speeds: ndarray | Sequence[float] | None = None, **kwargs)[source]#

An discrete action space of meta-actions: lane changes, and cruise control set-point.

Create a discrete action space of meta-actions.

Parameters:

env – the environment
longitudinal – include longitudinal actions
lateral – include lateral actions
target_speeds – the list of speeds the vehicle is able to track

ACTIONS_ALL = {0: 'LANE_LEFT', 1: 'IDLE', 2: 'LANE_RIGHT', 3: 'FASTER', 4: 'SLOWER'}#: A mapping of action indexes to labels.

ACTIONS_LONGI = {0: 'SLOWER', 1: 'IDLE', 2: 'FASTER'}#: A mapping of longitudinal action indexes to labels.

ACTIONS_LAT = {0: 'LANE_LEFT', 1: 'IDLE', 2: 'LANE_RIGHT'}#: A mapping of lateral action indexes to labels.

space() → Space[source]#: The action space.

property vehicle_class: Callable#

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: int | ndarray) → None[source]#

Execute the action on the ego-vehicle.

Parameters:: action – the action to execute

get_available_actions() → List[int][source]#

Get the list of currently available actions.

Lane changes are not available on the boundary of the road, and speed changes are not available at maximal or minimal speed.

Returns:: the list of available actions

class highway_env.envs.common.action.MultiAgentAction(env: AbstractEnv, action_config: dict, **kwargs)[source]#

space() → Space[source]#: The action space.

property vehicle_class: Callable#

The class of a vehicle able to execute the action.

Must return a subclass of highway_env.vehicle.kinematics.Vehicle.

act(action: int | ndarray) → None[source]#

Execute the action on the ego-vehicle.

Parameters:: action – the action to execute

get_available_actions()[source]#: For discrete action space, return the list of available actions.