gxm.Timestep

class Timestep(next_obs, action, info, reward, terminated, truncated, true_next_obs)

Bases: Step

Extends Step with episodic metadata.

The “time” refers to episodic time: reward, termination, and truncation signals only exist in the context of an episode, and are meaningless for a pure world model. Timestep represents one unit of time \((R_i, S_{i+1})\) within a bounded episode.

In case of truncation, true_next_obs holds the observation \(\hat{S}_{i+1}\) that would have been seen had the episode not been cut.

__init__(next_obs, action, info, reward, terminated, truncated, true_next_obs)

Methods

__init__(next_obs, action, info, reward, ...)
trajectory(first_obs)	Convert a sequence of timesteps \((R_0, S_1, ..., S_n)\) with the first observation \(S_0\) into a trajectory \((S_0, A_0, R_0, S_1, ..., S_n)\).
transition(obs)	Convert the current timestep \((R_t, S_{t+1})\) into a transition \((S_t, A_t, R_t, S_{t+1})\) given the previous observation \(S_t\).

Attributes

done	Whether the episode has terminated or been truncated.
reward	The reward \(R_i\) received at this timestep.
terminated	Whether the episode has terminated at this timestep.
truncated	Whether the episode has been truncated at this timestep.
true_next_obs	The true next observation before any auto-reset.
next_obs	The observation at the next state.
action	The action taken at this step.
info	Additional information about the step.

property done: Array

Whether the episode has terminated or been truncated.

reward: Array

The reward \(R_i\) received at this timestep.

terminated: Array

Whether the episode has terminated at this timestep.

trajectory(first_obs)

Convert a sequence of timesteps \((R_0, S_1, ..., S_n)\) with the first observation \(S_0\) into a trajectory \((S_0, A_0, R_0, S_1, ..., S_n)\).

Parameters:

first_obs (Any) – The observation at the first timestep.

Return type:

Trajectory

Returns:

A Trajectory object containing the sequence of timesteps.

transition(obs)

Convert the current timestep \((R_t, S_{t+1})\) into a transition \((S_t, A_t, R_t, S_{t+1})\) given the previous observation \(S_t\). :type obs: Any :param obs: The observation at the previous timestep.

Return type:

Transition

Returns:

A Transition object containing the current and next timesteps.

true_next_obs: Any

The true next observation before any auto-reset. Differs from next_obs only when truncated is True.

truncated: Array

Whether the episode has been truncated at this timestep.