ExternalCoreEnv¶

class maze.core.agent_integration.external_core_env.ExternalCoreEnv(state_queue: queue.Queue, maze_action_queue: queue.Queue, rollout_done_event: threading.Event, renderer: Optional[maze.core.rendering.renderer.Renderer])¶

Acts as a CoreEnv in the env stack in agent integration scenario.

Designed to be run on a separate thread, alongside the agent integration running on the main thread.

Hence, the control flow is: External env (like a Unity env) controlling the agent integration object, which in turn controls this external core env, which controls the execution of rollout loop by suspending it until the next state is available from the agent integration object.

Wrappers of this env and the agents acting on top of it see it as ordinary CoreEnv, but no actual logic happens here – instead, states and associated info are obtained from the agent integration running on the main thread, and executions produced by the agents are passed back to the agent integration.

During the step function, the execution of this thread is suspended while waiting for the next state from the agent integration.

Parameters

state_queue – Queue this core env uses to get states from agent integration object
maze_action_queue – Queue this core env uses to pass executions back to agent integration object
rollout_done_event – Set by the agent integration object. Used for detection of the end of rollout period.
renderer – If available, what renderer should be associated with the state data (for rendering, plus to be serialized with trajectory data)

actor_id() → Tuple[Union[str, int], int]¶: Current actor ID set by the agent integration.

close() → None ¶: No cleanup required.

get_kpi_calculator() → Optional[maze.core.log_events.kpi_calculator.KpiCalculator]¶: No KPI calculator available.

get_maze_state() → Any¶: Return the last state obtained from the external env through agent integration.

get_renderer() → Optional[maze.core.rendering.renderer.Renderer]¶: Renderer provided by the agent integration. Might be None if not available.

get_serializable_components() → Dict[str, Any]¶: No components required.

get_step_events() → Iterable[maze.core.events.event_record.EventRecord]¶: Return events provided by the agent integration.

is_actor_done() → bool ¶: Whether last actor is done, as set by the agent integration.

reset() → Any¶

Reset is expected to be run twice – at the beginning and end of external env rollout.

At the beginning, thread execution is suspended until the initial state is available.

At the end of the rollout, just the last state is returned, as there the reset serves the only purpose of notifying the wrappers to do their processing of the previous episode. (Also, no more states are available from the external env at this point.

seed(seed: int) → None ¶: No seed required – all operation handled by external env.

set_actor_id(new_value: Tuple[Union[str, int], int])¶: Hook for the agent integration to set actor_id before querying execution.

set_is_actor_done(new_value: bool)¶: Hook for the agent integration to set the actor_done flag before querying execution.

step(maze_action: Any) → Tuple[Any, Union[float, numpy.ndarray, Any], bool, Dict[Any, Any]]¶: Relays the execution back to the agent integration. Then suspends thread execution until the next state is provided by agent integration.