#include <servable_state_monitor.h>

Classes
struct	Options

struct	ServableStateAndTime

Public Types
using	ServableName = string

using	Version = int64

using	VersionMap = std::map< Version, ServableStateAndTime, std::greater< Version > >

using	ServableMap = std::map< ServableName, VersionMap >

using	ServableSet = std::set< ServableName >

using	BoundedLog = std::deque< ServableStateAndTime >

using	ServableStateNotifierFn = std::function< void(bool reached_goal_state, const std::map< ServableId, ServableState::ManagerState > &states_reached)>

using	NotifyFn = std::function< void(const ServableState &)>

Public Member Functions
	ServableStateMonitor (EventBus< ServableState > *bus, const Options &options=Options())

absl::optional< ServableState >	GetState (const ServableId &servable_id) const TF_LOCKS_EXCLUDED(mu_)

absl::optional< ServableStateAndTime >	GetStateAndTime (const ServableId &servable_id) const TF_LOCKS_EXCLUDED(mu_)

VersionMap	GetVersionStates (const string &servable_name) const TF_LOCKS_EXCLUDED(mu_)

ServableMap	GetAllServableStates () const TF_LOCKS_EXCLUDED(mu_)
	Returns the current states of all tracked versions of all servables.

ServableMap	GetLiveServableStates () const TF_LOCKS_EXCLUDED(mu_)

void	ForgetUnloadedServableStates () TF_LOCKS_EXCLUDED(mu_)

ServableSet	GetAvailableServableStates () const TF_LOCKS_EXCLUDED(mu_)

BoundedLog	GetBoundedLog () const TF_LOCKS_EXCLUDED(mu_)
	Returns the current bounded log of handled servable state events.

void	NotifyWhenServablesReachState (const std::vector< ServableRequest > &servables, ServableState::ManagerState goal_state, const ServableStateNotifierFn &notifier_fn) TF_LOCKS_EXCLUDED(mu_)

bool	WaitUntilServablesReachStateWithTimeout (const std::vector< ServableRequest > &servables, ServableState::ManagerState goal_state, absl::Duration timeout, std::map< ServableId, ServableState::ManagerState > *states_reached=nullptr) TF_LOCKS_EXCLUDED(mu_) TF_MUST_USE_RESULT

bool	WaitUntilServablesReachState (const std::vector< ServableRequest > &servables, ServableState::ManagerState goal_state, std::map< ServableId, ServableState::ManagerState > *states_reached=nullptr) TF_MUST_USE_RESULT

void	Notify (const NotifyFn &notify_fn) TF_LOCKS_EXCLUDED(notify_mu_)

Detailed Description

A utility that listens to an EventBus<ServableState>, and keeps track of the state of each servable mentioned on the bus. The intended use case is to track the states of servables in a Manager.

Offers an interface for querying the servable states. It may be useful as the basis for dashboards, as well as for testing a manager.

IMPORTANT: You must create this monitor before arranging for events to be published on the event bus, e.g. giving the event bus to a Manager.

Definition at line 46 of file servable_state_monitor.h.

Member Typedef Documentation

◆ ServableStateNotifierFn

using tensorflow::serving::ServableStateMonitor::ServableStateNotifierFn = std::function<void( bool reached_goal_state, const std::map<ServableId, ServableState::ManagerState>& states_reached)>

Notifies when all of the servables have reached the 'goal_state'.

Servables can be specified in two ways:

As specific versions of a servable stream name. In this case, we check whether the specific version has reached the 'goal_state' or kEnd.
As latest versions, in which case any version for a servable stream name will be matched against the 'goal_state' or kEnd.

We call the 'notifier_fn' when both conditions are true -

All of the specific servable requests have either reached the 'goal_state' or kEnd.
All of the latest servable requests have reached 'goal_state' or kEnd. The 'notifier_fn' will be called only once, and not repeatedly.

The 'reached_goal_state' argument is set as true iff all of the specific servables have reached 'goal_state'. So callers should verify that 'reached_goal_state' is true in the 'notifier_fn'.

The 'states_reached' argument is populated with the servable's id and the state it reached. The state would be 'goal_state' if 'reached_goal_state' is true, else it will contain one or more servables in kEnd state. For latest servable requests, the servable id will be the id of the servable in the stream which reached the state.

Definition at line 147 of file servable_state_monitor.h.

Member Function Documentation

◆ ForgetUnloadedServableStates()

void tensorflow::serving::ServableStateMonitor::ForgetUnloadedServableStates ( )

Removes all servable versions from the ServableMap whose states have transitioned to kEnd.

Definition at line 182 of file servable_state_monitor.cc.

◆ GetLiveServableStates()

ServableStateMonitor::ServableMap tensorflow::serving::ServableStateMonitor::GetLiveServableStates ( ) const

Returns the current states of all versions of all servables which have not transitioned to state ServableState::ManagerState::kEnd.

Definition at line 176 of file servable_state_monitor.cc.

◆ GetState()

absl::optional< ServableState > tensorflow::serving::ServableStateMonitor::GetState ( const ServableId & servable_id ) const

Returns the current state of one servable, or nullopt if that servable is not being tracked.

Definition at line 150 of file servable_state_monitor.cc.

◆ GetStateAndTime()

absl::optional< ServableStateMonitor::ServableStateAndTime > tensorflow::serving::ServableStateMonitor::GetStateAndTime ( const ServableId & servable_id ) const

Returns the current state and time of one servable, or nullopt if that servable is not being tracked.

Definition at line 145 of file servable_state_monitor.cc.

◆ GetVersionStates()

ServableStateMonitor::VersionMap tensorflow::serving::ServableStateMonitor::GetVersionStates ( const string & servable_name ) const

Returns the current states of all tracked versions of the given servable, if any.

Definition at line 160 of file servable_state_monitor.cc.

◆ WaitUntilServablesReachStateWithTimeout()

bool tensorflow::serving::ServableStateMonitor::WaitUntilServablesReachStateWithTimeout	(	const std::vector< ServableRequest > &	servables,
		ServableState::ManagerState	goal_state,
		absl::Duration	timeout,
		std::map< ServableId, ServableState::ManagerState > *	states_reached = `nullptr`
	)

Similar to NotifyWhenServablesReachState(...), but instead of notifying, we wait until the 'goal_state' or kEnd is reached.

To understand the return value and the return parameter 'states_reached', please read the documentation on NotifyWhenServablesReachState(...). WaitUntilServablesReachStateWithTimeout and WaitUntilServablesReachState perform the same function, but the former has a timeout while the latter waits indefinitely.

Definition at line 238 of file servable_state_monitor.cc.

The documentation for this class was generated from the following files:

tensorflow_serving/core/servable_state_monitor.h
tensorflow_serving/core/servable_state_monitor.cc

Classes

Public Types

Public Member Functions