|
TensorFlow Serving C++ API Documentation
|
#include <loader_harness.h>
Classes | |
| struct | Options |
| Options to configure a LoaderHarness. More... | |
Public Types | |
| enum class | State { kNew , kLoadRequested , kLoadApproved , kLoading , kReady , kUnloadRequested , kQuiescing , kQuiesced , kUnloading , kDisabled , kError } |
Public Member Functions | |
| LoaderHarness (const ServableId &id, std::unique_ptr< Loader > loader, const Options &options=Options()) | |
| template<typename T > | |
| LoaderHarness (const ServableId &id, std::unique_ptr< Loader > loader, std::unique_ptr< T > additional_state, const Options &options=Options()) | |
| ~LoaderHarness () | |
| ServableId | id () const |
| Returns the identifier of underlying Servable. | |
| State | state () const TF_LOCKS_EXCLUDED(mu_) |
| Returns the current state of underlying Servable. | |
| Loader * | loader () const |
| template<typename T = std::nullptr_t> | |
| ServableStateSnapshot< T > | loader_state_snapshot () const TF_LOCKS_EXCLUDED(mu_) |
| Returns the current overall state snapshot of the underlying Servable. | |
| Status | LoadRequested () TF_LOCKS_EXCLUDED(mu_) |
| Status | LoadApproved () TF_LOCKS_EXCLUDED(mu_) |
| Status | Load () TF_LOCKS_EXCLUDED(mu_) |
| Status | UnloadRequested () TF_LOCKS_EXCLUDED(mu_) |
| void | set_should_retry (std::function< bool(absl::Status)> should_retry) TF_LOCKS_EXCLUDED(mu_) |
| bool | should_retry (absl::Status status) TF_LOCKS_EXCLUDED(mu_) |
| Returns true if the servable should be retried. | |
| Status | Unload () TF_LOCKS_EXCLUDED(mu_) |
| Status | StartQuiescing () TF_LOCKS_EXCLUDED(mu_) |
| Status | DoneQuiescing () TF_LOCKS_EXCLUDED(mu_) |
| void | Error (const Status &status) TF_LOCKS_EXCLUDED(mu_) |
| Transitions the state to kError and invokes 'options_.error_callback'. | |
| Status | status () const TF_LOCKS_EXCLUDED(mu_) |
| template<typename T > | |
| T * | additional_state () |
| template<> | |
| ServableStateSnapshot< std::nullptr_t > | loader_state_snapshot () const |
Static Public Member Functions | |
| static string | StateDebugString (State state) |
LoaderHarness is a widget the Manager uses to hold on to and talk to a Loader while it owns it. It tracks the overall state of a Servable such that Manager can determine which state transitions to make at what times.
A manager implementation can also add some additional state with each harness. For example, a manager could put ACL or lifecycle metadata here. The ownership is maintained by the harness.
This class is thread-safe.
Definition at line 49 of file loader_harness.h.
|
strong |
State of the underlying servable, from the perspective of the LoaderHarness and for the purpose of communication between it and a Manager. Not equivalent to the semantic servable states in servable_state.h.
Valid transitions: kNew-->kLoading-->kReady-->kQuiescing-->kQuiesced-->kUnloading-->kDisabled as well as: any_state-->kError.
| Enumerator | |
|---|---|
| kNew | Initial state. |
| kLoadRequested | The manager has been requested to load this servable. |
| kLoadApproved | The servable has been approved for loading, e.g. resources have been set aside for it. |
| kLoading | 'loader_->Load()' has been called. |
| kReady | 'loader_->Load()' has succeeded. |
| kUnloadRequested | The manager has been requested to unload this servable. |
| kQuiescing | The servable is going to be made unavailable for serving. |
| kQuiesced | The servable has been made unavailable for serving. |
| kUnloading | 'loader_->Unload()' has been called. |
| kDisabled | 'loader_->Unload()' has finished. |
| kError | An error has occurred, either during 'loader_->Load()' or outside of the harness (and was reported to the harness via a call to Error()). |
Definition at line 59 of file loader_harness.h.
|
inline |
Constructor to create a harness with additional state, which a manager needs.
Definition at line 118 of file loader_harness.h.
| tensorflow::serving::LoaderHarness::~LoaderHarness | ( | ) |
Legal to destruct iff current state is one of kNew, kDisabled or kError. Check-fails if violated.
Definition at line 43 of file loader_harness.cc.
|
inline |
Gets the additional state. Returns nullptr if the type mismatches or if it wasn't set.
Definition at line 222 of file loader_harness.h.
| Status tensorflow::serving::LoaderHarness::DoneQuiescing | ( | ) |
Transitions the state to kQuiesced, which means that there are no more live handles to this servable available in the frontend. At this point, we can unload this object.
REQUIRES: State is kQuiescing when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.
Definition at line 162 of file loader_harness.cc.
| Status tensorflow::serving::LoaderHarness::Load | ( | ) |
Transitions to kLoading, delegates to Servable::Load(), then transitions either to kReady if Load() succeeds, or to kError (and invokes 'options_. error_callback') if Load() fails. This call may take a long time.
We retry the Servable::Load() according to the options set during construction of this class. We stop retrying and give up if 1. we have reached max_num_load_retries or, 2. if cancel_load() is set to true.
REQUIRES: State is kLoadApproved when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.
Definition at line 75 of file loader_harness.cc.
| Status tensorflow::serving::LoaderHarness::LoadApproved | ( | ) |
Transitions to kLoadApproved.
REQUIRES: State is kLoadRequested when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.
Definition at line 67 of file loader_harness.cc.
|
inline |
Returns a pointer to the wrapped loader. Ownership remains with this class.
Definition at line 139 of file loader_harness.h.
| Status tensorflow::serving::LoaderHarness::LoadRequested | ( | ) |
Transitions the state of the harness to kLoadRequested iff its current state is kNew. The test-and-change is done transactionally, so this method can be used to ensure that at most one Load() request can proceed.
Definition at line 55 of file loader_harness.cc.
| void tensorflow::serving::LoaderHarness::set_should_retry | ( | std::function< bool(absl::Status)> | should_retry | ) |
Sets the retry behavior for the servable using a function which accepts the status of the last load attempt and returns a boolean. If the boolean is false, we cancel the next retry. This is best-effort, and does not preempt a Load() which is already happening, only subsequent calls.
If the retries are cancelled, the servable goes into a state dependent on the last Load() called on it. If the last Load() was successful, it will be in state kReady, else in kError.
Definition at line 141 of file loader_harness.cc.
| Status tensorflow::serving::LoaderHarness::StartQuiescing | ( | ) |
Transitions the state to kQuiescing, which means that we would like to not give out any more handles to this servable.
REQUIRES: State is kUnloadRequested when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.
Definition at line 154 of file loader_harness.cc.
| Status tensorflow::serving::LoaderHarness::status | ( | ) | const |
Whether anything has gone wrong with this servable. If state is kError, this will be non-OK. If not OK, the error could be something that occurred in a Source or SourceAdapter, in the Loader, in the Manager, or elsewhere. All errors pertaining to the servable are reported here, regardless of origin.
Definition at line 200 of file loader_harness.cc.
| Status tensorflow::serving::LoaderHarness::Unload | ( | ) |
Transitions to kUnloading, delegates to Servable::Unload(), then transitions to kDisabled when Unload() is done.
REQUIRES: State is kQuiesced when called. Otherwise DCHECK-fails, transitions to state kError and invokes 'options_.error_callback'.
Definition at line 152 of file loader_harness.cc.
| Status tensorflow::serving::LoaderHarness::UnloadRequested | ( | ) |
Transitions the state of the harness to kUnloadRequested iff its current state is kReady. The test-and-change is done transactionally, so this method can be used to ensure that at most one Load() request can proceed.
Definition at line 110 of file loader_harness.cc.
1.9.1