#include <aspired_versions_manager.h>
Classes | |
| struct | Options |
Public Types | |
| using | PreLoadHook = BasicManager::PreLoadHook |
| using | CustomSortActionsFn = std::function< bool(const AspiredVersionPolicy::ServableAction &, const AspiredVersionPolicy::ServableAction &)> |
Public Member Functions | |
| std::vector< ServableId > | ListAvailableServableIds () const override |
| Source< std::unique_ptr< Loader > >::AspiredVersionsCallback | GetAspiredVersionsCallback () override |
| Returns a callback to set the list of aspired versions for a particular servable stream, using Loaders. | |
 Public Member Functions inherited from tensorflow::serving::Manager | |
| template<typename T > | |
| std::map< ServableId, ServableHandle< T > > | GetAvailableServableHandles () const |
| template<typename T > | |
| Status | GetServableHandle (const ServableRequest &request, ServableHandle< T > *const handle) |
Static Public Member Functions | |
| static Status | Create (Options options, std::unique_ptr< AspiredVersionsManager > *manager) |
Friends | |
| class | internal::AspiredVersionsManagerTargetImpl |
| class | test_util::AspiredVersionsManagerTestAccess |
| class | ServerCore |
| uint32 | internal::GetManagerNumLoadThreads (AspiredVersionsManager *manager) |
| std::function< void(uint32)> | internal::SetManagerNumLoadThreadsNotifier (AspiredVersionsManager *manager) |
Detailed Description
A manager that implements the Target<Loader> API which uses aspired-versions callbacks to dictate which servable versions to load. This manager also uses that API to infer which ones to unload: If a given servable version is currently loaded, and is omitted from an aspired-versions callback invocation pertaining to its servable stream, this manager interprets that omission as an implicit instruction to unload the version. See below for details.
(The implicit-unload semantics facilitates stateless Source implementations, whereby a given iteration of the Source's logic simply decides which versions of a servable ought to be loaded, without needing to know what it has decided in the past.)
This manager makes transitions between versions of a servable stream using a configured AspiredVersionPolicy. The manager prefers unloading before loading to free up resources in the server when deciding among transitions suggested by the policy.
Definition at line 85 of file aspired_versions_manager.h.
Member Function Documentation
◆ ListAvailableServableIds()
|
overridevirtual |
Gets a list of all available servable ids, i.e. each of these can be retrieved using GetServableHandle.
Implements tensorflow::serving::Manager.
Definition at line 243 of file aspired_versions_manager.cc.
The documentation for this class was generated from the following files:
- tensorflow_serving/core/aspired_versions_manager.h
- tensorflow_serving/core/aspired_versions_manager.cc
