#include <aspired_versions_manager.h>
Public Attributes | |
| std::unique_ptr< ResourceTracker > | resource_tracker |
| int64_t | manage_state_interval_micros = 100 * 1000 |
| EventBus< ServableState > * | servable_event_bus = nullptr |
| std::unique_ptr< AspiredVersionPolicy > | aspired_version_policy |
| The AspiredVersionPolicy to use for the manager. Must be non-null. | |
| CustomSortActionsFn | custom_sort_actions |
| uint32 | num_load_threads = 0 |
| uint32 | num_unload_threads = 0 |
| uint32 | max_num_load_retries = 5 |
| int64_t | load_retry_interval_micros = 1LL * 60 * 1000 * 1000 |
| std::function< bool(absl::Status)> | should_retry_model_load |
| bool | flush_filesystem_caches = false |
| Env * | env = Env::Default() |
| PreLoadHook | pre_load_hook |
| bool | enable_reload_servables_with_error = false |
| bool | with_current_context = false |
Detailed Description
Config options and pluggable objects that will be used by the AspiredVersionsManager.
Definition at line 96 of file aspired_versions_manager.h.
Member Data Documentation
◆ custom_sort_actions
| CustomSortActionsFn tensorflow::serving::AspiredVersionsManager::Options::custom_sort_actions |
Given a list of ServableAction, each ServableAction representing the chosen version for that servable, this provides a custom sort order on which action to take first. Useful when certain servable needs to be loaded or unloaded before some other servable
Definition at line 117 of file aspired_versions_manager.h.
◆ env
| Env* tensorflow::serving::AspiredVersionsManager::Options::env = Env::Default() |
The environment to use for starting threads in the thread-pool or for sleeping.
Definition at line 151 of file aspired_versions_manager.h.
◆ load_retry_interval_micros
| int64_t tensorflow::serving::AspiredVersionsManager::Options::load_retry_interval_micros = 1LL * 60 * 1000 * 1000 |
The interval, in microseconds, between each servable load retry. If set negative, we don't wait. Default: 1 minute.
Definition at line 138 of file aspired_versions_manager.h.
◆ manage_state_interval_micros
| int64_t tensorflow::serving::AspiredVersionsManager::Options::manage_state_interval_micros = 100 * 1000 |
The periodicity, in microseconds, of the thread which manages the state of the servables. Default: 100 milliseconds. If this is set less than or equal to 0, we don't run this thread at all.
Definition at line 104 of file aspired_versions_manager.h.
◆ max_num_load_retries
| uint32 tensorflow::serving::AspiredVersionsManager::Options::max_num_load_retries = 5 |
Maximum number of times we retry loading a servable, after the first failure, before we give up.
Definition at line 133 of file aspired_versions_manager.h.
◆ num_load_threads
| uint32 tensorflow::serving::AspiredVersionsManager::Options::num_load_threads = 0 |
The number of threads in the thread-pool used to load servables.
If set as 0, we don't use a thread-pool, and servable loads are performed serially in the manager's main work loop.
Definition at line 123 of file aspired_versions_manager.h.
◆ num_unload_threads
| uint32 tensorflow::serving::AspiredVersionsManager::Options::num_unload_threads = 0 |
The number of threads in the thread-pool used to unload servables.
If set as 0, we don't use a thread-pool, and servable unloads are performed serially in the manager's main work loop.
Definition at line 129 of file aspired_versions_manager.h.
◆ pre_load_hook
| PreLoadHook tensorflow::serving::AspiredVersionsManager::Options::pre_load_hook |
Callback to be called just before a servable is to be loaded. This will called on the same manager load thread which starts the load.
Definition at line 155 of file aspired_versions_manager.h.
◆ resource_tracker
| std::unique_ptr<ResourceTracker> tensorflow::serving::AspiredVersionsManager::Options::resource_tracker |
The resource tracker to use while managing servable resources. Optional. If left as nullptr, we do not validate servable resource usage.
Definition at line 99 of file aspired_versions_manager.h.
◆ servable_event_bus
| EventBus<ServableState>* tensorflow::serving::AspiredVersionsManager::Options::servable_event_bus = nullptr |
EventBus to publish servable state changes. This is optional, if unset, we don't publish.
Definition at line 108 of file aspired_versions_manager.h.
The documentation for this struct was generated from the following file:
- tensorflow_serving/core/aspired_versions_manager.h
