#include <loader.h>
Inheritance diagram for tensorflow::serving::ResourceUnsafeLoader:
Collaboration diagram for tensorflow::serving::ResourceUnsafeLoader:
Public Member Functions | |
| Status | EstimateResources (ResourceAllocation *estimate) const final |
 Public Member Functions inherited from tensorflow::serving::Loader | |
| virtual | ~Loader ()=default |
| virtual Status | Load () |
| virtual Status | LoadWithMetadata (const Metadata &metadata) |
| virtual void | Unload ()=0 |
| virtual AnyPtr | servable ()=0 |
Detailed Description
A Loader that is oblivious to resources. Its EstimateResources() method returns 0, thus effectively disabling resource-based safety checks in the serving system.
Loaders that are experimental, or run in environments that do not need the resource safety checks, can subclass ResourceUnsafeLoader instead of Loader.
Member Function Documentation
◆ EstimateResources()
|
inlinefinalvirtual |
Estimates the resources a servable will use.
IMPORTANT: This method's implementation must obey following requirements, which enable the serving system to reason correctly about which servables can be loaded safely:
- The estimate must represent an upper bound on the actual value.
- Prior to load, the estimate may include resources that are not bound to any specific device instance, e.g. RAM on one of the two GPUs.
- While loaded, for any devices with multiple instances (e.g. two GPUs), the estimate must specify the instance to which each resource is bound.
- The estimate must be monotonically non-increasing, i.e. it cannot increase over time. Reasons to have it potentially decrease over time
- Returns
- an estimate of the resources the servable will consume once loaded. If the servable has already been loaded, returns an estimate of the actual resource usage.
Implements tensorflow::serving::Loader.
The documentation for this class was generated from the following file:
- tensorflow_serving/core/loader.h
