tensorflow::serving::SimpleLoader< ServableType > Class Template Reference

Inheritance diagram for tensorflow::serving::SimpleLoader< ServableType >:

Collaboration diagram for tensorflow::serving::SimpleLoader< ServableType >:

Public Types
using	Creator = std::function< Status(std::unique_ptr< ServableType > *)>
using	CreatorWithMetadata = std::function< Status(const Metadata &, std::unique_ptr< ServableType > *)>
using	CreatorVariant = absl::variant< Creator, CreatorWithMetadata >
using	ResourceEstimator = std::function< Status(ResourceAllocation *)>

Public Member Functions
	SimpleLoader (Creator creator, ResourceEstimator resource_estimator)
	SimpleLoader (CreatorWithMetadata creator_with_metadata, ResourceEstimator resource_estimator)
	SimpleLoader (Creator creator, ResourceEstimator resource_estimator, ResourceEstimator post_load_resource_estimator)
	SimpleLoader (CreatorWithMetadata creator_with_metadata, ResourceEstimator resource_estimator, ResourceEstimator post_load_resource_estimator)
	SimpleLoader (CreatorVariant creator_variant, ResourceEstimator resource_estimator, absl::optional< ResourceEstimator > post_load_resource_estimator)
Status	EstimateResources (ResourceAllocation *estimate) const override
Status	Load () override
Status	LoadWithMetadata (const Metadata &metadata) override
void	Unload () override
AnyPtr	servable () override
Public Member Functions inherited from tensorflow::serving::Loader
virtual	~Loader ()=default

Static Public Member Functions
static ResourceEstimator	EstimateNoResources ()

Detailed Description

template<typename ServableType>
class tensorflow::serving::SimpleLoader< ServableType >

Definition at line 72 of file simple_loader.h.

Member Function Documentation

EstimateResources()

template<typename ServableType >

Status tensorflow::serving::SimpleLoader< ServableType >::EstimateResources ( ResourceAllocation *  estimate ) const

overridevirtual

Estimates the resources a servable will use.

IMPORTANT: This method's implementation must obey following requirements, which enable the serving system to reason correctly about which servables can be loaded safely:

1. The estimate must represent an upper bound on the actual value.
2. Prior to load, the estimate may include resources that are not bound to any specific device instance, e.g. RAM on one of the two GPUs.
3. While loaded, for any devices with multiple instances (e.g. two GPUs), the estimate must specify the instance to which each resource is bound.
4. The estimate must be monotonically non-increasing, i.e. it cannot increase over time. Reasons to have it potentially decrease over time

   Returns

   an estimate of the resources the servable will consume once loaded. If the servable has already been loaded, returns an estimate of the actual resource usage.

Implements tensorflow::serving::Loader.

Definition at line 279 of file simple_loader.h.

Load()

template<typename ServableType >

Status tensorflow::serving::SimpleLoader< ServableType >::Load

overridevirtual

Fetches any data that needs to be loaded before using the servable returned by servable(). May use no more resources than the estimate reported by EstimateResources().

If implementing Load(), you don't have to override LoadWithMetadata().

Reimplemented from tensorflow::serving::Loader.

Definition at line 294 of file simple_loader.h.

LoadWithMetadata()

template<typename ServableType >

Status tensorflow::serving::SimpleLoader< ServableType >::LoadWithMetadata ( const Metadata &  metadata )

overridevirtual

Similar to the above method, but takes Metadata as a param, which may be used by the loader implementation appropriately.

If you're overriding LoadWithMetadata(), because you can use the metadata appropriately, you can skip overriding Load().

Reimplemented from tensorflow::serving::Loader.

Definition at line 306 of file simple_loader.h.

servable()

template<typename ServableType >

AnyPtr tensorflow::serving::SimpleLoader< ServableType >::servable ( )

inlineoverridevirtual

Returns an opaque interface to the underlying servable object. The caller should know the precise type of the interface in order to make actual use of it. For example:

CustomLoader implementation:

class CustomLoader : public Loader {
 public:
  ...
  Status Load() override {
    servable_ = ...;
  }

  AnyPtr servable() override { return servable_; }

 private:
  CustomServable* servable_ = nullptr;
};

Serving user request:

ServableHandle<CustomServable> handle = ...
CustomServable* servable = handle.get();
servable->...

If servable() is called after successful Load() and before Unload(), it returns a valid, non-null AnyPtr object. If called before a successful Load() call or after Unload(), it returns null AnyPtr.

Implements tensorflow::serving::Loader.

Definition at line 133 of file simple_loader.h.

Unload()

template<typename ServableType >

void tensorflow::serving::SimpleLoader< ServableType >::Unload ( )

overridevirtual

Frees any resources allocated during Load() (except perhaps for resources shared across servables that are still needed for other active ones). The loader does not need to return to the "new" state (i.e. Load() cannot be called after Unload()).

Implements tensorflow::serving::Loader.

Definition at line 352 of file simple_loader.h.

---

The documentation for this class was generated from the following file:

• tensorflow_serving/core/simple_loader.h