SavedModel is the universal serialization format for TensorFlow models.
SavedModel provides a language-neutral format to save machine-learning models that is recoverable and hermetic. It enables higher-level systems and tools to produce, consume and transform TensorFlow models.
tf.saved_model.save
tf.saved_model.load
tf.saved_model.SaveOptions
tf.saved_model.LoadOptions
tf.saved_model.Asset
tf.saved_model.contains_saved_model
A SavedModel directory has the following structure:
assets/ assets.extra/ variables/ variables.data-?????-of-????? variables.index saved_model.pb
saved_model.pb
or saved_model.pbtxt
MetaGraphDef
protocol buffers.assets
.variables
.variables.data-?????-of-?????
variables.index
SavedModel had slightly different semantics in TF 1.x. Conventions that are generally only supported in TF 1.x are noted as such.
The following is a summary of the features in SavedModel:
SignatureDefs
Signature
.Assets
.assets
.The following is a summary of features that are NOT supported in SavedModel. Higher-level frameworks and tools that use SavedModel may provide these.
SavedModel manages and builds upon existing TensorFlow primitives such as TensorFlow Saver
and MetaGraphDef
. Specifically, SavedModel wraps a TensorFlow Saver. The Saver is primarily used to generate the variable checkpoints. SavedModel will replace the existing TensorFlow Inference Model Format as the canonical way to export TensorFlow graphs for serving.
The APIs for building and loading a SavedModel are described in this section.
The SavedModel builder is implemented in Python.
The SavedModelBuilder
class provides functionality to save multiple meta graph defs, associated variables and assets.
To build a SavedModel, the first meta graph must be saved with variables. Subsequent meta graphs will simply be saved with their graph definitions. If assets need to be saved and written or copied to disk, they can be provided when the meta graph def is added. If multiple meta graph defs are associated with an asset of the same name, only the first version is retained.
Each meta graph added to the SavedModel must be annotated with user specified tags, which reflect the meta graph capabilities or use-cases. More specifically, these tags typically annotate a meta graph with its functionality (e.g. serving or training), and possibly hardware specific aspects such as GPU. In the SavedModel, the meta graph def whose tag-set exactly matches those specified in the loader API, will be the one loaded by the loader. If no meta graph def is found matching the specified tags, an error is returned. For example, a loader with a requirement to serve on GPU hardware would be able to load only meta graph annotated with tags=‘serve,gpu’ by specifying this set of tags in tensorflow::LoadSavedModel(...).
The typical usage of builder
is as follows:
export_dir = ... ... builder = tf.saved_model.builder.SavedModelBuilder(export_dir) with tf.Session(graph=tf.Graph()) as sess: ... builder.add_meta_graph_and_variables(sess, [tf.saved_model.tag_constants.TRAINING], signature_def_map=foo_signatures, assets_collection=foo_assets) ... with tf.Session(graph=tf.Graph()) as sess: ... builder.add_meta_graph(["bar-tag", "baz-tag"]) ... builder.save()
The SavedModelBuilder class allows users to control whether default-valued attributes must be stripped from the NodeDefs while adding a meta graph to the SavedModel bundle. Both SavedModelBuilder.add_meta_graph_and_variables
and SavedModelBuilder.add_meta_graph
methods accept a Boolean flag strip_default_attrs
that controls this behavior.
If strip_default_attrs
is False
, the exported MetaGraphDef will have the default valued attributes in all it's NodeDef instances. This can break forward compatibility with a sequence of events such as the following:
Foo
) is updated to include a new attribute (T
) with a default (bool
) at version 101.Foo
.T
for Op Foo
, but tries to import this model. The model consumer doesn’t recognize attribute T
in a NodeDef that uses Op Foo
and therefore fails to load the model.By setting strip_default_attrs
to True
, the model producers can strip away any default valued attributes in the NodeDefs. This helps ensure that newly added attributes with defaults don't cause older model consumers to fail loading models regenerated with newer training binaries.
TIP: If you care about forward compatibility, then set strip_default_attrs
to True
while using SavedModelBuilder.add_meta_graph_and_variables
and SavedModelBuilder.add_meta_graph
.
The SavedModel loader is implemented in C++ and Python.
The Python version of the SavedModel loader provides load and restore capability for a SavedModel. The load
operation requires the session in which to restore the graph definition and variables, the tags used to identify the meta graph def to load and the location of the SavedModel. Upon a load, the subset of variables and assets supplied as part of the specific meta graph def, will be restored into the supplied session.
export_dir = ... ... with tf.Session(graph=tf.Graph()) as sess: tf.saved_model.loader.load(sess, [tag_constants.TRAINING], export_dir) ...
The C++ version of the SavedModel loader provides an API to load a SavedModel from a path, while allowing SessionOptions
and RunOptions
. Similar to the Python version, the C++ version requires the tags associated with the graph to be loaded, to be specified. The loaded version of SavedModel is referred to as SavedModelBundle
and contains the meta graph def and the session within which it is loaded.
const string export_dir = ... SavedModelBundle bundle; ... LoadSavedModel(session_options, run_options, export_dir, {kSavedModelTagTrain}, &bundle);
SavedModel offers the flexibility to build and load TensorFlow graphs for a variety of use-cases. For the set of most common expected use-cases, SavedModel's APIs provide a set of constants in Python and C++ that are easy to reuse and share across tools consistently.
Sets of tags can be used to uniquely identify a MetaGraphDef
saved in a SavedModel. A subset of commonly used tags is specified in:
SignatureDefs are used to define the signature of a computation supported in a TensorFlow graph. Commonly used input keys, output keys and method names are defined in: