Variable Structure Definition

[hedaoyuan]

Discuss the definition of variable structure

[pkuyym]

Blob of caffe2

Description

Blob is a general container that hosts a typed pointer. A Blob hosts a pointer as well as its type, and takes charge of deleting it properly when the blob is deallocated or re-allocated with a new type. A blob could contain anything, although the most common case is to contain a Tensor.

Key interfaces

Fucntion	Description
Blob()	Initializes an empty Blob.
IsType() const	Checks if the content stored in the blob is of type T
meta() const	Returns the meta info of the blob.
TypeName() const	Returns a printable typename of the blob.
Get() const	Gets the const reference of the stored object.
GetRaw() const
GetRaw()
GetMutable(bool* is_new=nullptr)	Gets a mutable pointer to the stored object.
Reset(T* allocated)	Sets the underlying object to the allocated one.
ShareExternal(typname std::remove_const::type* allocated)	Sets the underlying object to the allocated one, but does not take over the ownership of the passed in pointer.
ShareExternal(void* allocated, const TypeMeta& meta)
Reset()	Resets the Blob to an empty one.
Serialize (const string &name, BlobSerializerBase::SerializationAcceptor acceptor, int chunk_size=kDefaultChunkSize) const	Serializes the current blob, if possible.
Serialize (const string &name) const	Convenience function to serialize a blob to a string.
swap (Blob &rhs)	Swaps the underlying storage of two blobs.
Deserialize (const string &content)	Deserializes from a string containing either BlobProto or TensorProto.

Reference

Please go to https://caffe2.ai/doxygen-c/html/classcaffe2_1_1Blob.html for detailed description.

[hedaoyuan]

Define the Variable as the data structure of the parameter of the Operator. Currently, the data structure of the parameter of Layer in Paddle is Argument. So, I think the smallest definition of Variable is to be able to represent the variable type that Argument can represent.

[pkuyym]

Discussion about Blob()、Blob(Blob&& blob)、operator=(Blob&& other)、Reset(T* allocated) and ShareExternal(void* allocated, const TypeMeta& meta)

Input parameters:

• Blob() is default constructor.
• Blob(Blob&& blob) and operator=(Blob&& other) accept a right value reference of Blob object, the usage scenario is very clear and self-explaining.
• Reset and ShareExternal accept any type object. The old pointer_ will be replaced by passed in pointer.

Difference about Reset and ShareExternal:

• They all sets the underlying object (pointer_) to the allocated one.
• The Blob will take over the ownership of the passed in pointer when calling Reset, Reset is used when the underlying class T does not have a default ctor, or complex initializations needs to be done outside the blob.
• The Blob does not take over the ownership of the pointer when calling ShareExternal and the caller is responsible for making sure that the lifetime of the allocated blob outlasts the lifetime of any access to this blob, until another Reset call is made or the blob is destructed.

Implementation of Reset and ShareExternal:

template <class T>
T* Reset(T* allocated) { // notice the return type, doesn't need meta data
  if (pointer_ && destroy_) {
    destroy_(pointer_);
  }
  meta_ = TypeMeta::Make<T>(); // construct meta data
  pointer_ = static_cast<void*>(allocated);
  destroy_ = &Destroy<T>; // hold the destroy function
  return allocated;
}

void* ShareExternal(void* allocated, const TypeMeta& meta) { // return void* and need meta data
  if (pointer_ && destroy_) {
    destroy_(pointer_);
  }
  meta_ = meta;
  pointer_ = static_cast<void*>(allocated);
  destroy_ = nullptr; // doesn't hold the destroy function
  return allocated;
}

Calling snippet code:

Calling Reset in tutorial_blob.cc line 65~66.

std::string* pvec = new std::string();
myblob.Reset(pvec); // no need to release pvec, myblob takes ownership.

Calling ShareExternal in operator_fallback_gpu.h line 71~76.

// Note(jiayq): This removes a const but conceptually
// local_input_blobs will only be used as const blob input for the
// base op so we are still fine.
local_input_blobs_[i]->ShareExternal(
  const_cast<void*>(OperatorBase::Inputs()[i]->GetRaw()),
  OperatorBase::Inputs()[i]->meta());

[hedaoyuan]

According to the previous comment, here to discuss the type of Argument, and what needs to be retained in the type of Variable.
Argument member variable

  MatrixPtr in;
  MatrixPtr value;
  IVectorPtr ids;
  MatrixPtr grad;
  ICpuGpuVectorPtr sequenceStartPositions;
  int deviceId;
  size_t frameHeight;
  size_t frameWidth;

1. MatrixPtr can be used to represent a real type of dense data;
2. When the data in value is larger than two dimensions, it needs to be represented by additional member variable(for example, frameHeight and frameWidth are required for image data);
3. Use grad to represent the corresponding gradient of value;
4. MatrixPtr can be used to represent a real type of sparse data(SparseMatrix inherited from Matrix) data;
5. Using a MatrixPtr and an IVectorPtr to represent a dense sequence of data;
6. Use an IVectorPtr to represent a sequence id data;
7. Use a deviceId to indicate which device the Argument is on.

For the first and second points, in the Variable we should replace the Matrix type with a Tensor type that can represent multidimensionally.
For the third point, a Variable represents only one data. The grad of the value is recorded by the Scope not Variable.
For the 4,5,6 three points, need to design a data structure, able to store different types of data.
For the seventh point, deviceId is stored in the Tensor type.

[qingqing01]

TypeMeta in Caffe2.

TypeMeta is a thin class that allows us to store the type of a container such as a blob, or the data type of a tensor, with a unique run-time id. It also stores some additional data such as the item size and the name of the type for run-time inspection.

The basic interface:

 class TypeMeta {
 
 private:
   CaffeTypeId id_; 
   size_t itemsize_;
   PlacementNew ctor_; 
   TypedCopy copy_;
   TypedDestructor dtor_;
};

• ctor: A placement new wrapper. The usage scenarios are as follows:
  • determine whether fundamental type.
    • https://github.com/caffe2/caffe2/blob/master/caffe2/operators/utility_ops.h#L1431
  • type conversion for the non-fundamental type.
    • https://github.com/caffe2/caffe2/blob/master/caffe2/core/tensor.h#L507
• dtor_: corresponding destructor.
• copy_: corresponding copy function.

Usage in blob

Only the type id and name (typid(T).name()) are used.

• type check in run-time base the type id.
• return type name.
• create correct serializer based on the type id.

Summary

At first, we can build our tiny and complete Variable with simplified TypeMeta. Comparing to TypeMeta in Caffe2, the simplified TypeMeta only reserves a type id manager. We can use a global mapper and register to implement the manager, in which the mapper maps type id to type name and register makes the type id unique.

[hedaoyuan]

close by #2587