Object

com.amazonaws.services.sagemaker.sparksdk.protobuf

ProtobufConverter

Related Doc: package protobuf

Permalink

object ProtobufConverter

Utility functions that convert to and from the Amazon Record protobuf data format and encode Records in recordIO.

See also

https://mxnet.incubator.apache.org/architecture/note_data_loading.html for more information on recordIO

https://aws.amazon.com/sagemaker/latest/dg/cdf-training.html/ for more information on the Amazon Record data format

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. ProtobufConverter
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. val ValuesIdentifierString: String

    Permalink

    Certain first-party algorithms (like K-Means, PCA, and LinearLearner) on Amazon SageMaker expect each Amazon Record protobuf message to contain a "features" map with key "values" that encodes the example's features vector.

  5. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  6. def byteArrayToProtobuf(byteArray: Array[Byte]): Record

    Permalink

    Converts a byte array to a Protobuf Record.

    Converts a byte array to a Protobuf Record.

    byteArray

    byte array to convert to Protobuf Record

    returns

    Protobuf Record from byte array

  7. def byteArrayToRecordIOEncodedByteArray(byteArray: Array[Byte]): Array[Byte]

    Permalink

    Encodes a byte array in recordIO.

    Encodes a byte array in recordIO.

    byteArray

    Byte array to encode in recordIO

    returns

    RecordIO encoded byte array

    See also

    https://mxnet.incubator.apache.org/architecture/note_data_loading.html for more information on recordIO

  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  11. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  13. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  14. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  15. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  16. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  17. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  18. def recordIOByteArrayToProtobufs(byteArray: Array[Byte]): Iterator[Record]

    Permalink

    Converts a recordIO encoded byte array of Protobuf records to an iterator of Protobuf Records

    Converts a recordIO encoded byte array of Protobuf records to an iterator of Protobuf Records

    byteArray

    recordIO encoded byte array to convert to Protobuf Record

    returns

    An Iterator over Protobuf Records

    See also

    https://mxnet.incubator.apache.org/architecture/note_data_loading.html for more information on recordIO

  19. def rowToProtobuf(row: Row, featuresFieldName: String, labelFieldName: Option[String] = Option.empty): Record

    Permalink

    Converts a Row to a SageMaker Protobuf record for training or inference.

    Converts a Row to a SageMaker Protobuf record for training or inference.

    row

    a Row with a column of Doubles for labels and a Vector for features

    featuresFieldName

    index in the row corresponding to the features column

    labelFieldName

    index in the row corresponding to the label column

    returns

    a SageMaker Protobuf record representing the Row

  20. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  21. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  22. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  24. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped