Class Dl4jModelReader

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable

    public class Dl4jModelReader
    extends java.lang.Object
    implements java.io.Closeable
    Dl4jModelReader reads the file generated by the library Deeplearning4j and provide a Word2VecModel with normalized vectors

    Dl4j Word2Vec documentation: https://deeplearning4j.konduit.ai/v/en-1.0.0-beta7/language-processing/word2vec Example to generate a model using dl4j: https://github.com/eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/advanced/modelling/embeddingsfromcorpus/word2vec/Word2VecRawTextExample.java

    • Field Detail

      • MODEL_FILE_NAME_PREFIX

        private static final java.lang.String MODEL_FILE_NAME_PREFIX
        See Also:
        Constant Field Values
      • word2VecModelZipFile

        private final java.util.zip.ZipInputStream word2VecModelZipFile
    • Constructor Detail

      • Dl4jModelReader

        public Dl4jModelReader​(java.io.InputStream stream)
    • Method Detail

      • read

        public Word2VecModel read()
                           throws java.io.IOException
        Throws:
        java.io.IOException
      • extractTermAndVector

        private static TermAndVector extractTermAndVector​(java.lang.String[] tokens,
                                                          int vectorDimension,
                                                          boolean isTermB64Encoded)
      • decodeB64Term

        static BytesRef decodeB64Term​(java.lang.String term)
      • close

        public void close()
                   throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Throws:
        java.io.IOException