Class Dl4jModelReader
- java.lang.Object
-
- org.apache.lucene.analysis.synonym.word2vec.Dl4jModelReader
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
public class Dl4jModelReader extends java.lang.Object implements java.io.Closeable
Dl4jModelReader reads the file generated by the library Deeplearning4j and provide a Word2VecModel with normalized vectorsDl4j Word2Vec documentation: https://deeplearning4j.konduit.ai/v/en-1.0.0-beta7/language-processing/word2vec Example to generate a model using dl4j: https://github.com/eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/advanced/modelling/embeddingsfromcorpus/word2vec/Word2VecRawTextExample.java
-
-
Field Summary
Fields Modifier and Type Field Description private static java.lang.String
MODEL_FILE_NAME_PREFIX
private java.util.zip.ZipInputStream
word2VecModelZipFile
-
Constructor Summary
Constructors Constructor Description Dl4jModelReader(java.io.InputStream stream)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
(package private) static BytesRef
decodeB64Term(java.lang.String term)
private static TermAndVector
extractTermAndVector(java.lang.String[] tokens, int vectorDimension, boolean isTermB64Encoded)
Word2VecModel
read()
-
-
-
Field Detail
-
MODEL_FILE_NAME_PREFIX
private static final java.lang.String MODEL_FILE_NAME_PREFIX
- See Also:
- Constant Field Values
-
word2VecModelZipFile
private final java.util.zip.ZipInputStream word2VecModelZipFile
-
-
Method Detail
-
read
public Word2VecModel read() throws java.io.IOException
- Throws:
java.io.IOException
-
extractTermAndVector
private static TermAndVector extractTermAndVector(java.lang.String[] tokens, int vectorDimension, boolean isTermB64Encoded)
-
decodeB64Term
static BytesRef decodeB64Term(java.lang.String term)
-
close
public void close() throws java.io.IOException
- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
- Throws:
java.io.IOException
-
-