Class TokenInfoDictionary

  • All Implemented Interfaces:
    Dictionary

    public final class TokenInfoDictionary
    extends BinaryDictionary
    Binary dictionary implementation for a known-word dictionary model: Words are encoded into an FST mapping to a list of wordIDs.
    • Constructor Detail

      • TokenInfoDictionary

        private TokenInfoDictionary()
                             throws java.io.IOException
        Throws:
        java.io.IOException
      • TokenInfoDictionary

        @Deprecated(forRemoval=true,
                    since="9.1")
        public TokenInfoDictionary​(BinaryDictionary.ResourceScheme resourceScheme,
                                   java.lang.String resourcePath)
                            throws java.io.IOException
        Deprecated, for removal: This API element is subject to removal in a future version.
        replaced by TokenInfoDictionary(Path, Path, Path, Path) for files and TokenInfoDictionary(URL, URL, URL, URL) for classpath/module resources
        Parameters:
        resourceScheme - - scheme for loading resources (FILE or CLASSPATH).
        resourcePath - - where to load resources (dictionaries) from.
        Throws:
        java.io.IOException
      • TokenInfoDictionary

        public TokenInfoDictionary​(java.nio.file.Path targetMapFile,
                                   java.nio.file.Path posDictFile,
                                   java.nio.file.Path dictFile,
                                   java.nio.file.Path fstFile)
                            throws java.io.IOException
        Create a TokenInfoDictionary from an external resource path.
        Parameters:
        targetMapFile - where to load target map resource
        posDictFile - where to load POS dictionary resource
        dictFile - where to load dictionary entries resource
        fstFile - where to load encoded FST data resource
        Throws:
        java.io.IOException - if resource was not found or broken
      • TokenInfoDictionary

        public TokenInfoDictionary​(java.net.URL targetMapUrl,
                                   java.net.URL posDictUrl,
                                   java.net.URL dictUrl,
                                   java.net.URL fstUrl)
                            throws java.io.IOException
        Create a TokenInfoDictionary from an external resource URL (e.g. from Classpath with ClassLoader.getResource(String)).
        Parameters:
        targetMapUrl - where to load target map resource
        posDictUrl - where to load POS dictionary resource
        dictUrl - where to load dictionary entries resource
        fstUrl - where to load encoded FST data resource
        Throws:
        java.io.IOException - if resource was not found or broken
      • TokenInfoDictionary

        private TokenInfoDictionary​(IOSupplier<java.io.InputStream> targetMapResource,
                                    IOSupplier<java.io.InputStream> posResource,
                                    IOSupplier<java.io.InputStream> dictResource,
                                    IOSupplier<java.io.InputStream> fstResource)
                             throws java.io.IOException
        Throws:
        java.io.IOException
    • Method Detail

      • getClassResource

        static java.io.InputStream getClassResource​(java.lang.String suffix)
                                             throws java.io.IOException
        Throws:
        java.io.IOException