Class TranslationModel

Nested Relationships

Nested Types

Class Documentation

class TranslationModel

A TranslationModel is associated with the translation of a single language direction.

Holds the graph and other structures required to run the forward pass of the neural network, along with preprocessing logic (TextProcessor) and a BatchingPool to create batches that are to be used in conjuction with an instance.

Thread-safety is not handled here, but the methods are available at granularity enough to be used in threaded async workflow for translation.

Public Types

using Config = Ptr<Options>
using ShortlistGenerator = Ptr<data::ShortlistGenerator const>

Public Functions

TranslationModel(const std::string &config, MemoryBundle &&memory, size_t replicas = 1)

Equivalent to options based constructor, where options is parsed from string configuration.

Configuration can be JSON or YAML. Keys expected correspond to those of marian-decoder, available at https://marian-nmt.github.io/docs/cmd/marian-decoder/

Note that replicas is not stable. This is a temporary workaround while a more daunting task of separating workspace from TranslationModel and binding it to threads is to be undertaken separately. Until the separation is achieved, both TranslationModel and Service will need to be aware of workers. This is expected to be resolved eventually, with only Service having the knowledge of how many workers are active.

WebAssembly uses only single-thread, and we can hardcode replicas = 1 and use it anywhere and (client) needn’t be aware of this ugliness at the moment, thus providing a stable API solely for WebAssembly single-threaded modus operandi.

TODO(@jerinphilip): Clean this up.

TranslationModel(const Config &options, MemoryBundle &&memory, size_t replicas = 1)

Construct TranslationModel from marian-options.

If memory is empty, TranslationModel is initialized from paths available in the options object, backed by filesystem. Otherwise, TranslationModel is initialized from the given MemoryBundle composed of AlignedMemory holding equivalent parameters.

Parameters

  • [in] options: Marian options object.

  • [in] memory: MemoryBundle object holding memory buffers containing parameters to build MarianBackend, ShortlistGenerator, Vocabs and SentenceSplitter.

TranslationModel(const Config &options, size_t replicas = 1)
Ptr<Request> makeRequest(size_t requestId, std::string &&source, CallbackType callback, const ResponseOptions &responseOptions, std::optional<TranslationCache> &cache)

Make a Request to be translated by this TranslationModel instance.

Parameters

  • [in] requestId: Unique identifier associated with this request, available from Service.

  • [in] source: Source text to be translated. Ownership is accepted and eventually returned to the client in Response corresponding to the Request created here.

  • [in] callback: Callback (from client) to be issued upon completion of translation of all sentences in the created Request.

  • [in] responseOptions: Configuration used to prepare the Response corresponding to the created request.

Ptr<Request> makePivotRequest(size_t requestId, AnnotatedText &&previousTarget, CallbackType callback, const ResponseOptions &responseOptions, std::optional<TranslationCache> &cache)
size_t enqueueRequest(Ptr<Request> request)

Relays a request to the batching-pool specific to this translation model.

Parameters

  • [in] request: Request constructed through makeRequest

size_t generateBatch(Batch &batch)

Generates a batch from the batching-pool for this translation model, compiling from several active requests.

Note that it is possible that calls to this method can give empty-batches.

Return

number of sentences that constitute the Batch.

Parameters

  • [out] batch: Batch to write a generated batch on to.

void translateBatch(size_t deviceId, Batch &batch)

Translate a batch generated with generateBatch.

Parameters

  • [in] deviceId: There are replicas of backend created for use in each worker thread. deviceId indicates which replica to use.

  • [in] batch: A batch generated from generateBatch from the same TranslationModel instance.

size_t modelId() const

Returns a unique-identifier for the model.