This is a memory network agent that inherits from the TorchAgent class. Read more about Memory Networks here.

Basic Examples

Train a memory network on the bAbi task.

parlai train_model -m memnn -t babi:task10k:1 -mf /tmp/memnn_babi.mdl

MemnnAgent Options

MemNN Arguments



-esz, --embedding-size

Size of token embeddings

Default: 128.

-hops, --hops

Number of memory hops

Default: 3.


Size of memory, set to 0 for “nomemnn” model which just embeds query and candidates and picks most similar candidate

Default: 32.

-tf, --time-features

Use time features for memory embeddings

Default: True.

-pe, --position-encoding

Use position encoding instead of bag of words embedding

Default: False.

TorchAgent Arguments



-i, --interactive-mode

Whether in full interactive mode or not, which means generating text or retrieving from a full set of candidates, which is necessary to actually do full dialogue. However, during training or quick validation (e.g. PPL for generation or ranking a few candidates for ranking models) you might want these set to off. Typically, scripts can set their preferred default behavior at the start, e.g. eval scripts.

Default: False.

-emb, --embedding-type

Choose between different strategies for initializing word embeddings. Default is random, but can also preinitialize from Glove or Fasttext. Preinitialized embeddings can also be fixed so they are not updated during training.

Choices: random, glove, glove-fixed, fasttext, fasttext-fixed, fasttext_cc, fasttext_cc-fixed.

Default: random.

-embp, --embedding-projection

If pretrained embeddings have a different dimensionality than your embedding size, strategy for projecting to the correct size. If the dimensions are the same, this is ignored unless you append “-force” to your choice.

Default: random.


Use fp16 computations.

Default: False.


Implementation of FP16 to use

Choices: safe, mem_efficient.

Default: safe.

-rc, --rank-candidates

Whether the model should parse candidates for ranking.

Default: False.

-tr, --truncate

Truncate input lengths to increase speed / use less memory.

Default: -1.


Text input truncation length: if not specified, this will default to truncate


Label truncation length: if not specified, this will default to truncate


Reverse the history

Default: False.

-histsz, --history-size

Number of past dialog utterances to remember.

Default: -1.

-pt, --person-tokens

Add person tokens to history. adds p1 in front of input text and p2 in front of past labels when available or past utterances generated by the model. these are added to the dictionary during initialization.

Default: False.


Split the dialogue history on newlines and save in separate vectors

Default: False.


Join history lines with this token, defaults to newline

Default: \n.


Comma separated list of special tokens. In case of ambiguous parses from special tokens, the ordering provided in this arg sets precedence.

-gpu, --gpu

Which GPU to use

Default: -1.


Disable GPUs even if available. otherwise, will use GPUs if available on the device.

Default: False.

Optimizer Arguments



-opt, --optimizer

Optimizer choice. Possible values: adadelta, adagrad, adam, adamw, sparseadam, adamax, asgd, sgd, rprop, rmsprop, optimizer, lbfgs, mem_eff_adam, adafactor.

Choices: adadelta, adagrad, adam, adamw, sparseadam, adamax, asgd, sgd, rprop, rmsprop, optimizer, lbfgs, mem_eff_adam, adafactor.

Default: sgd.

-lr, --learningrate

Learning rate

Default: 1.

-clip, --gradient-clip

Gradient clipping using l2 norm

Default: 0.1.


Epsilon values for adafactor optimizer: regularization constants for square gradient and parameter scale respectively

Default: 1e-30,1e-3. Recommended: 1e-30,1e-3.

-mom, --momentum

If applicable, momentum value for optimizer.

Default: 0.


If applicable, whether to use nesterov momentum.

Default: True.

-nu, --nus

If applicable, nu value(s) for optimizer. can use a single value like 0.7 or a comma-separated tuple like 0.7,1.0

Default: 0.7.

-beta, --betas

If applicable, beta value(s) for optimizer. can use a single value like 0.9 or a comma-separated tuple like 0.9,0.999

Default: 0.9,0.999.

-wdecay, --weight-decay

Weight decay on the weights.

Learning Rate Scheduler




Learning rate scheduler.

Choices: reduceonplateau, none, fixed, invsqrt, cosine, linear.

Default: reduceonplateau.


LR scheduler patience. In number of validation runs. If using fixed scheduler, LR is decayed every validations.

Default: 3.


Decay factor for LR scheduler, or how much LR is multiplied by when it is lowered.

Default: 0.5.


Constant used only to find the lr multiplier for the invsqrt scheduler. Must be set for –lr-scheduler invsqrt

Default: -1.




-cands, --candidates

The source of candidates during training (see TorchRankerAgent._build_candidates() for details).

Choices: batch, inline, fixed, batch-all-cands.

Default: inline.

-ecands, --eval-candidates

The source of candidates during evaluation (defaults to the samevalue as –candidates if no flag is given)

Choices: batch, inline, fixed, vocab, batch-all-cands.

Default: inline.

-icands, --interactive-candidates

The source of candidates during interactive mode. Since in interactive mode, batchsize == 1, we cannot use batch candidates.

Choices: fixed, inline, vocab.

Default: fixed.


Block repeating previous utterances. Helpful for many models that score repeats highly, so switched on by default.

Default: True.

-fcp, --fixed-candidates-path

A text file of fixed candidates to use for all examples, one candidate per line


One of “reuse”, “replace”, or a path to a file with vectors corresponding to the candidates at –fixed-candidates-path. The default path is a /path/to/model-file.<cands_name>, where <cands_name> is the name of the file (not the full path) passed by the flag –fixed-candidates-path. By default, this file is created once and reused. To replace it, use the “replace” option.

Default: reuse.


Cache and save the encoding of the candidate vecs. This might be used when interacting with the model in real time or evaluating on fixed candidate set when the encoding of the candidates is independent of the input.

Default: True.


Initialize model with weights from this file.


Get predictions and calculate mean rank during the train step. Turning this on may slow down training.

Default: False.


Limit to the number of predictions in output.text_candidates

Default: 100.


Ignore examples for which the label is not present in the label candidates. Default behavior results in RuntimeError.

Default: False.


Ranking returns the top k results of k > 0, otherwise sorts every single candidate according to the ranking.

Default: -1.


Final response output algorithm

Choices: topk, max.

Default: max.


K used in Top K sampling inference, when selected

Default: 5.


Return sorted candidate scores from eval_step

Default: False.

BPEHelper Arguments




Path to pre-trained tokenizer vocab


Path to pre-trained tokenizer merge


Use BPE dropout during training.