BERT Ranker

This directory contains several implementations of a ranker based on a pretrained language model BERT (Devlin et al. https://arxiv.org/abs/1810.04805). It relies on the pytorch implementation provided by Hugging Face (https://github.com/huggingface/pytorch-pretrained-BERT).

Content

This directory contains 3 Torch Ranker Agents (see parlai/core/torch_ranker_agent.py). All of them are rankers, which means that given a context, they try to guess what is the next utterance among a set of candidates.

  • BiEncoderRankerAgent associates a vector to the context and a vector to every possible utterance, and is trained to maximize the dot product between the correct utterance and the context.

  • CrossEncoderRankerAgent concatenate the text with a candidate utterance and gives a score. This scales much less that BiEncoderRankerAgent at inference time since you can not precompute a vector per candidate. However, it tends to give higher accuracy.

  • BothEncoderRankerAgent does both, it ranks the top N candidates using a BiEncoder and follows it by a CrossEncoder. Resulting in a scalable and precise system.

Preliminary

In order to use those agents you need to install pytorch-pretrained-bert (https://github.com/huggingface/pytorch-pretrained-BERT). If you have not installed, running the model will prompt you to run: pip install pytorch-pretrained-bert

Basic Examples

Train a BiEncoder BERT model on ConvAI2:

parlai train_model -t convai2 -m bert_ranker/bi_encoder_ranker --batchsize 20 -vtim 30 --model-file /tmp/bert_biencoder_test --data-parallel True

Train a CrossEncoder BERT model on ConvAI2:

parlai train_model -t convai2 -m bert_ranker/cross_encoder_ranker --batchsize 2 -vtim 30 --model-file /tmp/bert_crossencoder_test --data-parallel True

BiEncoderRankerAgent Options

TorchAgent Arguments

Argument

Description

--interactive-mode, --i

Whether in full interactive mode or not, which means generating text or retrieving from a full set of candidates, which is necessary to actually do full dialogue. However, during training or quick validation (e.g. PPL for generation or ranking a few candidates for ranking models) you might want these set to off. Typically, scripts can set their preferred default behavior at the start, e.g. eval scripts.

Default: False.

--embedding-type, --emb

Choose between different strategies for initializing word embeddings. Default is random, but can also preinitialize from Glove or Fasttext. Preinitialized embeddings can also be fixed so they are not updated during training.

Choices: random, glove, glove-fixed, fasttext, fasttext-fixed, fasttext_cc, fasttext_cc-fixed.

Default: random.

--embedding-projection, --embp

If pretrained embeddings have a different dimensionality than your embedding size, strategy for projecting to the correct size. If the dimensions are the same, this is ignored unless you append “-force” to your choice.

Default: random.

--fp16

Use fp16 computations.

Default: False.

--fp16-impl

Implementation of FP16 to use

Choices: safe, mem_efficient.

Default: safe.

--rank-candidates, --rc

Whether the model should parse candidates for ranking.

Default: False.

--truncate, --tr

Truncate input lengths to increase speed / use less memory.

Default: -1.

--text-truncate

Text input truncation length: if not specified, this will default to truncate

Default: 300.

--label-truncate

Label truncation length: if not specified, this will default to truncate

Default: 300.

--history-reversed

Reverse the history

Default: False.

--history-size, --histsz

Number of past dialog utterances to remember.

Default: -1.

--person-tokens, --pt

Add person tokens to history. adds p1 in front of input text and p2 in front of past labels when available or past utterances generated by the model. these are added to the dictionary during initialization.

Default: False.

--split-lines

Split the dialogue history on newlines and save in separate vectors

Default: False.

--delimiter

Join history lines with this token, defaults to newline

Default: \n.

--special-tok-lst

Comma separated list of special tokens. In case of ambiguous parses from special tokens, the ordering provided in this arg sets precedence.

-gpu, --gpu

Which GPU to use

Default: -1.

--no-cuda

Disable GPUs even if available. otherwise, will use GPUs if available on the device.

Default: False.

Optimizer Arguments

Argument

Description

--optimizer, --opt

Optimizer choice. Possible values: adadelta, adagrad, adam, adamw, sparseadam, adamax, asgd, sgd, radam, rprop, rmsprop, optimizer, nadam, lbfgs, mem_eff_adam, adafactor.

Choices: adadelta, adagrad, adam, adamw, sparseadam, adamax, asgd, sgd, radam, rprop, rmsprop, optimizer, nadam, lbfgs, mem_eff_adam, adafactor.

Default: sgd.

--learningrate, --lr

Learning rate

Default: 5e-05.

--gradient-clip, --clip

Gradient clipping using l2 norm

Default: 0.1.

--adafactor-eps

Epsilon values for adafactor optimizer: regularization constants for square gradient and parameter scale respectively

Default: 1e-30,1e-3. Recommended: 1e-30,1e-3.

--momentum, --mom

If applicable, momentum value for optimizer.

Default: 0.

--nesterov

If applicable, whether to use nesterov momentum.

Default: True.

--nus, --nu

If applicable, nu value(s) for optimizer. can use a single value like 0.7 or a comma-separated tuple like 0.7,1.0

Default: 0.7.

--betas, --beta

If applicable, beta value(s) for optimizer. can use a single value like 0.9 or a comma-separated tuple like 0.9,0.999

Default: 0.9,0.999.

--weight-decay, --wdecay

Weight decay on the weights.

BPEHelper Arguments

Argument

Description

--bpe-vocab

Path to pre-trained tokenizer vocab

--bpe-merge

Path to pre-trained tokenizer merge

--bpe-dropout

Use BPE dropout during training.

Learning Rate Scheduler

Argument

Description

--lr-scheduler

Learning rate scheduler.

Choices: reduceonplateau, none, fixed, invsqrt, cosine, linear.

Default: reduceonplateau.

--lr-scheduler-patience

LR scheduler patience. In number of validation runs. If using fixed scheduler, LR is decayed every validations.

Default: 3.

--lr-scheduler-decay

Decay factor for LR scheduler, or how much LR is multiplied by when it is lowered.

Default: 0.5.

--invsqrt-lr-decay-gamma

Constant used only to find the lr multiplier for the invsqrt scheduler. Must be set for –lr-scheduler invsqrt

Default: -1.

TorchRankerAgent

Argument

Description

--candidates, --cands

The source of candidates during training (see TorchRankerAgent._build_candidates() for details).

Choices: batch, inline, fixed, batch-all-cands.

Default: batch.

--eval-candidates, --ecands

The source of candidates during evaluation (defaults to the samevalue as –candidates if no flag is given)

Choices: batch, inline, fixed, vocab, batch-all-cands.

Default: inline.

--interactive-candidates, --icands

The source of candidates during interactive mode. Since in interactive mode, batchsize == 1, we cannot use batch candidates.

Choices: fixed, inline, vocab.

Default: fixed.

--repeat-blocking-heuristic

Block repeating previous utterances. Helpful for many models that score repeats highly, so switched on by default.

Default: True.

--fixed-candidates-path, --fcp

A text file of fixed candidates to use for all examples, one candidate per line

--fixed-candidate-vecs

One of “reuse”, “replace”, or a path to a file with vectors corresponding to the candidates at –fixed-candidates-path. The default path is a /path/to/model-file.<cands_name>, where <cands_name> is the name of the file (not the full path) passed by the flag –fixed-candidates-path. By default, this file is created once and reused. To replace it, use the “replace” option.

Default: reuse.

--encode-candidate-vecs

Cache and save the encoding of the candidate vecs. This might be used when interacting with the model in real time or evaluating on fixed candidate set when the encoding of the candidates is independent of the input.

Default: True.

--init-model

Initialize model with weights from this file.

--train-predict

Get predictions and calculate mean rank during the train step. Turning this on may slow down training.

Default: False.

--cap-num-predictions

Limit to the number of predictions in output.text_candidates

Default: 100.

--ignore-bad-candidates

Ignore examples for which the label is not present in the label candidates. Default behavior results in RuntimeError.

Default: False.

--rank-top-k

Ranking returns the top k results of k > 0, otherwise sorts every single candidate according to the ranking.

Default: -1.

--inference

Final response output algorithm

Choices: max, topk.

Default: max.

--topk

K used in Top K sampling inference, when selected

Default: 5.

--return-cand-scores

Return sorted candidate scores from eval_step

Default: False.

Bert Ranker Arguments

Argument

Description

--add-transformer-layer

Also add a transformer layer on top of Bert

Default: False.

--pull-from-layer

Which layer of Bert do we use? Default=-1=last one.

Default: -1.

--out-dim

For biencoder, output dimension

Default: 768.

--topn

For the biencoder: select how many elements to return

Default: 10.

--data-parallel

Use model in data parallel, requires multiple gpus. NOTE This is incompatible with distributed training

Default: False.

--bert-aggregation

How do we transform a list of output into one

Choices: first, max, mean.

Default: first.

CrossEncoderRankerAgent Options

TorchAgent Arguments

Argument

Description

--interactive-mode, --i

Whether in full interactive mode or not, which means generating text or retrieving from a full set of candidates, which is necessary to actually do full dialogue. However, during training or quick validation (e.g. PPL for generation or ranking a few candidates for ranking models) you might want these set to off. Typically, scripts can set their preferred default behavior at the start, e.g. eval scripts.

Default: False.

--embedding-type, --emb

Choose between different strategies for initializing word embeddings. Default is random, but can also preinitialize from Glove or Fasttext. Preinitialized embeddings can also be fixed so they are not updated during training.

Choices: random, glove, glove-fixed, fasttext, fasttext-fixed, fasttext_cc, fasttext_cc-fixed.

Default: random.

--embedding-projection, --embp

If pretrained embeddings have a different dimensionality than your embedding size, strategy for projecting to the correct size. If the dimensions are the same, this is ignored unless you append “-force” to your choice.

Default: random.

--fp16

Use fp16 computations.

Default: False.

--fp16-impl

Implementation of FP16 to use

Choices: safe, mem_efficient.

Default: safe.

--rank-candidates, --rc

Whether the model should parse candidates for ranking.

Default: False.

--truncate, --tr

Truncate input lengths to increase speed / use less memory.

Default: -1.

--text-truncate

Text input truncation length: if not specified, this will default to truncate

Default: 300.

--label-truncate

Label truncation length: if not specified, this will default to truncate

Default: 300.

--history-reversed

Reverse the history

Default: False.

--history-size, --histsz

Number of past dialog utterances to remember.

Default: -1.

--person-tokens, --pt

Add person tokens to history. adds p1 in front of input text and p2 in front of past labels when available or past utterances generated by the model. these are added to the dictionary during initialization.

Default: False.

--split-lines

Split the dialogue history on newlines and save in separate vectors

Default: False.

--delimiter

Join history lines with this token, defaults to newline

Default: \n.

--special-tok-lst

Comma separated list of special tokens. In case of ambiguous parses from special tokens, the ordering provided in this arg sets precedence.

-gpu, --gpu

Which GPU to use

Default: -1.

--no-cuda

Disable GPUs even if available. otherwise, will use GPUs if available on the device.

Default: False.

Optimizer Arguments

Argument

Description

--optimizer, --opt

Optimizer choice. Possible values: adadelta, adagrad, adam, adamw, sparseadam, adamax, asgd, sgd, radam, rprop, rmsprop, optimizer, nadam, lbfgs, mem_eff_adam, adafactor.

Choices: adadelta, adagrad, adam, adamw, sparseadam, adamax, asgd, sgd, radam, rprop, rmsprop, optimizer, nadam, lbfgs, mem_eff_adam, adafactor.

Default: sgd.

--learningrate, --lr

Learning rate

Default: 5e-05.

--gradient-clip, --clip

Gradient clipping using l2 norm

Default: 0.1.

--adafactor-eps

Epsilon values for adafactor optimizer: regularization constants for square gradient and parameter scale respectively

Default: 1e-30,1e-3. Recommended: 1e-30,1e-3.

--momentum, --mom

If applicable, momentum value for optimizer.

Default: 0.

--nesterov

If applicable, whether to use nesterov momentum.

Default: True.

--nus, --nu

If applicable, nu value(s) for optimizer. can use a single value like 0.7 or a comma-separated tuple like 0.7,1.0

Default: 0.7.

--betas, --beta

If applicable, beta value(s) for optimizer. can use a single value like 0.9 or a comma-separated tuple like 0.9,0.999

Default: 0.9,0.999.

--weight-decay, --wdecay

Weight decay on the weights.

BPEHelper Arguments

Argument

Description

--bpe-vocab

Path to pre-trained tokenizer vocab

--bpe-merge

Path to pre-trained tokenizer merge

--bpe-dropout

Use BPE dropout during training.

Learning Rate Scheduler

Argument

Description

--lr-scheduler

Learning rate scheduler.

Choices: reduceonplateau, none, fixed, invsqrt, cosine, linear.

Default: reduceonplateau.

--lr-scheduler-patience

LR scheduler patience. In number of validation runs. If using fixed scheduler, LR is decayed every validations.

Default: 3.

--lr-scheduler-decay

Decay factor for LR scheduler, or how much LR is multiplied by when it is lowered.

Default: 0.5.

--invsqrt-lr-decay-gamma

Constant used only to find the lr multiplier for the invsqrt scheduler. Must be set for –lr-scheduler invsqrt

Default: -1.

TorchRankerAgent

Argument

Description

--candidates, --cands

The source of candidates during training (see TorchRankerAgent._build_candidates() for details).

Choices: batch, inline, fixed, batch-all-cands.

Default: inline.

--eval-candidates, --ecands

The source of candidates during evaluation (defaults to the samevalue as –candidates if no flag is given)

Choices: batch, inline, fixed, vocab, batch-all-cands.

Default: inline.

--interactive-candidates, --icands

The source of candidates during interactive mode. Since in interactive mode, batchsize == 1, we cannot use batch candidates.

Choices: fixed, inline, vocab.

Default: fixed.

--repeat-blocking-heuristic

Block repeating previous utterances. Helpful for many models that score repeats highly, so switched on by default.

Default: True.

--fixed-candidates-path, --fcp

A text file of fixed candidates to use for all examples, one candidate per line

--fixed-candidate-vecs

One of “reuse”, “replace”, or a path to a file with vectors corresponding to the candidates at –fixed-candidates-path. The default path is a /path/to/model-file.<cands_name>, where <cands_name> is the name of the file (not the full path) passed by the flag –fixed-candidates-path. By default, this file is created once and reused. To replace it, use the “replace” option.

Default: reuse.

--encode-candidate-vecs

Cache and save the encoding of the candidate vecs. This might be used when interacting with the model in real time or evaluating on fixed candidate set when the encoding of the candidates is independent of the input.

Default: True.

--init-model

Initialize model with weights from this file.

--train-predict

Get predictions and calculate mean rank during the train step. Turning this on may slow down training.

Default: False.

--cap-num-predictions

Limit to the number of predictions in output.text_candidates

Default: 100.

--ignore-bad-candidates

Ignore examples for which the label is not present in the label candidates. Default behavior results in RuntimeError.

Default: False.

--rank-top-k

Ranking returns the top k results of k > 0, otherwise sorts every single candidate according to the ranking.

Default: -1.

--inference

Final response output algorithm

Choices: max, topk.

Default: max.

--topk

K used in Top K sampling inference, when selected

Default: 5.

--return-cand-scores

Return sorted candidate scores from eval_step

Default: False.

Bert Ranker Arguments

Argument

Description

--add-transformer-layer

Also add a transformer layer on top of Bert

Default: False.

--pull-from-layer

Which layer of Bert do we use? Default=-1=last one.

Default: -1.

--out-dim

For biencoder, output dimension

Default: 768.

--topn

For the biencoder: select how many elements to return

Default: 10.

--data-parallel

Use model in data parallel, requires multiple gpus. NOTE This is incompatible with distributed training

Default: False.

--bert-aggregation

How do we transform a list of output into one

Choices: first, max, mean.

Default: first.