IR Baseline

This agent is a simple information retrieval baseline. If label_candidates are provided in the task, it simply ranks them according to their similarity to the input message. To use TF-IDF similarity, you must build a dictionary and “train” on the train set first. See the examples below for clarification.

Basic Examples

Evaluate the IR baseline model (without using TF-IDF) on the Persona-Chat task.

python examples/ -m ir_baseline -t personachat

Build a dictionary on the Persona-Chat task, and then evaluate the model using TF-IDF.

python examples/ -m ir_baseline -t personachat --dict-file /tmp/personachat.dict -ttim 5

DictionaryAgent Options

BPEHelper Arguments


Path to pre-trained tokenizer vocab


Path to pre-trained tokenizer merge

IrBaselineAgent Options

IrBaseline Arguments

-lp, --length-penalty

Length penalty for responses

Default: 0.5.

-hsz, --history-size

Number of utterances from the dialogue history to take use as the query

Default: 1.


File of candidate responses to choose from