Abstract
The ability to rank creative natural language provides an im-
portant general tool for downstream language understanding
and generation. However, current deep ranking models require
substantial amounts of labeled data that are difficult and expen-
sive to obtain for new domains, languages and creative charac-
teristics. A recent neural approach, DirectRanker, reduces the
amount of training data needed but has not previously been
used to rank creative text. We therefore adapt DirectRanker
to provide a new deep model for ranking creative language
with small numbers of training instances, and compare it with
a Bayesian approach, Gaussian process preference learning
(GPPL), which was previously shown to work well with sparse
data. Our experiments with short creative language texts show
the effectiveness of DirectRanker even with small training
datasets. Combining DirectRanker with GPPL outperforms
the previous state of the art on humor and metaphor novelty
tasks, increasing Spearman’s ρ by 25% and 29% on average.
Furthermore, we provide a possible application to validate
jokes in the process of creativity generation.
portant general tool for downstream language understanding
and generation. However, current deep ranking models require
substantial amounts of labeled data that are difficult and expen-
sive to obtain for new domains, languages and creative charac-
teristics. A recent neural approach, DirectRanker, reduces the
amount of training data needed but has not previously been
used to rank creative text. We therefore adapt DirectRanker
to provide a new deep model for ranking creative language
with small numbers of training instances, and compare it with
a Bayesian approach, Gaussian process preference learning
(GPPL), which was previously shown to work well with sparse
data. Our experiments with short creative language texts show
the effectiveness of DirectRanker even with small training
datasets. Combining DirectRanker with GPPL outperforms
the previous state of the art on humor and metaphor novelty
tasks, increasing Spearman’s ρ by 25% and 29% on average.
Furthermore, we provide a possible application to validate
jokes in the process of creativity generation.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 13th International Conference on Computational Creativity, ICCC’22 |
| Publication status | Published - 2022 |