Please use this identifier to cite or link to this item:
https://hdl.handle.net/10356/151218
Full metadata record
DC Field | Value | Language |
---|---|---|
dc.contributor.author | D'Haro, Luis Fernando | en_US |
dc.contributor.author | Banchs, Rafael E. | en_US |
dc.contributor.author | Hori, Chiori | en_US |
dc.contributor.author | Li, Haizhou | en_US |
dc.date.accessioned | 2021-07-02T03:31:40Z | - |
dc.date.available | 2021-07-02T03:31:40Z | - |
dc.date.issued | 2018 | - |
dc.identifier.citation | D'Haro, L. F., Banchs, R. E., Hori, C. & Li, H. (2018). Automatic evaluation of end-to-end dialog systems with adequacy-fluency metrics. Computer Speech and Language, 55, 200-215. https://dx.doi.org/10.1016/j.csl.2018.12.004 | en_US |
dc.identifier.issn | 0885-2308 | en_US |
dc.identifier.other | 0000-0002-4201-7578 | - |
dc.identifier.uri | https://hdl.handle.net/10356/151218 | - |
dc.description.abstract | End-to-end dialog systems are gaining interest due to the recent advances of deep neural networks and the availability of large human–human dialog corpora. However, in spite of being of fundamental importance to systematically improve the performance of this kind of systems, automatic evaluation of the generated dialog utterances is still an unsolved problem. Indeed, most of the proposed objective metrics shown low correlation with human evaluations. In this paper, we evaluate a two-dimensional evaluation metric that is designed to operate at sentence level, which considers the syntactic and semantic information carried along the answers generated by an end-to-end dialog system with respect to a set of references. The proposed metric, when applied to outputs generated by the systems participating in track 2 of the DSTC-6 challenge, shows a higher correlation with human evaluations (up to 12.8% relative improvement at the system level) than the best of the alternative state-of-the-art automatic metrics currently available. | en_US |
dc.language.iso | en | en_US |
dc.relation.ispartof | Computer Speech and Language | en_US |
dc.rights | © 2018 Elsevier Ltd. All rights reserved. | en_US |
dc.subject | Engineering::Computer science and engineering | en_US |
dc.title | Automatic evaluation of end-to-end dialog systems with adequacy-fluency metrics | en_US |
dc.type | Journal Article | en |
dc.contributor.school | School of Computer Science and Engineering | en_US |
dc.identifier.doi | 10.1016/j.csl.2018.12.004 | - |
dc.identifier.scopus | 2-s2.0-85059347815 | - |
dc.identifier.volume | 55 | en_US |
dc.identifier.spage | 200 | en_US |
dc.identifier.epage | 215 | en_US |
dc.subject.keywords | Automatic Evaluation Metrics | en_US |
dc.subject.keywords | Dialog Systems | en_US |
item.fulltext | No Fulltext | - |
item.grantfulltext | none | - |
Appears in Collections: | SCSE Journal Articles |
SCOPUSTM
Citations
20
16
Updated on Mar 21, 2024
Web of ScienceTM
Citations
20
11
Updated on Oct 28, 2023
Page view(s)
177
Updated on Mar 28, 2024
Google ScholarTM
Check
Altmetric
Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.