Tianyi Zhang and colleagues, 2019

“We propose BERTScore, an automatic evaluation metric for text generation.”

Tianyi Zhang and colleagues wrote this while addressing evaluating generated text when exact word overlap misses semantically similar phrasing. The research matched candidate and reference tokens using contextual embedding similarity. In the original BERTScore setting, the claim is bounded by reported data and experimental conditions. That specificity matters because embedding similarity can reward plausible wording while overlooking factual errors, missing information, bias, or unsafe advice.

For BERTScore, the quotation matters because the method offered a semantic complement to surface-form metrics and correlated more strongly with human judgments in studied settings. By explaining that the work matched candidate and reference tokens using contextual embedding similarity, the authors made their design testable. Later studies can compare alternatives for evaluating generated text when exact word overlap misses semantically similar phrasing and ask whether the evidence transfers to comparing text-generation systems with several metrics and targeted human review.

Applied today, BERTScore remains useful for comparing text-generation systems with several metrics and targeted human review. Responsible teams should define intended users, a baseline for evaluating generated text when exact word overlap misses semantically similar phrasing, and a rollback signal before deployment. Because embedding similarity can reward plausible wording while overlooking factual errors, missing information, bias, or unsafe advice, evaluation also needs rare-case tests, ongoing monitoring, an appeal path for affected people, and named ownership for correction.

The central caution is that embedding similarity can reward plausible wording while overlooking factual errors, missing information, bias, or unsafe advice. For BERTScore, the result supports the finding that the method offered a semantic complement to surface-form metrics and correlated more strongly with human judgments in studied settings, but not every possible use. Deployment should include independent review, incident reporting, and a practical way to withdraw the system. The sound reading preserves the contribution while leaving room to revise it when real-world evidence disagrees. The BERTScore quotation endures because matched candidate and reference tokens using contextual embedding similarity offers a testable response to evaluating generated text when exact word overlap misses semantically similar phrasing. Readers considering comparing text-generation systems with several metrics and targeted human review should reproduce the result, publish negative cases, and compare simpler baselines. They should also explain that embedding similarity can reward plausible wording while overlooking factual errors, missing information, bias, or unsafe advice, assign responsibility, and prepare a correction path. Continued value for BERTScore depends on transparent evidence, careful limits, and correction when new conditions expose failure. That balance lets later work build on the contribution without erasing the limits that make its evidence interpretable.