Evaluation metric

Could you give us some more info about the evaluation metric?

1 Like

The evaluation metric is Levenshtein Distance. There are lots of great articles about how this equation works. Here’s a link to a nice introduction:

Thanks for the reference. My question though is not on the understanding of the distance.

In the competition description I read:
“Submissions will be scored against the test answer key using a Levenshtein distance. A higher score is considered more successful.”

The problem is that a perfect match would produce a distance of 0 whereas if I submit the template submission I get a score of zero. Moreover, you say that a higher score is better in contrast to the distance definition.

My guess is, that you somehow modified the distance and I would like to know what is exactly the equation that you are using so that I can evaluate the performance of my solutions locally.

The final score utilizes the Levenshtein ratio and is applied to a 10-point scale.

    score = (1 - leven/length) * 10

Which returns a score between 0 and 10. This scoring algorithm also prevents negative scores.


Great, thanks for the clarification!