Creating a column of match probabilities from hmni package python

I have a dataframe that looks like this

import pandas as pd
from pandas import DataFrame
df = pd.DataFrame({'CEOThisYr': ['Douglas Davis', 'Doug Davis', 'James Taylor', 'Jane Smith'], ['CEOLastYr': 'Doug Davis', 'James Taylor', 'Jane Smith', 'Sarah Jones']})

My goal is to use the hmni package to create a new column with the match probability scores of each name across rows. My attempted code is:

import hmni
matcher = hmni.Matcher(model='latin')
df['MatchPercent'] = matcher.similarity(df['CEOThisYr'], df['CEOLastYr'])

That returns the error: TypeError: Only string comparison is supported in similarity method

I've tried converting both columns to string just to be sure but that still returns the same error. Any ideas where I'm going wrong?

Update*

Thanks to the helpful comment below, I was able to come up with

import hmni
matcher = hmni.Matcher(model='latin')
df['MatchPercent'] = df.apply(lambda x: matcher.similarity(x['CEOThisYr'], x['CEOLastYr']), axis=1)

I went from 9 seconds to process 1 row to processing approx 5000 rows per second.

1 Answer

According to hmni's docs, similarity accepts twos strs as its first and second arguments. You are trying to pass two pandas.Series, i.e., df['CEOThisYr'] and df['CEOLastYr']. You could try using pandas.DataFrame.apply to apply similarity to each row.

>>> import hmni
>>> import pandas as pd
>>>
>>> df = pd.DataFrame({'CEOThisYr': ['Douglas Davis', 'Doug Davis', 'James Taylor', 'Jane Smith'], 'CEOLastYr': ['Doug Davis', 'James Taylor', 'Jane Smith', 'Sarah Jones']})
>>> df CEOThisYr CEOLastYr
0 Douglas Davis Doug Davis
1 Doug Davis James Taylor
2 James Taylor Jane Smith
3 Jane Smith Sarah Jones
>>>
>>> matcher = hmni.Matcher(model='latin')
>>> df['MatchPercent'] = df.apply(lambda x: matcher.similarity(x['CEOThisYr'], x['CEOLastYr']), axis=1)
>>> df CEOThisYr CEOLastYr MatchPercent
0 Douglas Davis Doug Davis 0.922682
1 Doug Davis James Taylor 0.000000
2 James Taylor Jane Smith 0.000000
3 Jane Smith Sarah Jones 0.000000