#Spelling corrector in r code#
The actual writing of the spellcheck code and testing/fiddling with it took perhaps a normal work day. All told it took me a couple of days to get very clean and well tested code with a complete API to present this and glue it into multiple components. You have covered almost all of the common scenarios at this point as well. No matter how clever your solution you won't get more than a few percent extra for a lot of extra work. The law of diminishing returns kicks in here. Its obvious that approaching this with a more rigorous model in mind also helped his code come out cleaner than mine did. It takes a little work to represent all of the data structures in Java. I think this followed with Norvig's probability model could get you even closer with just a little extra effort. The algorithms for calculating phonetic representation help make a kind of best guess when people are truly guessing at a spelling or they know they can get it "close". I just took a few ideas and a couple of well known algorithms and ideas and built around that. I must say using phonetic representations of words in spellcheck is also a well known thing, though. This approach would catch things that were edit distance 4, 5, and even 6 off with no problem. Then I would just pull up all of the values from my two sets of phonetic representations to see what was in there. So then in addition to the four basic operations I would also compute the approximate phonetic representation of each misspelled word. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Using Soundex and some other algorithm that escapes me as keys to a set. A tag already exists with the provided branch name. Essentially for every word in my dictionary I computed a couple of different phonetic representations of the words. However, given that this was a server side application that had the memory to spare no worries. I took a different approach that achieved better accuracy at the cost of more memory. His correct function is the real insight that gives his program such short length with his error rate, and where I will agree his solution is pretty clever. It takes just a day or two to implement this in a well tested fashion.
It was easy to understand Java with relatively little code. If you think about it that part is easy His code is nice and idiomatic python, but I had something similar with (minus boilerplate) probably 50-75 lines of Java (final version was more like 500 lines of Java). That is the simple approach and it gets you a long ways towards having an effective suggestion engine.
With each operation you check against your known dictionary and for each one you find to be a known word, voila it is a spelling suggestion. The operations he performs (transpose, delete, alter, insert) are well known as as the basic things to do. I wrote a spell check that uses the Levenshtein distance (edit distance) algorithm. Actually with a little research this is not a difficult problem as you might think.
0 kommentar(er)
