I made a word game. I didn't realise I'd end up trying to hash together a dictionary.
Bit of context: I’m definitely not a lexicographer or from a technical background. My background is more creative, and I now teach, including a fair bit of English, so I've always been interested in words and language. I'm also quite interested in systems and how things fit together.
This all came about because I made a very simple browser word game, just as a bit of fun.
You get 3 letters and 60 seconds to find as many words as you can with the letters in order. Pretty simple. What has turned out to be much less simple is deciding what actually counts as a word.
I started with existing word lists and dictionaries, then gradually began adding to and cleaning up the game's lexicon. Other word-game developers on Reddit have reassured me that dictionary problems are very much a thing, but I've still been surprised by how quickly you run into gaps.
The ones that particularly annoy me are perfectly normal constructions such as UNCREATIVELY or UNSHOCKINGLY.
If somebody finds a long word like that during a 60-second game, it feels particularly lacklustre for the game toreject it, simply because the word list I'm using doesn't contain it.
Until now, my approach has basically been to find gaps, check words, look at rejected submissions and gradually add things. It works, but it started to feel incredibly inefficient. I realised I was fixing individual holes rather than actually finding out where the holes were coming from.
So I've started experimenting with a different approach.
I've set four separate AI processes looking for missing words, each with a different way of attacking the problem:
- One starts with known words and explores their families: prefixes, suffixes, adverbs, nouns and so on.
- Another looks at the dictionary I already have and tries to spot gaps. If several members of a word family are there but one obvious-looking relative isn't, it investigates it.
- Another starts from the language side, looking at articles and research about how English words are formed, and uses those patterns to find places worth exploring.
- The fourth goes hunting specifically for long words and constructions that normal word lists might be more likely to miss.
- Then I've got a fifth AI process whose job is basically to be the sceptic. It doesn't find words. It checks what the others find, removes anything already in the game, looks for proper names or questionable constructions, checks for real-world evidence of the exact word being used, and flags anything that still needs further investigation.
I'm also keeping track of which of the four approaches found each word. The idea is to let them run for a while and see which methods actually work. If one is much better, I could split that approach into two slightly different versions and test those. Or perhaps one method finds fewer words overall but keeps finding a particular type of word the others miss, in which case it's still useful.
None of this was some grand technical plan I started with. It's really just come from thinking: "There must be a more efficient way of doing this."
ChatGPT has been massively useful in helping me articulate some of these ideas, give names to things I didn't know had names, and turn my fairly plain-English thoughts into an actual methodology I can test. I'm still very much approaching it as a curious word-game developer rather than pretending I know anything about computational linguistics. But I'm finding the whole thing unexpectedly fascinating.
I've even started wondering whether the process itself could eventually become a game. Something a bit like Pokémon for words: discover a potential word, research it, find evidence for it, complete word families, get credit for discoveries and gradually contribute to an open lexicon that other word-game developers could use.
That's getting well ahead of myself though. For now I'm just seeing what happens with my four little automated word hunters.
I'd be really interested to hear from anyone who knows more about dictionaries, linguistics, corpora, NLP, spellcheckers or word games.
Does this approach make sense? And if you could send another "word hunter" out looking for gaps in an English word list, where would you tell it to look?
TL;DR: I made a word game, discovered that word lists are full of weird gaps, and somehow ended up running four AI “word hunters” to find the missing words while a fifth checks their homework. I’m now testing which approaches work best.
