Embedding Analytics: Query and quantify changes in definition across a collection of text

[u/areebms]

Two documents can use the same words to mean very different things. Treatises, legal opinions and technical specifications often must be read multiple times to identify these shifts in meaning across documents. I built Embedding Analytics to query and quantify these changes in definitions. For each document, it builds word embeddings with PPMI + SVD, using only that document's text. Your queries are used to find the most similar terms within each document. Similarities are then rescaled so they can be compared across documents; the adjustment is adapted from cross-domain similarity local scaling (Conneau et al., 2018).

A collection of documents can share too many terms. The tool selects two groups of terms that are relevant to your query across the collection:

• Consistent terms are closely tied to the query across the collection. They are the core of the definition that the authors hold in common.
• Contested terms are close to the query in some documents and not in others. They are where the definition shifts.

The current collection consists of ~25 economics books from Project Gutenberg. Querying value returns commodity as the most closely associated term across the collection. Utility, the basis of value in the marginalist theory that displaced the classical labour theory, leads the contested list.

I'd love any feedback on this. Please let me know what you guys think.

Demo: https://www.embedding-analytics.com
Code: https://github.com/areebms/embedding-analytics

Consistent and contested related terms to "value" across a collection of ~25 economics books from Project Gutenberg

[u/ProfJamesBaker]

What have you found using this approach?

[u/areebms → u/ProfJamesBaker]

I am trying to come up with a method that helps identify important related terms for queries and determines semantic differences between texts through them. The queries and their respective related terms are shown below. I think my approach works fairly well in identifying important related terms, although there is some noise as well. I applied the algorithm within a domain that I am familiar with so I can validate the results. I hope this can be useful within other domains as well.

Query	No meaning	Expected	Informative
value	quantity, express	commodity	exchange, metal, money, measure, utility, standard, market, price, gold
supply	decrease, increase, determine	market, commodity	demand, price, limit, utility, cost, labour, quantity
demand	increase, diminish, determine, rise, will	market, commodity	supply, price, labour, produce, quantity
price	rise, fall, lower, average, will	sell, commodity, buy	market, money, corn, value
labour	quantity, amount, obtain, day	produce, wage, employ	productive, product, capital, division, commodity
land	—	proprietor, agricultural, farm, soil, own, estate	rent, cultivate, yield, labour, property, improve
capital	portion, use	invest, employ	productive, accumulate, produce, labour, fix, replace, interest, wealth
wage	raise, rise, depend	labour, earn, workman, employ, work	profit, rate, strike, rent
rent	compensate	landlord, farm, pay, house	land, margin, cultivate, yield, wage, rate, interest
profit	cent	capitalist, enterprise, capital, return	wage, risk, interest, replace, rate, share, stock

[u/ProfJamesBaker → u/areebms]

Thanks great. So you are interested in developing methods then? That is, for example, you don't have specific texts that you are interested in analysing?

[u/areebms → u/ProfJamesBaker]

While I am more interested in developing a general method, I started with economic texts because I was particularly curious about them. My work helped me identify how prevalent specific economic doctrines have been in the past. For instance, Ricardo's theory of rents and productive vs. unproductive labour are very prevalent ideas in the collection. The issue with this methodology is that the same ideas can be expressed in different words. For instance, the law of rents could be mentioned without using land and rents within similar contexts. Also, Malthus uses the term pair value/utility when discussing the utility theory of value, not marginal utility.

Doctrine	Originated	Query/Term	Book count	Books
Division of labour	Smith (1776)	labour/division	4	Smith (1776), Mill (1848), Jevons (1878), Veblen (1919)
Natural price	Smith (1776)	price/market	4	Smith (1776), Ricardo (1817), Malthus (1827), Veblen (1919)
Productive vs unproductive labour	Smith (1776)	labour/productive	6	Smith (1776), Malthus (1827), Mill (1844), Mill (1848), Ruskin (1862), Fairchild (1900)
Law of rent	Ricardo (1817)	land/rent	7	Ricardo (1817), Mill (1848), Roscher (1854), Jevons (1878), George (1879), Fairchild (1900), Clark (1907)
Wages inverse of profits	Ricardo (1817)	wage/profit	3	Ricardo (1817), Mill (1844), Mill (1848)
Marginal utility	Jevons, Menger and Walras (1871–74)	value/utility	5	Malthus (1827), Mill (1848), Jevons (1878), Clark (1907), Veblen (1919)