I’m always slightly wary of people who describe themselves as experts in Search Engine Optimisation (SEO). Partly this is because it seems like an ephemeral area of knowledge: the only true fundamental principles are obvious generalities, and little of any substance will still apply 2 years after you learn it. Partly it’s because SEO is a zero-sum game: only one site can have the top rank, and time spent fighting your competitors for SEO is time you could have spent writing good content.
Furthermore, SEO isn’t hard because it’s a conceptually hard task (which would have intellectual respectability), it’s hard because Google and others have deliberately made it hard. The search engines publish far less information about their algorithms than they could do, because security by obscurity is much easier than creating an algorithm that can’t be subverted.
A further thought occurred to me more recently, based on the idea of “rent exhaustion”. This is a term that has been thrown around on a few blogs (notably Tim Harford’s and Tyler Cowen’s) and is allegedly a mainstream term in economics, though I haven’t found any authoritative definitions of it. In any case, the example usually given is this: handing out money to beggars doesn’t increase the overall wealth of the community of beggars, it just causes them to waste time and effort crowding round for handouts. Assuming a beggar’s time isn’t worthless, any time they spend queuing for a donation represents a net loss to society.
I wonder whether SEO doesn’t represent just such a waste of resources. The “handout” in search engine terms is the large slice of customers’ cash that goes fairly indiscriminately to whoever is at the top of the Google rankings. The more customers there are, the more effort is expended on scrabbling for their attention. Customers aren’t being served by better products at keener prices, and they’re not even benefitting from attractive and exciting marketing (which we may enjoy more than we care to admit).
Is there a way to resolve this? A perfect search engine could distinguish good content from spam, but doing this is probably a strong AI problem. Perhaps search consumers can make the world a better place by venturing beyond the first result page more often, and critically evaluating the sites they are offered, but it’s not clear that this highly-duplicated effort doesn’t represent a waste of resources in itself. Ultimately a solution might be to pool the opinions of savvy consumers to contribute to the ranking, which was of course the original goal of using link data as a proxy for recommendation. Until we solve the problem of identity on the internet, any such system will be exposed to manipulation, leaving the opinions of real consumers a vastly under-utilised resource.