|
By Nigel Hamilton
Humans are great at discovering new things, but not always so good at
remembering how we found them. Since the first caveman discovered
fire, the knowledge has been passed down through the ages. In a way
we've all been touched by that first flame and the idea of it is still
burning.
Fire was a fantastic discovery, but human language is our best
discovery yet as a way of infecting others with new ideas. With
language we can pass more than DNA on to the next generation: we can
pass on discoveries too.
But spoken language is ephemeral ('Sorry, how do I make it spark
again? I can't hear properly; The Neanderthals are shouting.') So
humankind came up with ways of fossilizing our ideas into written
symbols on various media: cave walls, stone tablets, papyrus scrolls,
books and more recently 1GB memory sticks.
Bringing order to chaos
The Internet is a massive archaeological pile of fossilized ideas. But
armed with an average query of 2.5 keywords and Google, we can still
dive into the pile and come out relatively unscathed. But for how
long? The pile of information is growing and growing. According to the
World Wide Web Consortium (W3C), there are 1,173 million Internet
users as of June 2007, compared with 938 million two years ago.
Currently Google only crawls part of the pile. Vast tracts of valuable
information reside in topic-specific search engines out of the Google
crawler's reach.
The Google PageRank algorithm <http://www.google.com/technology/>
tries to bring order to some of the pile by ranking Web pages:
'Google provides its views on pages' relative importance,' according
to the Google website. But the polarity of PageRank is still skewed
in favour of the publishers. PageRank is conferred by Web publishers
to other Web publishers in the form of referral links - not by users.
Consequently Web publishers, ably assisted by an army of search engine
optimisers (SEOs), are playing Google's PageRank for profit. The
raison d'etre of SEOs is to create optimally relevant pages tuned to
the Google algorithm so their clients' pages appear high in Google's
results. Although these pages appear relevant they lack a crucial
ingredient: authority.
It was 'authority' conferred by referral links that propelled Google's
PageRank to be the premier retrieval algorithm for the Web, and it is
authority that is ebbing away as the SEOs and Web publishers take
control of their position in Google's results by manipulating
PageRank.
Power to the people
We need a system of Darwinian information selection where the users,
not publishers, decide the best answer for a given query. Shouldn't
users be given more authority in deciding what is, or isn't relevant?
Social search provides this by determining the relevance of search
results in accordance with actual usage.
Every day millions of people search the Web and apply human intellect
to making search discoveries, yet this effort is mostly wasted. The
pile grows and grows but remains untouched by the humans sifting
through it. The promise of social search is to harness this communal
effort for the good of all. But how can we bring human order to the
pile?
I've been wrestling with this question for the past six years and I
believe the answer lies in the prescient vision of Dr Vannevar Bush in
his 1945 seminal paper "As We May Think"
<http://www.theatlantic.com/doc/194507/bush>. Bush describes a machine
called a Memex that augments your memory and searching powers by
helping you to create and share 'trails of association' between things
in the 'common record' (analogous to the Internet).
Back in 1945 this concept must have seemed wild but Bush successfully
predicted the future by directly influencing it. Bush inspired Ted
Nelson (hypertext), Sir Tim Berners-Lee (WWW) and many others to
realise his vision of an 'interlinked common record'. However, there
are still major parts of his functional specification missing.
Trailblazing, for example, is a crucial, yet largely overlooked part
of Bush's invention:
'There is a new profession of trail blazers, those who find delight in
the task of establishing useful trails through the enormous mass of
the common record. The inheritance from the master becomes, not only
his additions to the world's record, but for his disciples the entire
scaffolding by which they are erected.'
Trailblazing
Every day millions of people blaze search trails into the pile but
most of that search effort just goes up in smoke.
How can all this search effort be harnessed simply and unobtrusively?
Fortunately something as simple as saving and displaying a search
trail can capture the association between human search desire and its
satisfaction.
Let me give you a personal example. I spent my first five years at the
University of Queensland in Australia trudging from the car park
(known then as the Dust Bowl) up a winding path to lectures. The
winding path, colloquially called the Goat Trail, was etched into the
grass thanks to the collective unconscious of all the students rushing
to lectures - a planner could not have designed a more optimal route.
Around November each year the Goat Trail would inexplicably change
route - it suddenly diverted around a large pair of jacaranda trees.
The jacaranda trees at the university flower beautifully and smell
even better, but local legend has it that they are deadly to students
- especially around exam time. The legend warns that if a jacaranda
flower lands on your head you're certain to fail your exams. New
students who had never heard of the brain-busting jacaranda flower
were spared failure thanks to following the Goat Trail. I remember
marvelling at the high technology of a simple path on the ground. It
was, in essence, a meme-filtering machine.
The Goat Trail actively encoded the association between search desire
and the destination (lecture hall) and was malleable enough to move
with the times (watch out for those flowers). A similar system is
required for Web search. Who needs all that fancy AJAX-ian widgetry on
the Web? Search engines just need to provide a simple system of way-
finding - we can trust humans to do the rest.
A trail is a simple method of showing the way. On the Web a search
trail begins with a search engine, typically 2.5 keywords, a sequence
of clicks and ends hopefully, but not always, at a relevant result.
Over time the association between a search keyword (desire) and its
most traversed destination becomes stronger. This is analogous to
human memory. Each time a memory trail is traversed the synaptic gaps
fire between neurons and the engram, or unconscious memory, is
reinforced. The more times this trail of neurons fires the stronger
the memory becomes. If it fires less frequently, the memory fades.
Saving memory lane
The real achievement of human memory is not what we remember but what
we forget. Everyday life is full of forgettable factoids and our
memory does a great job of filtering them out. Shouldn't search
engines do the same?
Despite what the publishers say a lot of the pile is worth forgetting,
and we need a system that behaves just like human memory on a communal
scale.
I believe large-scale trailblazing can act like a global search
memory: reinforcing but also fading the associations between search
desires (keywords) and destinations (URLs).
Vannevar Bush lamented the 'artificiality of indexing' and hoped that
'selection by association, rather than indexing, may yet be
mechanised'. I hope so too. The time has come for a pure interaction
layer that sits above the Web and acts as an associative lens that can
burn trails through the pile.
It's time to let the users leave their mark on the pile - not just
publishers. There is a simple democratic truth to a physical trail in
the grass - it's transparent, open and honest. Millions of micro-
discoveries are made every day and we have the means to pass them on
to each other - search trails offer a simple way for us to do it.
Nigel Hamilton is the CEO and mastermind behind Trexy.com. Trexy
enables users to remember and share their search discoveries on the
Web by creating search trails. Nigel initially developed Turbo10.com's
'Deep Net' metasearch technology for connecting to search engines in
the 'invisible Web' or Deep Net. Based on this Deep Net technology
Nigel invented a way of sharing search trails. Prior to establishing
Trexy.com and its sister search engine Turbo10.com, Nigel worked as an
intellectual property barrister and information systems lecturer.
Nigel holds a Bachelor and Masters in Information Technology
(BinfTech, MInfSys) and a Masters of Law (LLM) in intellectual
property. He is an active member of the British Computer Society
(MBCS).
Related FreePint links:
![[Get Copyright Permissions]](http://license.icopyright.net/images/icopy-w.gif "[Get Copyright Permissions]")
Click here for copyright permissions!
Copyright 2008 Free Pint Ltd.
You may also be interested in:
|
