Information Explosion and Access-Emerging Issues: Kartik Hosanagar

What we really want is knowledge. Not information.

Cost of accessing iformation is dropping exponentially
Cost of searching through the stock of information is increasing

Outside the human brain – infmn
Inside the human briain – knowledge


Movies
Books
Research papers
Photographs
Blogs usenet postings


Infmn obesity and knowledge health


Kartik Hosanagar, BangOnIT

Information search and retrieval
10 bn webpages, 1997 100 million

Invisible web
Information overload ,
poverty of attention
Excess of information


Highlevel view
Bot looks for new pages every night
Local copy
Parse
Keywords, index
Page rank

Cosine measure
How authoritative the page is

Triplew

Official biography of GW bush
Miserable failure
US and French
French military victory


Metrics
Precision
Recall


Algorithms
Bots
Enhanced features


Tip of the iceberg
- search process
- -NLP and AI

Salary of president – Ask jeeves
- archieve.org
- searching multimedia

Microsoft research
- you don’t need to understand the question

gigabeat,alltheweb

sovereignity, and Bush joke
global entertainer


make machine learn cricket video

- specialty search engine
job search
medical information
desktop search

shopbot

comparision pricing search

unbiased search results
- life insurance

search industry
B2C intermediary

Most used applications on internet
Email
Search

AOL

User interface
Search technology
adnetwork

bidding model, overture
click through rate

NYT search wars

Issues
Information reliability
Invisible web issue ( 500 times the visible web)
International politics


Data mining
Looking for patterns in large data repository

Examples
- insurance claims
good credit ratings, fewer accidents

1$ transfer

Healthcare,
Reynaud’s disease
Viscosity of blood

-personalisation
Recommendation system in amazon

Celluar service provider
Who is likely to leave the service
NTT do co mo


High variation in the billing
Suggest better plans
Customer acquisition in US>300$

P2P networks
Paradigm shift in networking
Content delivery
Kazaa

Scalability
Common grazing land

Over consumption of resources

Incentives to share

SETI@home
Berkeley
Processing power ET

Intel
Grid computing
Netbatch