Building Web Indexes with Webcoins

NU 2018-155

 

Primary Inventor

Aleksandar Kuzmanovic*

 

Short Description

A software algorithm that lowers the energy consumption burden of network-based activities while incentivizing secure and trustworthy blockchain technology

 

Background

Web indexes are monopolized by a few large companies due to the challenging, multi-billion-dollar-worth task of crawling and indexing the exabyte-scale Web space. Efforts to decentralised this space by promoting the use of distributed, objective and unbiased indexes have fallen short. This is mostly due to inferior results associated with small Web indices used by such systems, induced by the lack of economic incentives for participation in distributed Web crawling, or due to the small-scale nature of user communities interested in collaborative Web indexing. Additionally, current blockchain networks utilize proof-of-work as their security measure, which requires solving complex mathematical puzzles. Proof-of-work requires a high internet band-width and uses a considerable amount of energy.  As of Q1 2019, Bitcoin contributes more than 49 TWh annually, which is more than Hong Kong’s annual energy consumption.

 

Abstract

Northwestern researchers have developed an innovative solution to solve two digital dilemmas:  (1) offer an unbiased crowdsourced solution to decentralize web indexing; and (2) address the unsustainable computing power required by the blockchain security method, proof-of-work. This is accomplished through miners with as little as 1Mbps internet speed doing network intensive tasks, such as Web indexing, in return for entering a randomized cryptocurrency lottery. These web indexes are then verified with other miners index to validate the blockchain, instead of using expensive computational calculations. By avoiding the proof-of-work method, this novel blockchain algorithm addresses the growing need for a more energy efficient blockchain network, as well as, the trend towards a decentralized web, to disrupt key online industries such as Web search, cloud services, CDNs, and ISPs.

 

Applications

  • Web indexing
  • Sustainable cryptocurrency
  • Blockchain security
  • Blockchain infrastructure

Advantages

For General: 

  • Independent verifiable lottery

For Web-services:

  • Incentivizes people to be users
  • Provides complete and objective Web indexes

For Cryptocurrency

  • Doesn’t require proof-of-work security method
  • Significantly reduces the energy footprint associated with crypto-mining
  • Low band-width users are as capable as high band-width users

Publications

Klarman U, Flores M & Kuzmanovic A (2018) Mining the web with Webcoin. Proceedings of the 14th International Conference on Emerging Networking EXperiments and Technologies - CoNEXT 18.

 

IP Status

Provisional patent application has been filed.

 

Webcoin mining process: (left) (1) Miner crawls web-pages along a directed path, (2) miner indexes web-pages, hashes the index, propagates digest to peers, (3) collectors request and receive compressed indices from miners. (right) (4) Miners wait until some miner becomes eligible to mine a new block, (5) eligible miner propagates block and index, accelerated by collectors, (6) each miner verifies the block, compares index to past digest and verifies a small subset of web-pages. 

Mining the Web with Webcoin

Patent Information: