Impact and development of an Open Web Index for Open Web Search

ngi4all.e 14 de diciembre de 2023
Impact and development of an Open Web Index for Open Web Search

In a thought-provoking opinion piece published in the Journal of the Association for Information Science and Technology, a team of experts led by Michael Granitzer and Stefan Voigt delves into the concept of creating an Open Web Index (OWI) for a more open and diverse web search ecosystem.

This article, first published on August 7, 2023, explores the limitations of current web search engines dominated by a few major players and proposes a transformative approach to web indexing and search functionality.

The authors argue that the web search industry, primarily controlled by companies like Google and Bing, often leads to content optimization for these gatekeepers rather than fostering a diverse and open information space. They propose the OWI to solve this problem, focusing on these core principles:

✅ Treating the index as open data.
✅ Ensuring legal compliance.
✅ Fostering collaborative technology development.

Key highlights from the article include an in-depth analysis of the challenges in building a web search engine, emphasizing the technical and financial hurdles involved. The authors compare various search engines and examine the feasibility and implications of creating an independent web search infrastructure.

Furthermore, the piece outlines the principles of an OWI, emphasizing its potential to democratize web search by offering alternative search engines with reduced costs and increased utility. It discusses the technical aspects of constructing an OWI, including coordinated and legally compliant crawling, preprocessing, semantic enrichment, and declarative search engines.

Finally, the article delves into an OWI's potential impacts and applications, envisioning a future where users have genuine choices in web search platforms, leading to a more equitable and diverse information space.

The authors conclude with insights into the ongoing EU-funded project OpenWebSearch.EU, which aims to prototype this concept.

👉 Read the full article here