STELLA enables researchers and coders to deploy and evaluate search and recommendation algorithms in a real-world scenario. These so-called “Living Labs” provide a user-focused and thus more realistic evaluation approach. What are the crucial steps when implementing a participant-system in STELLA?
In academia, we often rely on standard test collections with static relevance judgments that somewhat abstract away the specific information need of single users. Thus, evaluating search engine algorithms with real-world users is a desiderata. Finally, we want to know how users can benefit from innovative search engines. The STELLA project tries to bridge this gap with the help of an innovative infrastructure that makes it possible to evaluate experimental search algorithms in live environments. In this post we want to outline the general idea of our infrastructure.
We invite you to submit a early research of open ideas paper to the LiLAS workshop. The Living Labs for Academic Search (LiLAS) Workshop at CLEF 2020 will be held at Thessaloniki, Greece between 22-25 September 2020. The Call for Contributions is open until 24 May 2020. Please feel free to contact us if you have any questions!
Follow this link to see the official press announcement by the TH Köln.
Follow this link to see the project description by the TH Köln.
Follow this link to see the official press announcement by HN NRW Hochschulnetzwerk.