STELLA

Infrastructures for Living Labs

Demo Tutorial for the STELLA Evaluation Infrastructure.

In this demo, we’ll explore the STELLA evaluation infrastructure, designed to assess various retrieval systems in a live, online environment. This tutorial provides a technical overview covering the setup, configuration, and operation of the STELLA infrastructure. You’ll learn how to interact with the system components, including the STELLA Server, STELLA App, experimental search systems, and a rudimentary search interface that simulates a search portal for testing.

Insights from the Second Community Workshop

The STELLA project has successfully reached its second phase, and we’re happy to share the highlights of our latest community workshop! Held on June 17th and 18th, 2024, at GESIS in Cologne, this two-day event brought together six invited participants from DIPF Frankfurt, TIB Hannover, and ZBW Kiel with the STELLA team from TH Köln, GESIS and ZB Med. The workshop was designed to explore the future of the DFG-funded STELLA project, discuss various use cases and research questions related to the evaluation of academic search systems, and foster connections within the community.

Report of the 1st STELLA Community Workshop.

The 1st STELLA Community Workshop was held on 20 June 2022 at TH Köln. Eleven invited participants from DIPF Frankfurt, TIB Hannover, and ZBW Kiel attended the workshop to learn about the DFG-funded STELLA project’s outcomes and discuss the future directions and research questions regarding the evaluation of academic search systems.

Evaluating clicks on different SERP elements.

One shortcoming of the previous measures (wins, losses, ties) derived from interleaving experiments is the simplified interpretation of click interactions. By weighting clicks differently, it is possible to account for the meaning of the corresponding elements on the search engine result page (SERP). In this blog post, we share our results on weighting clicks on elements of SERPs differently.

CLEF2021 lab overview

In this blog post, we share some general information about the [Living Labs for Academic Search (LiLAS)](https://clef-lilas.github.io/) at [CLEF2021](clef2021.clef-initiative.eu/). More specifically, we give an overview about the schedule, participants, and the volume of the logged user interaction data.

STELLA Documentation released!

We have released a public documentation that covers introductory guides for participants, sites & organizers, as wells as REST endpoints and other technical details. The documentation is available at: https://stella-project.org/stella-documentation/

STELLA at ISI2021

We have presented STELLA at [ISI 2021 - 16th International Symposium for Information Science](https://isi2021.net/)! In our talk, we provide an updated overview of the STELLA infrastructure and outline how participants can contribute to the living lab experiments. It was held before our CLEF2021 lab in order to promote the infrastructure and discuss it with an expert community. You will find the corresponding paper here: https://epub.uni-regensburg.de/44953 You will find a record of our talk here: https://av.

New tutorial video available on YouTube!

In one of our [earlier blog posts](../posts/STELLA-participant-systems-in-STELLA), we outlined how to participate. The corresponding repository of the template can be found at [GitHub](https://github.com/stella-project/stella-micro-template).

For those who prefer videos instead of reading guides, we have prepared an introduction that is now available on YouTube!

Integrating Participant Systems in STELLA - An Overview

STELLA enables researchers and coders to deploy and evaluate search and recommendation algorithms in a real-world scenario. These so-called "Living Labs" provide a user-focused and thus more realistic evaluation approach. What are the crucial steps when implementing a participant-system in STELLA?