Computational methods for depression detection aim to mine traces of depression from online publications posted by Internet users. However, solutions trained on existing collections exhibit limited generalisation and interpretability. To tackle these issues, recent studies have shown that identifying specific depressive symptoms can lead to more robust and effective models. The eRisk initiative fosters research on this area and has recently proposed a new ranking task focused on developing search methods to find sentences related to depressive symptoms. This search challenge relies on the symptoms specified by the Beck Depression Inventory-II (BDI-II), a questionnaire widely used in clinical practice. It includes symptoms such as sadness, irritability or lack of sleep. Given the input submitted by systems participating in eRisk, we first apply top-k pooling over the systems’ relevance rankings, obtaining a diverse set of sentences. These sentences are judged for relevance, leading to DepreSym, a dataset consisting of 21,580 sentences annotated according to their relevance to the 21 BDI-II symptoms. This dataset serves as a valuable resource for advancing the development of models that monitor depression markers. Due to the complex nature of this relevance annotation, we designed a robust assessment methodology carried out by three expert assessors, including a trained psychologist. As part of this study, we explore the potential of recent Large Language Models (ChatGPT, GPT4 and Vicuna) as assessors in this complex task. We undertake a comprehensive examination of the LLMs’ performance, studying their main limitations and analysing their role as a complement or replacement for human annotators. Finally, we incorporate our dataset into the Benchmarking Information Retrieval (BEIR) framework for a thorough search evaluation. We use state-of-the-art retrieval systems, including lexical, sparse, dense and re-ranking architectures, to gain insights about the dataset’s complexity and identify potential avenues for improvement.

Dice ser muy optimista respecto a que en otros planetas se haya podido desarrollar de una manera u otra la vida: «Estamos en ello»

El Salón de Actos Concepción Arenal será escenario este mes de octubre del segundo ciclo Auto-Sinerx-IA, centrado en los retos de la industria y la sociedad ante la transformación tecnológica Te puede interesar: La UDC desarrolla una herramienta de análisis con IA para los puertos de A Coruña, Ferrol y Malpica

A Coruña acoge desde este viernes la quinta edición del foro internacional Ecosystems

A intelixencia artificial está a transformar o panorama dixital ofrecendo unha comodidade e eficiencia sen precedentes, pero tamén xorden novos retos de privacidade e seguridade. Unha opción é…

O prazo de solicitude de axuda da nova convocatoria está aberto ata o 5 de outubro. Esta iniciativa enmárcase no programa internacional MIT-Spain (MISTI) e supón un paso decisivo na estratexia de internacionalización do CITIC.