Post-training quantization reduces the computational demand of Large Language Models (LLMs) but can weaken some of their capabilities. Since LLM abilities emerge with scale, smaller LLMs are more sensitive to quantization. In this paper, we explore how quantization affects smaller LLMs’ ability to perform retrieval-augmented generation (RAG), specifically in longer contexts. We chose personalization for evaluation because it is a challenging domain to perform using RAG as it requires long-context reasoning over multiple documents. We compare the original FP16 and the quantized INT4 performance of multiple 7B and 8B LLMs on two tasks while progressively increasing the number of retrieved documents to test how quantized models fare against longer contexts. To better understand the effect of retrieval, we evaluate three retrieval models in our experiments. Our findings reveal that if a 7B LLM performs the task well, quantization does not impair its performance and long-context reasoning capabilities. We conclude that it is possible to utilize RAG with quantized smaller LLMs.
MULTIFILE
The research described in this paper provides insights into tools and methods which are used by professional information workers to keep and to manage their personal information. A literature study was carried out on 23 scholar papers and articles, retrieved from the ACM Digital Library and Library and Information Science Abstracts (LISA). The research questions were: - How do information workers keep and manage their information sources? - What aims do they have when building personal information collections? - What problems do they experience with the use and management of their personal collections? The main conclusion from the literature is that professional information workers use different tools and approaches for personal information management, depending on their personal style, the types of information in their collections and the devices which they use for retrieval. The main problem that they experience is that of information fragmentation over different collections and different devices. These findings can provide input for improvement of information literacy curricula in Higher Education. It has been remarked that scholar research and literature on Personal Information Management do not pay a lot of attention to the keeping and management of (bibliographic) data from external documentation. How people process the information from those sources and how this stimulates their personal learning, is completely overlooked. [The original publication is available at www.elpub.net]
DOCUMENT
A common strategy to assign keywords to documents is to select the most appropriate words from the document text. One of the most important criteria for a word to be selected as keyword is its relevance for the text. The tf.idf score of a term is a widely used relevance measure. While easy to compute and giving quite satisfactory results, this measure does not take (semantic) relations between words into account. In this paper we study some alternative relevance measures that do use relations between words. They are computed by defining co-occurrence distributions for words and comparing these distributions with the document and the corpus distribution. We then evaluate keyword extraction algorithms defined by selecting different relevance measures. For two corpora of abstracts with manually assigned keywords, we compare manually extracted keywords with different automatically extracted ones. The results show that using word co-occurrence information can improve precision and recall over tf.idf.
DOCUMENT
