In this paper the principle of minimum relative entropy (PMRE) is proposed as a fundamental principle and idea that can be used in the field of AGI. It is shown to have a very strong mathematical foundation, that it is even more fundamental then Bayes rule or MaxEnt alone and that it can be related to neuroscience. Hierarchical structures, hierarchies in timescales and learning and generating sequences of sequences are some of the aspects that Friston (Fri09) described by using his free-energy principle. These are aspects of cognitive architectures that are in agreement with the foundations of hierarchical memory prediction frameworks (GH09). The PMRE is very similar and often equivalent to Friston's free-energy principle (Fri09), however for actions and the defi nitions of surprise there is a diff erence. It is proposed to use relative entropy as the standard definition of surprise. Experiments have shown that this is currently the best indicator of human surprise (IB09). The learning rate or interestingness can be defi ned as the rate of decrease of relative entropy, so curiosity can then be implemented as looking for situations with the highest learning rate.
DOCUMENT
The principle of minimum relative entropy is proposed as a general fundamental principle that could be used by the brain to do inference and update beliefs about the world. It originates from information and probability theory, but we relate it it to the brain, to the concept of surprise and to a minimum free-energy principle that has already been proposed for the brain. The measure of surprise that is based on relative entropy (Bayesian surprise) is compared with another definition of surprise (Shannon surprise) that is used by Friston for a minimum free-energy principle. Theoretical and experimental justifi cations are given to propose to use Bayesian surprise as a better and more natural definition of surprise. It can be used as a novel way to quantify surprise or related concepts in developmental robotics. This can then be used in implementations of intrinsic motivations like curiosity to drive exploration, interactive learning and autonomous mental development.
DOCUMENT
Background: Lexical access problems of inflected verbs are common in aphasia. Previous research addressed these problems either in purely linguistic terms (e.g., verb movement) or in terms of lexical characteristics (e.g., frequency). We propose a new measure of verb complexity, which combines linguistic and lexical characteristics and is formulated in terms of Shannon’s information theory. Aims: We aim to explore the complexity of individual verbs and verb paradigms and its effect on lexical access, both in unimpaired people and people with aphasia (PWA). We apply information theory to investigate the impact of verb complexity on reaction time (RT) for lexical decision. Methods & Procedures: 20 non-fluent aphasic subjects and 11 age-matched and education-matched peers performed an auditory lexical decision task containing 286 real and 286 phonotactically legal non-word past tense forms (regulars and irregulars). RTs and error rates were measured. Two information-theoretic measures were calculated: inflectional entropy (reflecting probabilistic variability of forms within a given verbal family) and information load (I) (reflecting complexity of an individual verb form). The effect for these and other more traditional measures on RT were measured. Outcomes & Results: Linear mixed model analyses to the data for each group with participant and verb as crossed random effects were performed. Results show that for all groups inflectional entropy had a facilitatory effect on RT. There was a group effect for inflectional entropy indicating that for the patients with aphasia the effect of inflectional entropy was less pronounced. At the same time, I did correlate with latencies for healthy adults but not for individuals with aphasia. Conclusions: Our results demonstrate that the decrease in lexical processing capacity characteristic for PWA has a measurable effect that can be calculated using information theoretical means. According to our model, these individuals have particular difficulties with processing lexical items of higher complexity, as measured by individual I, and benefit less from the support normally provided (in comprehension) by other members of the corresponding lexical network. Finally, the proposed information-theoretic complexity measures, which encompass both frequency effects and linguistic parameters, provide a superior measure of lexical access, and have a better explanatory power for the analyses of access problems found in non-fluent aphasia, compared to analyses based on frequency only.
LINK
The purpose of this paper is to perform a metaphorical analysis of knowledge as energy. This paper is based on a theoretical research concerning the nature, perception, basic laws and challenges brought up by these fundamental concepts of knowledge and energy. The metaphorical analysis of knowledge and intellectual capital has been initiated by Daniel Andriessen and his findings have been presented in several seminal works (Andriessen, 2006; 2008; Andriessen and Boom, 2007). In his work, Andriessen concluded we need to find new metaphors for knowledge. In our theoretical research we shall consider the knowledge as energy metaphor, with energy as the source domain, and knowledge as the target domain, and we are interested in identifying the metaphorical semantic kernel and the limitations of this analysis. The semantic kernel contains: (1) the concept of field as a nonuniform and nonlinear distribution of knowledge; (2) dynamics of potential and kinetic forms of manifestations; (3) dynamics of work and heat, and (4) entropy and syntropy process characteristics. Limitations of this analysis come from the conservation laws of energy transformation which cannot be applied to the knowledge domain.
DOCUMENT
from the article: ABSTRACT Independence of design, information and complexity are the basic concepts of Axiomatic Design. These basic concepts have proven to be generic; axiomatic design was successfully applied in many markets and on a broad range of products and services. Information, or entropy, plays a central role in Axiomatic Design. In this paper an attempt is made to organise the different kinds of information, understand them, and evaluate the consequences of the ways they can be applied. A number of six kinds of information are reduced to two most determining kinds of information for the design. Unorganised information is about choosing the right and independent design relations. Axiomatic information is about further optimisation of these design relations. This paper leads to the confirmation that axiom 1 & 2 are in fact corollaries of the complexity axiom that is constituted of the two kinds of information. Though this revises the foundation of Axiomatic Design, the operation and practical application are not much affected for a number of reasons. One of them is that a higher axiom does not alter the basic ideas behind Axiomatic Design; it remains axiomatic.
LINK
Background: G-protein coupled receptors (GPCRs) are involved in many different physiological processes and their function can be modulated by small molecules which bind in the transmembrane (TM) domain. Because of their structural and sequence conservation, the TM domains are often used in bioinformatics approaches to first create a multiple sequence alignment (MSA) and subsequently identify ligand binding positions. So far methods have been developed to predict the common ligand binding residue positions for class A GPCRs.Results: Here we present 1) ss-TEA, a method to identify specific ligand binding residue positions for any receptor, predicated on high quality sequence information. 2) The largest MSA of class A non olfactory GPCRs in the public domain consisting of 13324 sequences covering most of the species homologues of the human set of GPCRs. A set of ligand binding residue positions extracted from literature of 10 different receptors shows that our method has the best ligand binding residue prediction for 9 of these 10 receptors compared to another state-of-the-art method.Conclusions: The combination of the large multi species alignment and the newly introduced residue selection method ss-TEA can be used to rapidly identify subfamily specific ligand binding residues. This approach can aid the design of site directed mutagenesis experiments, explain receptor function and improve modelling. The method is also available online via GPCRDB at http://www.gpcr.org/7tm/. © 2011 Sanders et al; licensee BioMed Central Ltd.
DOCUMENT
Wereldwijd schieten ze als paddenstoelen uit de grond: living labs. Deze ‘levende laboratoria’ zijn er in alle soorten en maten. Meestal wordt het lab gezien als een onderzoeks- en ontwikkelomgeving om een probleem met verschillende partijen op een innovatieve manier op te lossen. De thema’s van de labs variëren van het ontwerpen van de lichtste boot of de snelste (zonne)auto tot het opnieuw inrichten van een havengebied, halvering van koolstof-uitstoot van het goederenvervoer in steden, nieuwe zorgconcepten of het versterken van de lokale democratie. Wat iemand van het meedoen aan een living lab zou kunnen leren wordt uitgelegd aan de hand van een living lab waaraan de auteur zelf als deelnemer drie keer heeft meegedaan, Aalto Camp for Societal Innovation (ACSI).
DOCUMENT
We present a novel architecture for an AI system that allows a priori knowledge to combine with deep learning. In traditional neural networks, all available data is pooled at the input layer. Our alternative neural network is constructed so that partial representations (invariants) are learned in the intermediate layers, which can then be combined with a priori knowledge or with other predictive analyses of the same data. This leads to smaller training datasets due to more efficient learning. In addition, because this architecture allows inclusion of a priori knowledge and interpretable predictive models, the interpretability of the entire system increases while the data can still be used in a black box neural network. Our system makes use of networks of neurons rather than single neurons to enable the representation of approximations (invariants) of the output.
LINK
Preprint submitted to Information Processing & Management Tags are a convenient way to label resources on the web. An interesting question is whether one can determine the semantic meaning of tags in the absence of some predefined formal structure like a thesaurus. Many authors have used the usage data for tags to find their emergent semantics. Here, we argue that the semantics of tags can be captured by comparing the contexts in which tags appear. We give an approach to operationalizing this idea by defining what we call paradigmatic similarity: computing co-occurrence distributions of tags with tags in the same context, and comparing tags using information theoretic similarity measures of these distributions, mostly the Jensen-Shannon divergence. In experiments with three different tagged data collections we study its behavior and compare it to other distance measures. For some tasks, like terminology mapping or clustering, the paradigmatic similarity seems to give better results than similarity measures based on the co-occurrence of the documents or other resources that the tags are associated to. We argue that paradigmatic similarity, is superior to other distance measures, if agreement on topics (as opposed to style, register or language etc.), is the most important criterion, and the main differences between the tagged elements in the data set correspond to different topics
DOCUMENT
Background and purpose The aim of this study is to investigate changes in movement behaviors, sedentary behavior and physical activity, and to identify potential movement behavior trajectory subgroups within the first two months after discharge from the hospital to the home setting in first-time stroke patients. Methods A total of 140 participants were included. Within three weeks after discharge, participants received an accelerometer, which they wore continuously for five weeks to objectively measure movement behavior outcomes. The movement behavior outcomes of interest were the mean time spent in sedentary behavior (SB), light physical activity (LPA) and moderate to vigorous physical activity (MVPA); the mean time spent in MVPA bouts ≥ 10 minutes; and the weighted median sedentary bout. Generalized estimation equation analyses were performed to investigate overall changes in movement behavior outcomes. Latent class growth analyses were performed to identify patient subgroups of movement behavior outcome trajectories. Results In the first week, the participants spent an average, of 9.22 hours (67.03%) per day in SB, 3.87 hours (27.95%) per day in LPA and 0.70 hours (5.02%) per day in MVPA. Within the entire sample, a small but significant decrease in SB and increase in LPA were found in the first weeks in the home setting. For each movement behavior outcome variable, two or three distinctive subgroup trajectories were found. Although subgroup trajectories for each movement behavior outcome were identified, no relevant changes over time were found. Conclusion Overall, the majority of stroke survivors are highly sedentary and a substantial part is inactive in the period immediately after discharge from hospital care. Movement behavior outcomes remain fairly stable during this period, although distinctive subgroup trajectories were found for each movement behavior outcome. Future research should investigate whether movement behavior outcomes cluster in patterns.
MULTIFILE
