In this post I give an overview of the theory, tools, frameworks and best practices I have found until now around the testing (and debugging) of machine learning applications. I will start by giving an overview of the specificities of testing machine learning applications.
LINK
A previous study found a variety of unusual sexual interests to cluster in a five-factor structure, namely submission/masochism, forbidden sexual activities, dominance / sadism, mysophilia, and fetishism (Schippers et al., 2021). The current study was an empirical replication to examine whether these findings generalized to a representative population sample. An online, anonymous sample (N = 256) representative of the Dutch adult male population rated 32 unusual sexual interests on a scale from 1 (very unappealing) to 7 (very appealing). An exploratory factor analysis assessed whether similar factors would emerge as in the original study. A subsequent confirmatory factor analysis served to confirm the factor structure. Four slightly different factors of sexual interest were found: extreme, illegal and mysophilic sexual activities; light BDSM without real pain or suffering; heavy BDSM that may include pain or suffering; and illegal but lower-sentenced and fetishistic sexual activities. The model fit was acceptable. The representative replication sample was more sexually conservative and showed less sexual engagement than the original convenience sample. On a fundamental level, sexual interest in light BDSM activities and extreme, forbidden, and mysophilic activities seem to be relatively separate constructs.
DOCUMENT
Purpose – purpose of this article is to report about the progress of the development of a method that makes sense of knowledge productivity, in order to be able to give direction to knowledge management initiatives. Methodology/approach – the development and testing of the method is based on the paradigm of the Design Sciences. In order to increase the objectivity of the research findings, and in order to test the transferability of the method, this article suggests a methodology for beta testing. Findings – based on the experiences within this research, the concept of beta testing seems to fit Design Science Research very well. Moreover, applying this concept within this research resulted in valuable findings for further development of the method. Research implications – this is the first article that explicitly applies the concept of beta testing to the process of developing solution concepts. Originality/value – this article contributes to the further operationalization of the relatively new concept of knowledge productivity. From a methodological point of view, this article aims to contribute to the paradigm of the Design Sciences in general, and the concept of beta testing in particular.
DOCUMENT
In the last decade, the automotive industry has seen significant advancements in technology (Advanced Driver Assistance Systems (ADAS) and autonomous vehicles) that presents the opportunity to improve traffic safety, efficiency, and comfort. However, the lack of drivers’ knowledge (such as risks, benefits, capabilities, limitations, and components) and confusion (i.e., multiple systems that have similar but not identical functions with different names) concerning the vehicle technology still prevails and thus, limiting the safety potential. The usual sources (such as the owner’s manual, instructions from a sales representative, online forums, and post-purchase training) do not provide adequate and sustainable knowledge to drivers concerning ADAS. Additionally, existing driving training and examinations focus mainly on unassisted driving and are practically unchanged for 30 years. Therefore, where and how drivers should obtain the necessary skills and knowledge for safely and effectively using ADAS? The proposed KIEM project AMIGO aims to create a training framework for learner drivers by combining classroom, online/virtual, and on-the-road training modules for imparting adequate knowledge and skills (such as risk assessment, handling in safety-critical and take-over transitions, and self-evaluation). AMIGO will also develop an assessment procedure to evaluate the impact of ADAS training on drivers’ skills and knowledge by defining key performance indicators (KPIs) using in-vehicle data, eye-tracking data, and subjective measures. For practical reasons, AMIGO will focus on either lane-keeping assistance (LKA) or adaptive cruise control (ACC) for framework development and testing, depending on the system availability. The insights obtained from this project will serve as a foundation for a subsequent research project, which will expand the AMIGO framework to other ADAS systems (e.g., mandatory ADAS systems in new cars from 2020 onwards) and specific driver target groups, such as the elderly and novice.
The automobile industry is presently going through a rapid transformation towards autonomous driving. Nearly all vehicle manufacturers (such as Mercedes Benz, Tesla, BMW) have commercial products, promising some level of vehicle automation. Even though the safe and reliable introduction of technology depends on the quality standards and certification process, but the focus is primarily on the introduction of (uncertified) technology and not on developing knowledge for certification. Both industry and governments see the lack of knowledge about certification, which can ensure the safety of autonomous technology and thus will guarantee the safety of the driver, passenger, and environment. HAN-AR recognized the lack of knowledge and the need for novel certification methodology for emerging vehicle technology and initiated the PRAUTOCOL project together with its SME partners. The PRAUTOCOL project investigated certification methodology for two use-cases: certification for automated highway overtaking pilot; and certification for automatic valet parking. The PRAUTOCOL research is conducted in two parallel streams: certification of the driver by human factors experts and certification of vehicle by technology experts. The results from both streams are published and presented in respective but limited target groups. Also, an overview of the PRAUTOCOL certification methodology is missing, which can enable its translation to different use-cases of automated technology (other than the used ones). Therefore, to realize a better pass-through of PRAUTOCOL's results to a broader audience, the top-up is required. Firstly, to write a (peer-reviewed) Open Access article, focusing on the application and translation of PRAUTOCOL's methodology to other automated technology use-cases. Secondly, to write a journal article, focusing on the validation of automatic highway overtaking system using naturalistic driving data. Thirdly, to organize a workshop to present PRAUTOCOL's results (valorization) to industrial, research, and government representatives and to discuss a follow-up initiative.
