Semantic modeling, integration and episodic organization of personal digital traces
Description
TitleSemantic modeling, integration and episodic organization of personal digital traces
Date Created2022
Other Date2022-10 (degree)
Extent1 online resource (116 pages) : illustrations
DescriptionMemory plays a fundamental role in life and is critical to our everyday functioning. We use memories to maintain our personal identity, to support our relationships, to learn, and to solve problems. Today, many tools support the digital capture of aspects of our lives. These tools produce a multitude of data objects, which we call Personal Digital Traces (PDTs). Our goal is to use PDTs to help reconstruct people’s episodic memories for those events for which the system has been given generic stereotypical common-sense descriptions (to be called “scripts”). This reconstruction of script instances can have several applications: helping the recall of patients with neurodegenerative diseases; helping people remember past events and better manage their data and time if this information is used in activity-centric applications, like personal assistants; or even helping people identify recent contacts and places visited – a critical new application for the recent COVID-19 health crisis.
However, the above endeavour comes with many challenges. For one, the urge to capture as much information as possible, as well as the continuous recordings made by many apps, result in large heterogeneous collections of data scattered over disparate sources. These must be structured, integrated, and searched. In addition, people’s activities are complex and some may overlap with, or subsume, one another. Another challenge is that of evaluation, which is difficult due to (well-founded) privacy concerns, a lack of standardized methodology, and the difficulty in obtaining personal datasets for research purposes.
This dissertation takes steps towards supporting autobiographical memory by associating (heterogeneous) PDTs with events according to their higher-level purposes and uses, and by summarizing them into episodic narratives. We start by presenting a unified and formalized conceptual modeling language whose novel features include the properties “who, what, when, where, why, how” applied uniformly to both PDTs and the corresponding atomic and complex events that produce them. We then proceed by describing complex activities, and thereby organize PDTs, making use of stereotypical commonsense higher level plans (“scripts”) for common everyday events. We show how families of related scripts can be defined by incrementally modifying more general scripts through inheritance. We present a new Regular Expression based Description Logic for scripts in order to reason about them (e.g., organize/verify inheritance hierarchies), explaining why the current standard use of DLs for this purpose is not good. For instantiating those scripts based on the lower level actions of which scripts are composed of, we present a bottom up merging algorithm that groups and relates several digital traces from many different sources into script instances (episodes) as well as a software architecture that supports systematic and declarative specification of evidence. This also utilizes a scoring scheme to account for the varied strength of evidence provided by PDTs or script steps. We then present a multiplayer web-based game called OneOfUs, which indicates possible associated digital trace descriptions that can be produced by each activity/script through crowdsourcing techniques that act as evidence for the execution of such scripts. The game is able to automatically validate and assess knowledge at the time of the game, as well as dynamically acquire new pieces of information, as it has a way of not neglecting uncommon answers through players’ votes.
Finally, to evaluate the efficacy of our methodology, we designed and implemented YourDigitalSelf, an Android mobile device application that gathers and integrates personal digital traces from various popular services into narratives. A thorough evaluation performed over real user’s data collections shows that our approach is able to integrate and combine successfully different traces from different popular sources into coherent episodes/activities. In addition, we show evidence that our approach does augment user’s memory of their past actions, and thereby forms a powerful retrospective memory aid.
NotePh.D.
NoteIncludes bibliographical references
Genretheses
LanguageEnglish
CollectionSchool of Graduate Studies Electronic Theses and Dissertations
Organization NameRutgers, The State University of New Jersey
RightsThe author owns the copyright to this work.
Statistical Profile