i-Treasures overall goal is to develop an open and extendable platform to provide access to intangible cultural heritage resources for research and education. The core of i-Treasures lies in the identification of specific media patterns (e.g. postures, gestures, audio patterns, actions etc) using multi-sensor technology (e.g. 2D/3D optical sensors, ultrasound sensors, microphones, EEG etc) in order to reveal hidden intangible treasures from different ICH forms. More specifically, within the framework of the i-Treasures project, four different ICH cases will be studied in detail: 1. Rare traditional songs 2. Rare dance interactions 3. Traditional craftsmanship 4. Contemporary music composition
In all these ICH domains, the human body motion is a dominant factor, however a specific sensors set up will be defined for each use case, according to the requirements of each case. Special care will be taken for the use of non-obtrusive technology, which does not affect the performance of experts; however, especially in the case of rare songs, the use of special equipment (e.g. ultra sound sensor) for the modelling of vocal tract is inevitable.
In order to successfully fulfil the aforementioned goals, the establishment of an active group of "Living Human Treasures" (LHT) is required. The identification of these bearers of ICH will be carried out taking into account the guidelines of UNESCO regarding the implementation of living human treasures systems. In a typical application scenario, the exemplary bearers of intangible cultural heritage are captured using multi modal sensors and subsequently image/signal processing and pattern recognition techniques are applied to extract low (e.g. motion tracking) and medium (media patterns, e.g. postures, actions, audio tempos etc) level features.
The next step aims at a) applying data fusion analysis to exploit information across different modalities and b) integrating context and content for mapping the set of low or medium-level multimedia features to high-level concepts using probabilistic inference i.e the transformation of extracted data into a level of interpretation that is understandable by humans. Given that digital heritage resources include heritage artifacts in digitized form, the semantic analysis of multimedia content, taking into account the specifications defined in ESE (Europeana Semantic Elements), is necessary since the automatic annotation of such artifacts will enable their quick integration in digital collections and facilitate various research or educational tasks.
This completely novel information coupled with other cultural resources will be accessible through the i-Treasures platform in order to enable the widest possible participation of communities, groups and individuals in the safeguarding of ICH. The platform will give access to different types of content (e.g. text, audio, images, video, 3D graphics) from different types of heritage or educational institutions. Using the technology developed within the i-Treasures project, the cultural institutions will be capable, in the future, to decide what content they will produce, thereby contributing to the enrichment of the cultural content. The platform will be developed as an open-source content management system offering a variety of advantages (automated templates allowing multi-lingual and universal access to the content, scalable expansion, easily editable content, access and version control, scalable feature sets with installable software plug-ins or modules to extend functionality, etc.).
For this reason, the web platform will consist of four different sub-systems for i) user interaction (browsing, searching, viewing, playing), ii) back-end management of the web platform (user management, template management, etc.), iii) database management and iv) educational management of the ICH treasures. The main focus of interest on the central database of the platform will be the metadata repository and the corresponding knowledgebase (instantiated ontologies) that will host the products of the semantic analysis. Much like the Europeana project, i-treasures will accept and host - after validation - in its metadata repository compatible metadata of ICH treasures submitted by ICH co-operating partners who wish to promote their cultural heritage through the i-treasures platform.
In addition to the aforementioned functionalities, 3D technology taking advantage of recent advances in web-based game engines will be used to develop, either as a standalone application invoked by relevant interface handles, or as an embedded into the platform application which will enhance training and evaluation of the learner’s performance by means of sensorimotor learning. The application will enable users to create an avatar, i.e. a 3D representation of them that will visualize their motions using multimodal inputs. Animation of vocal tract gestures is a challenging task, which can contribute significantly to the improvement of learners’ performance.
Another challenging issue, concerning especially the use case of rare singing knowledge, is the novel text to singing voice service of the i-Treasures platform, which will be able to produce personalized singing voices on the fly for the Byzantine, Corsican and Sardinian music specific cases. The educational aspect of the project will be technically realized by a seamlessly interoperable Learning Management System (LMS) for planning, implementing and assessing learning processes in order to support educational scenarios. The learning scenarios to be developed will encompass and distinguish among formal, non-formal and informal learning and different learning approaches for different target populations will be addressed. Existing models for the transmission of ICH, which combine traditional forms of acquisition of knowledge with modern pedagogical and academic methods, e.g. the Hungarian Tanchaz method, will also be considered.