| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| en:iot-open:data_management_and_semantics_ume [2018/05/12 11:46] – salvatdi | en:iot-open:data_management_and_semantics_ume [2018/05/12 12:07] (current) – removed salvatdi |
|---|
| ==== Data and Information Management in the Internet of Things ==== | |
| |
| At the center of the IoT ecosystem consisting of billions of connected devices is the wealth of information that can be made available through the fusion of data that is produced in real-time, as well as data stored in permanent repositories. | |
| This information can make the realisation of innovative and unconventional applications and value-added services possible, and will act as an immense source for trend analysis and strategic business opportunities. A comprehensive management framework of data and information that is generated and stored by the objects within the IoT is thus required to achieve this goal. | |
| |
| Data management is a broad concept referring to the architectures, practices, and procedures for proper management of the data lifecycle requirements of a certain IT system. As far as the IoT is concerned, data management should act as a layer between the physical sensing objects and devices generating the data - on the one hand, and the applications accessing the data for analysis purposes and services - on the other. | |
| |
| The IoT data has distinctive characteristics that make traditional relational-based database management an obsolete solution. A massive volume of heterogeneous, streaming and geographically-dispersed real-time data will be created by millions of diverse devices periodically sending observations about certain monitored phenomena or reporting the occurrence of certain or abnormal events of interest. Periodic observations are most demanding in terms of communication overhead and storage due to their streaming and continuous nature, while events present time-strain with end-to-end response times depending on the urgency of the response required for the event. Furthermore, in addition to the data that is generated by IoT entities, there is also metadata that describes these entities (i.e. “things”), such as object identification, location, processes and services provided. The IoT data will statically reside in fixed- or flexible-schema databases and roam the network from dynamic and mobile objects to concentration storage points. This will continue until it reaches centralised data stores. Communication, storage and process will thus be defining factors in the design of data management solutions for the IoT. | |
| |
| Traditional data management systems handle the storage, retrieval, and update of elementary data items, records and files. In the context of the IoT, data management systems must summarise data online while providing storage, logging, and auditing facilities for offline analysis. This expands the concept of data management from offline storage, query processing, and transaction management operations into online-offline communication/storage dual operations. We first define the data lifecycle within the context of the IoT and then discuss some of the phases in order to have a better understanding of the IoT data management. | |
| |
| [[en:iot-open:data_lifecycle]] | |
| |
| [[en:iot-open:iotdatavsdb]] | |
| |
| [[en:iot-open:data_sources]] | |
| |
| [[en:iot-open:data_gen_domain]] | |
| |
| [[en:iot-open:cfe]] | |
| |
| [[en:iot-open:data_storage_models_frameworks]] | |
| |
| [[en:iot-open:data_processing_models_frameworks]] | |
| |
| |
| |
| |
| === IoT data semantics=== | |
| |
| With some 25 billion devices expected to be connected to the Internet by 2015 and 50 billion by 2020, providing interoperability among the things on the IoT is one of the most fundamental requirements to support object addressing, tracking, and discovery as well as information representation, storage, and exchange. | |
| |
| The lack of explicit and formal representation of the IoT knowledge could cause ambiguity in terminology, hinder interoperability and mostly semantic interoperability of entities in the IoT world. Furthermore, lack of shared and agreed semantics for this domain (and for any domain) may easily result to semantic heterogeneity - i.e. to the need to align and merge a vast number of different modeling efforts to semantically describe IoT entities, efforts conducted by many different ontology engineers and IoT vendors (domain experts). Although there are tools nowadays to overcome such a problem, it is not a fully automated and precise process and it would be much easier to do so if there is at least a partial agreement between the related stakeholders - i.e. a commonly agreed IoT ontology. | |
| |
| In these circumstances, an ontology can be used as a semantic registry for the facilitation of the automated deployment of generic and legacy IoT solutions in environments where heterogeneous devices also have been deployed. Such a service can be delivered by IoT solution providers, supporting remotely the interoperability problems of their clients/buyers when buying third-party devices or applications. Practically, this will require the existence of a central point - e.g. a web service/portal for both end users (buyers of the devices) and the IoT solution providers (sellers of the applications) to register their resources, i.e. both the devices and the IoT solutions, in an ontology-based registry. | |
| |
| == Sensor Web Enablement and Semantic Sensor Networks== | |
| |
| The Sensor Web Enablement (SWE) standards enable developers to make all types of sensors, transducers and sensor data repositories discoverable, accessible and usable via the Web. Sensor technology, computer technology and network technology are advancing together while demand grows for ways to connect information systems with the real world. Linking diverse technologies in this fertile market environment, integrators are offering new solutions for plant security, industrial controls, meteorology, geophysical survey, flood monitoring, risk assessment, tracking, environmental monitoring, defense, logistics and many other applications. The SWE effort develops the global framework of standards and best practices that make linking of diverse sensor related technologies fast and practical. Standards make it possible to put the pieces together in an efficient way that protects earlier investments, prevents lock-in to specific products and approaches, and allows for future expansion. Standards also influence the design of new component products. Business needs drive the process. Technology providers and solutions providers need to stay abreast of these evolving standards if they are to stay competitive. | |
| |
| Semantic Web technologies have been proposed as a means to enable interoperability for sensors and sensing systems in the context of SWE. Semantic Web technologies could be used in isolation or in augmenting SWE standards in the form of the Semantic Sensor Web (SSW). Semantic technologies can assist in managing, querying, and combining sensors and observation data. Thus allowing users to operate at abstraction levels above the technical details of format and integration, instead working with domain concepts and restrictions on quality. Machine-interpretable semantics allows autonomous or semi-autonomous agents to assist in collecting, processing, reasoning about, and acting on sensors and their observations. Linked Sensor Data may serve as a means to interlink sensor data with external sources on the Web. | |
| |
| One of the main outcomes of the SSW research is the Semantic Sensor Network (SSN) ontology (by W3C Semantic Sensor Network Incubator Group). This IoT ontology provides all the necessary semantics for the specification of IoT devices as well as the specifications of the IoT solution (input, output, control logic) that is deployed using these devices. These semantics include terminology related to sensors and observations, reusing the one already provided by the SSN ontology, and extended to capture also the semantics of devices beyond sensors - i.e. actuators, identity devices (tags), embedded devices, and of course the semantics of the devices and things that are observed by sensors, that change their status by actuators, that are attached to identity tags, etc. Furthermore, the ontology includes semantics for the description of the registered IoT solutions - i.e. input, output, control logic - in terms of aligning and matching their requirements with the specifications and services of the registered devices. | |
| |
| === IoT data visualisation=== | |
| |
| One of the challenges for the IoT industry is data analysis and interpretation. Big Data generated by the IoT devices is impractical if it cannot be translate into a language that is easy to understand, process and present as visual language. For this reason, IoT data visualisation is becoming an integral part of the IoT. Data visualisation provides a way to display this avalanche of collected data in meaningful ways that clearly present insights hidden within this mass amount of information. This can assist us in making fast, informed decisions with more certainty and accuracy than ever before. It is thus vital for business professionals, developers, designers, entrepreneurs and consumers alike to be aware of the role that Visualization will and can play in the near future. It is crucial to know how it can affect the experience and effectiveness of the IoT products and services. | |
| |
| |
| === Machine learning and data science=== | |
| |
| TODO? | |
| |
| |
| |
| === Sources=== | |
| |
| https://www.cbronline.com/internet-of-things/10-of-the-biggest-iot-data-generators-4586937/ | |
| |
| https://www.sam-solutions.com/blog/how-much-data-will-iot-create-2017/ | |
| |
| http://www.enterprisefeatures.com/6-important-stages-in-the-data-processing-cycle/ | |
| |
| https://pinaclsolutions.com/blog/2017/cloud-computing-and-iot | |
| |
| http://internetofthingsagenda.techtarget.com/blog/IoT-Agenda/Its-time-for-fog-edge-computing-in-the-internet-of-things | |
| |
| https://www.digitalocean.com/community/tutorials/hadoop-storm-samza-spark-and-flink-big-data-frameworks-compared | |
| |
