Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| en:iot-reloaded:iot_data_analysis [2024/12/03 17:20] – pczekalski | en:iot-reloaded:iot_data_analysis [2025/05/17 08:56] (current) – agrisnik | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== IoT Data Analysis ====== | ====== IoT Data Analysis ====== | ||
| - | IoT systems are built to provide better insights into different processes and systems | + | IoT systems are built to provide better insights into different processes and systems |
| Today, IoT systems produce a vast amount of data, which is very hard to use manually. Thanks to modern hardware and software developments, | Today, IoT systems produce a vast amount of data, which is very hard to use manually. Thanks to modern hardware and software developments, | ||
| Line 11: | Line 11: | ||
| === Variety === | === Variety === | ||
| - | Jain explained that big data is highly heterogeneous | + | Jain explained that Big Data is highly heterogeneous |
| === Veracity === | === Veracity === | ||
| Line 19: | Line 19: | ||
| === Velocity === | === Velocity === | ||
| - | Data velocity characterises the data bound to the time and its importance during a specific period or at a particular time instant. A good example might be any real-time system like an industrial process control system, where reactions or decisions must be made during a fixed period | + | Data velocity characterises the data bound to the time and its importance during a specific period or at a particular time instant. A good example might be any real-time system like an industrial process control system, where reactions or decisions must be made during a fixed period, requiring data at particular time instants. In this case, data has a flow nature of a specific density. |
| === Value === | === Value === | ||
| Line 26: | Line 26: | ||
| ====== ====== | ====== ====== | ||
| - | Dealing with big data requires specific hardware and software infrastructure. While there is a certain number of typical solutions and a lot more customise, some of the most popular are explained here: | + | Dealing with Big Data requires specific hardware and software infrastructure. While there is a certain number of typical solutions and a lot more customised, some of the most popular are explained here: |
| === Relational DB-based systems === | === Relational DB-based systems === | ||
| Line 36: | Line 36: | ||
| * Enables asynchronous reactions to events by triggering internal events. | * Enables asynchronous reactions to events by triggering internal events. | ||
| * Data reading might be scaled out using multiple entities, while writing might be scaled up using more productive servers. | * Data reading might be scaled out using multiple entities, while writing might be scaled up using more productive servers. | ||
| - | Unfortunately, | + | Unfortunately, |
| <figure RelationalDBMS> | <figure RelationalDBMS> | ||
| - | {{ : | + | {{ : |
| - | < | + | < |
| </ | </ | ||
| === Complex Event Processing (CEP) systems === | === Complex Event Processing (CEP) systems === | ||
| - | CEP systems are very application-tailored, | + | CEP systems are very application-tailored, |
| Some of the most common drawbacks to be considered are: | Some of the most common drawbacks to be considered are: | ||
| * It might be scaled up only by introducing higher productivity hardware, which is limited by the application-specific design. To some extent, the design might be more flexible if microservices and containerisation are applied. | * It might be scaled up only by introducing higher productivity hardware, which is limited by the application-specific design. To some extent, the design might be more flexible if microservices and containerisation are applied. | ||
| - | * Due to the factors mentioned above and the complexity, the maintenance costs are usually higher than a universal design. | + | * Due to the factors mentioned above and the complexity, the maintenance costs are usually higher than a universal design |
| - | < | + | < |
| - | {{ : | + | {{ : |
| - | < | + | < |
| </ | </ | ||
| === NoSQL systems === | === NoSQL systems === | ||
| - | As the name suggests, the main characteristic is higher flexibility in data models, which overcomes the limitations of highly structured relational data models. NoSQL systems are usually distributed, | + | As the name suggests, the main characteristic is higher flexibility in data models, which overcomes the limitations of highly structured relational data models |
| It also provides a means for scalability out and up, enabling high future tolerance and resilience. A typical approach uses a key-value or key-document approach, where a unique key indexes incoming data blocks or documents (JSON, for instance). | It also provides a means for scalability out and up, enabling high future tolerance and resilience. A typical approach uses a key-value or key-document approach, where a unique key indexes incoming data blocks or documents (JSON, for instance). | ||
| - | Some other designs might extend the SQL data models by others – object models, graph models, or the mentioned key-value models, providing highly purpose-driven and, therefore, productive designs. However, the complexity of the design raises problems of data integrity as well as the complexity of maintenance. | + | Some other designs might extend the SQL data models by others – object models, graph models, or the mentioned key-value models, providing highly purpose-driven and, therefore, productive designs. However, the complexity of the design raises problems of data integrity as well as the complexity of maintenance |
| - | < | + | < |
| - | {{ : | + | {{ : |
| - | < | + | < |
| </ | </ | ||
