Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
en:iot-open:networking2:applicationnetworkprotocols [2023/11/22 13:07] pczekalskien:iot-open:networking2:applicationnetworkprotocols [2024/08/28 15:56] (current) pczekalski
Line 1: Line 1:
 ====== Application Protocols ====== ====== Application Protocols ======
 +{{:en:iot-open:czapka_b.png?50| General audience classification icon }}{{:en:iot-open:czapka_m.png?50| General audience classification icon }}{{:en:iot-open:czapka_e.png?50| General audience classification icon }}\\
 The host layers protocols include session (SES), presentation (PRES) and application (APP) levels. In particular, the APP (application) layer in regular Internet communication is dominated by the HTTP protocol and XML-related derivatives, e.g. SoAP. Also, the FTP protocol for file transfer is ubiquitous; it has existed since the beginning of the Internet. Most of them are somehow related to the textual presentation of the information. They're referenced as "WEB" protocols.\\ The host layers protocols include session (SES), presentation (PRES) and application (APP) levels. In particular, the APP (application) layer in regular Internet communication is dominated by the HTTP protocol and XML-related derivatives, e.g. SoAP. Also, the FTP protocol for file transfer is ubiquitous; it has existed since the beginning of the Internet. Most of them are somehow related to the textual presentation of the information. They're referenced as "WEB" protocols.\\
 Although advanced and more powerful IoT devices frequently use these protocols, this is problematic to implement in the constrained IoT devices world: even the simplest HTTP header occupies at least 24 + 8 + 8 + 31 bytes without payload! \\ Although advanced and more powerful IoT devices frequently use these protocols, this is problematic to implement in the constrained IoT devices world: even the simplest HTTP header occupies at least 24 + 8 + 8 + 31 bytes without payload! \\
-There is also a problem to cross firewall boundaries when communication between subnetworks of IoT devices is expected to occur. \\+There is also a problem of crossing firewall boundaries when communication between subnetworks of IoT devices is expected to occur. \\
 \\ \\
 Some IoT-designed protocols are reviewed below. Some IoT-designed protocols are reviewed below.
Line 23: Line 23:
 MQTT requires a centralised MQTT Broker located outside firewalls and NATs, where all clients connect, send and receive messages via the **publish/subscribe** model. The client can act as publisher and subscriber simultaneously. Figure {{ref>mqtt_broker}} presents a publish-subscribe model idea. MQTT requires a centralised MQTT Broker located outside firewalls and NATs, where all clients connect, send and receive messages via the **publish/subscribe** model. The client can act as publisher and subscriber simultaneously. Figure {{ref>mqtt_broker}} presents a publish-subscribe model idea.
 <figure mqtt_broker> <figure mqtt_broker>
-{{ :en:iot-open:communications_and_communicating_sut:mqtt_broker.png?200 | MQTT Broker, publishers and subscribers}}+{{ :en:iot-open:communications_and_communicating_sut:mqtt_broker.png?320 | MQTT Broker, publishers and subscribers}}
 <caption>MQTT Broker, Publishers and Subscribers</caption> <caption>MQTT Broker, Publishers and Subscribers</caption>
 </figure> </figure>
Line 58: Line 58:
 | RTU/# | | RTU/# |
  
-The payload (data) of the message is text as well, so if one needs to send binary data, it is necessary to encode it (e.g. using Base64 encoding).+The message'payload (data) is also text, so if binary data is to be sent, it must be encoded (e.g.using Base64 encoding).
  
 **MQTT Broker**\\ **MQTT Broker**\\
 MQTT Broker is a server for both publishers and subscribers. The connection is initiated from the client to the Broker, so assuming the Broker is located outside a firewall, it breaks its boundaries.\\ MQTT Broker is a server for both publishers and subscribers. The connection is initiated from the client to the Broker, so assuming the Broker is located outside a firewall, it breaks its boundaries.\\
-The Broker provides QoS (Quality of Service) and can retain message payload. There are three levels of MQTT Broker QoS (supplied in the message level).+The Broker provides QoS (Quality of Service) and can retain message payload. MQTT Broker QoS (supplied at the message level) has three levels.
   * Unacknowledged service: Ensures that MQTT message is delivered at most once to each subscriber.   * Unacknowledged service: Ensures that MQTT message is delivered at most once to each subscriber.
-  * Acknowledged service: Ensures message delivery at least once to every subscriber. The Broker expects acknowledgement to be sent from the subscriber. Otherwise, it retransmits data. +  * Acknowledged service: Ensures message delivery to every subscriber at least once. The Broker expects acknowledgement to be sent from the subscriber. Otherwise, it retransmits data. 
-  * Assured service: This is a two-step delivery of the message and ensures the message is delivered exactly once to every subscriber.+  * Assured service: This two-step message delivery ensures the transmission is delivered exactly once to every subscriber.
 Providing unique packet IDs in the MQTT frame is vital for Acknowledged and Assured services.\\ Providing unique packet IDs in the MQTT frame is vital for Acknowledged and Assured services.\\
  
-The DUP flag (byte 1, bit 3) represents information sent by the publisher if the message is a "first try" (0) or a retransmitted one (1). It is primarily for internal purposesand this flag is never propagated to the subscribers.+The DUP flag (byte 1, bit 3) represents information sent by the publisher, indicating whether the message is a "first try" (0) or a retransmitted one (1). This flag is primarily for internal purposes and is never propagated to the subscribers.
  
 MQTT offers a limited set of features (options): MQTT offers a limited set of features (options):
Line 79: Line 79:
 Interestingly, MQTT is a protocol used by Facebook Messenger ((https://www.facebook.com/notes/facebook-engineering/building-facebook-messenger/10150259350998920/)). It is also implemented natively in Microsoft Azure and Amazon Web Services (among many others). Interestingly, MQTT is a protocol used by Facebook Messenger ((https://www.facebook.com/notes/facebook-engineering/building-facebook-messenger/10150259350998920/)). It is also implemented natively in Microsoft Azure and Amazon Web Services (among many others).
  
-MQTT security is relatively weak. MQTT Broker can offer user and password verification sent in plain text. However, all communication between client and Broker may be encapsulated in SSLencrypted stream. +MQTT security is relatively weak. The MQTT Broker can offer user and password verification sent in plain text. However, all communication between the client and Broker may be encapsulated in an SSL-encrypted stream. 
  
 A short comparison of MQTT and HTTP protocols is presented in table {{ref>http_vs_mqtt}}. A short comparison of MQTT and HTTP protocols is presented in table {{ref>http_vs_mqtt}}.
Line 96: Line 96:
 == CoAP == == CoAP ==
  
-CoAP protocol (RFC7252) originates from the REST (Representational State Transfer). CoAP does not use a centralised server as MQTT does, but every single device "hosts" a server on its own to provide available resources to the clients asking for service offering distributed resources. CoAP uses UDP (compared to MQTT, which uses TCP) and is stateless; thus, it does not require memory for tracking the state. The CoAP implementation assumes every IoT device has a unique ID, and things can have multiple representations. It is intended to link "things" together using existing standard methods. It is resource-oriented (not document-oriented like HTTP/HTML) and designed for slow IoT networks with high packet loss. It also supports devices to be periodically offline.\\ +CoAP protocol (RFC7252) originates from the REST (Representational State Transfer). CoAP does not use a centralised server as MQTT does, but every single device "hosts" a server on its own to provide available resources to the clients asking for service offering distributed resources. CoAP uses UDP (compared to MQTT, which uses TCP) and is stateless; thus, it does not require memory to track the state. The CoAP implementation assumes every IoT device has a unique ID, and things can have multiple representations. It is intended to link "things" together using existing standard methods. It is resource-oriented (not document-oriented like HTTP/HTML) and designed for slow IoT networks with high packet loss. It also supports devices that are periodically offline.\\ 
 CoAP uses URIs to address services: CoAP uses URIs to address services:
   * %%coap://<host>[:<port>]/<path>[?<query>]%% to access a service/resource,   * %%coap://<host>[:<port>]/<path>[?<query>]%% to access a service/resource,
Line 125: Line 125:
 Sample CoAP message exchange scenarios between client and server are presented (two per image) in figure {{ref>CoAP1}} and figure {{ref>CoAP2}}. Sample CoAP message exchange scenarios between client and server are presented (two per image) in figure {{ref>CoAP1}} and figure {{ref>CoAP2}}.
 <figure CoAP1> <figure CoAP1>
-{{ :en:iot-open:communications_and_communicating_sut:grafika8.png?400 | CoAP scenario 1: confirmable with time delay payload answer (0 × 70) and immediate payload answer (0 × 71)}}+{{ :en:iot-open:communications_and_communicating_sut:grafika8.png?620 | CoAP scenario 1: confirmable with time delay payload answer (0 × 70) and immediate payload answer (0 × 71)}}
 <caption>CoAP scenario 1: confirmable with time delay payload answer (0 × 70) and immediate payload answer (0 × 71)</caption> <caption>CoAP scenario 1: confirmable with time delay payload answer (0 × 70) and immediate payload answer (0 × 71)</caption>
 </figure> </figure>
 <figure CoAP2> <figure CoAP2>
-{{ :en:iot-open:communications_and_communicating_sut:grafika7.png?400 | CoAP scenario 2: unrecognized request (0 × 72) and non-confirmable request (0 × 73)}}+{{ :en:iot-open:communications_and_communicating_sut:grafika7.png?620 | CoAP scenario 2: unrecognized request (0 × 72) and non-confirmable request (0 × 73)}}
 <caption>CoAP scenario 2: unrecognized request (0 × 72) and non-confirmable request (0 × 73)</caption> <caption>CoAP scenario 2: unrecognized request (0 × 72) and non-confirmable request (0 × 73)</caption>
 </figure> </figure>
Line 138: Line 138:
  
 == AMQP == == AMQP ==
-In its principles, the AMQP (Advanced Message Queuing Protocol) somehow recalls MQTT: it is message-oriented and uses a central broker. There are data publishers and consumers (that, in the case of the MQTT, are called subscribers). Messages are routed from publishers to the broker, where they hit so-called exchanges, and then they are copied to the queues (0, 1 or more) that the consumer can later read from. A diagram of the message's flow is present in the figure {{ref>amqp1}}.+In its principles, the AMQP (Advanced Message Queuing Protocol) somehow recalls MQTT: it is message-oriented and uses a central broker. There are data publishers and consumers (that, in the case of the MQTT, are called subscribers). Messages are routed from publishers to the Broker, where they hit so-called exchanges, and then they are copied to the queues (0, 1 or more) from which the consumer can later read. A diagram of the message's flow is present in the figure {{ref>amqp1}}.
 <figure amqp1> <figure amqp1>
-{{ :en:iot-open:networking2:amqp-page-1.png?560 | AMQP protocol messages flow}}+{{ :en:iot-open:networking2:amqp-page-1.png?620 | AMQP protocol messages flow}}
 <caption>AMQP protocol messages flow</caption> <caption>AMQP protocol messages flow</caption>
 </figure> </figure>
- 
 AMQP uses TCP/IP. AMQP is intended to work in non-reliable networks; thus, the protocol has a message acknowledgement mechanism to ensure delivery. A message is removed from the queue only if it has been acknowledged. Besides acknowledged delivery, it is also possible to use an unacknowledged one that does not involve acknowledgements. If a message cannot be routed (for any reason), it can be returned to the publisher, dropped or placed in the "dead letter queue". The behaviour is defined along with a message. Opposite to MQTT, in AMQP protocol, the connection status is unknown; thus, there is no mechanism to let other devices know that some node has disconnected, such as the last will in MQTT.  AMQP uses TCP/IP. AMQP is intended to work in non-reliable networks; thus, the protocol has a message acknowledgement mechanism to ensure delivery. A message is removed from the queue only if it has been acknowledged. Besides acknowledged delivery, it is also possible to use an unacknowledged one that does not involve acknowledgements. If a message cannot be routed (for any reason), it can be returned to the publisher, dropped or placed in the "dead letter queue". The behaviour is defined along with a message. Opposite to MQTT, in AMQP protocol, the connection status is unknown; thus, there is no mechanism to let other devices know that some node has disconnected, such as the last will in MQTT. 
  
 **Queues**\\ **Queues**\\
-AMQP is a programmable protocol, so bindings are not defined by the broker but rather by the publisher. Queues are also created on-demand via external actors (mostly consumers). Routing via bindings is provided with a message, and the broker analyses it to provide correct message handling and delivery.\\ +AMQP is a programmable protocol, so bindings are not defined by the Broker but rather by the publisher. Queues are also created on-demand via external actors (mostly consumers). Routing via bindings is provided with a message, and the Broker analyses it to provide correct message handling and delivery.\\ 
-Consumers can subscribe to the exchange and define a queue. Bindings then act as filters so they receive only selected messages. A single queue is intended to handle one consumer, but there does exist a possibility to let many consumers use a single queue in the round-robin model.+Consumers can subscribe to the exchange and define a queue. Bindings then act as filters so they receive only selected messages. A single queue is intended to handle one consumer, but there is a possibility of letting many consumers use a single queue in the round-robin model.
 As in the protocol version 0.9, queues have the following properties: As in the protocol version 0.9, queues have the following properties:
   * Name,   * Name,
Line 161: Line 160:
   * Direct Exchange (its name is empty string or ''amq.direct''); The default, Direct Exchange, has a special feature that automatically creates and binds new queues, where the queue's name is the same as a routing key; thus, it is ideal for unicast communication. Assuming the queue's name is ''K'' (there can be more than one) and there comes a message with routing key ''N'', the message is routed only to those queues, where ''K=N'' (figure {{ref>amqp2}}).   * Direct Exchange (its name is empty string or ''amq.direct''); The default, Direct Exchange, has a special feature that automatically creates and binds new queues, where the queue's name is the same as a routing key; thus, it is ideal for unicast communication. Assuming the queue's name is ''K'' (there can be more than one) and there comes a message with routing key ''N'', the message is routed only to those queues, where ''K=N'' (figure {{ref>amqp2}}).
 <figure amqp2> <figure amqp2>
-{{ :en:iot-open:networking2:amqp-page-2.png?400 | Direct Exchange working principles}}+{{ :en:iot-open:networking2:amqp-page-2.png?230 | Direct Exchange working principles}}
 <caption>Direct Exchange working principles</caption> <caption>Direct Exchange working principles</caption>
 </figure> </figure>
   * Fanout Exchange (''amq.fanout''); In the Fanout Exchange, all messages are routed to all queues bound to the Fanout Exchange, regardless of their routing key. They help broadcast the information (figure {{ref>amqp3}}).   * Fanout Exchange (''amq.fanout''); In the Fanout Exchange, all messages are routed to all queues bound to the Fanout Exchange, regardless of their routing key. They help broadcast the information (figure {{ref>amqp3}}).
 <figure amqp3> <figure amqp3>
-{{ :en:iot-open:networking2:amqp-page-3.png?400 | Fanout Exchange working principles}}+{{ :en:iot-open:networking2:amqp-page-3.png?230 | Fanout Exchange working principles}}
 <caption>Fanout Exchange working principles</caption> <caption>Fanout Exchange working principles</caption>
 </figure> </figure>
   * Topic Exchange (''amq.topic''); Topic Exchange works similarly to MQTT topic subscriptions: an AMQP queue bound to the Topic Exchange defines a pattern rather than the fixed name, and messages with the matching routing key are copied to the queue ({{ref>amqp4}}). It is great for multicasting.   * Topic Exchange (''amq.topic''); Topic Exchange works similarly to MQTT topic subscriptions: an AMQP queue bound to the Topic Exchange defines a pattern rather than the fixed name, and messages with the matching routing key are copied to the queue ({{ref>amqp4}}). It is great for multicasting.
 <figure amqp4> <figure amqp4>
-{{ :en:iot-open:networking2:amqp-copy_of_page-5.png?400 | Topic Exchange working principles}}+{{ :en:iot-open:networking2:amqp-copy_of_page-5.png?230 | Topic Exchange working principles}}
 <caption>Topic Exchange working principles</caption> <caption>Topic Exchange working principles</caption>
 </figure> </figure>
   * Headers Exchange (''amq.match''); This exchange uses message headers for routing instead of routing keys. The routing key is ignored, and it is possible to bind a queue to the Headers Exchange using more than one header. It is also possible to specify whether it is enough to find a single matching among many conditions or all must be satisfied. Sample routing is present in the figure {{ref>amqp5}}: Q1 requires one of the conditions to be satisfied, while Q2 requires all of the conditions to be satisfied to execute a binding and route a message.   * Headers Exchange (''amq.match''); This exchange uses message headers for routing instead of routing keys. The routing key is ignored, and it is possible to bind a queue to the Headers Exchange using more than one header. It is also possible to specify whether it is enough to find a single matching among many conditions or all must be satisfied. Sample routing is present in the figure {{ref>amqp5}}: Q1 requires one of the conditions to be satisfied, while Q2 requires all of the conditions to be satisfied to execute a binding and route a message.
 <figure amqp5> <figure amqp5>
-{{ :en:iot-open:networking2:amqp-page-4.png?400 | Headers Exchange working principles}}+{{ :en:iot-open:networking2:amqp-page-4.png?230 | Headers Exchange working principles}}
 <caption>Headers Exchange working principles</caption> <caption>Headers Exchange working principles</caption>
 </figure> </figure>
Line 188: Line 187:
  
 **Messages**\\ **Messages**\\
-The essential, immutable AMQP frame size is 8 bytes, and the payload is up to 2GB. Besides the header frame, messages have several virtually freely definable attributes, but the broker uses some predefined ones. Common attributes cover:+The essential, immutable AMQP frame size is 8 bytes, and the payload is up to 2GB. Besides the header frame, messages have several virtually freely definable attributes, but the Broker uses some predefined ones. Common attributes cover:
   * content type,   * content type,
   * content encoding,   * content encoding,
Line 206: Line 205:
 </note> </note>
  
-The address of the broker is referenced with URI, similar to the CoAP (e.g.):+The address of the Broker is referenced with URI, similar to the CoAP (e.g.):
   *%%amqp://<user>:<password>@<host>[:<port>]/<path>%% for raw connections,   *%%amqp://<user>:<password>@<host>[:<port>]/<path>%% for raw connections,
   *"amqps" for TLS/SSL secure connections.   *"amqps" for TLS/SSL secure connections.
en/iot-open/networking2/applicationnetworkprotocols.1700658462.txt.gz · Last modified: 2023/11/22 13:07 by pczekalski
CC Attribution-Share Alike 4.0 International
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0