literature review for iot

The Internet of Things: : Review and theoretical framework

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options.

• Jha N Guo H Lau V (2024) Analog Product Coding for Over-the-Air Aggregation Over Burst-Sparse Interference Multiple-Access Channels IEEE Transactions on Signal Processing 10.1109/TSP.2023.3338046 72 (157-172) Online publication date: 1-Jan-2024 https://dl.acm.org/doi/10.1109/TSP.2023.3338046
• Tusa F Clayman S Buzachis A Fazio M (2024) Microservices and serverless functions—lifecycle, performance, and resource utilisation of edge based real-time IoT analytics Future Generation Computer Systems 10.1016/j.future.2024.02.006 155 :C (204-218) Online publication date: 1-Jun-2024 https://dl.acm.org/doi/10.1016/j.future.2024.02.006
• Mir F Meziane F (2024) Novel adaptive DCOPA using dynamic weighting for vector of performances indicators optimization of IoT networks Expert Systems with Applications: An International Journal 10.1016/j.eswa.2024.123212 247 :C Online publication date: 1-Aug-2024 https://dl.acm.org/doi/10.1016/j.eswa.2024.123212
• Show More Cited By

Index Terms

Computer systems organization

Recommendations

Cyberentity security in the internet of things.

A proposed Internet of Things system architecture offers a solution to the broad array of challenges researchers face in terms of general system security, network security, and application security.

The internet of things: a survey

In recent year, the Internet of Things (IoT) has drawn significant research attention. IoT is considered as a part of the Internet of the future and will comprise billions of intelligent communicating `things'. The future of the Internet will consist of ...

Internet of Things security

The Internet of things (IoT) has recently become an important research topic because it integrates various sensors and objects to communicate directly with one another without human intervention. The requirements for the large-scale deployment of the ...

Information

Published in.

Pergamon Press, Inc.

United States

Publication History

Author tags.

• Internet of Things
• IoT literature review
• IoT priorities
• IoT challenges
• IoT theoretical framework
• Review-article

Contributors

Other metrics, bibliometrics, article metrics.

• 34 Total Citations View Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0
• Pirbhulal S Chockalingam S Shukla A Abie H (2024) IoT cybersecurity in 5G and beyond: a systematic literature review International Journal of Information Security 10.1007/s10207-024-00865-5 23 :4 (2827-2879) Online publication date: 1-Aug-2024 https://dl.acm.org/doi/10.1007/s10207-024-00865-5
• BahraniPour F Ebrahimi Mood S Farshi M (2024) Energy-delay aware request scheduling in hybrid Cloud and Fog computing using improved multi-objective CS algorithm Soft Computing - A Fusion of Foundations, Methodologies and Applications 10.1007/s00500-023-09381-5 28 :5 (4037-4050) Online publication date: 1-Mar-2024 https://dl.acm.org/doi/10.1007/s00500-023-09381-5
• Zhang H Zhang F Gong B Zhang X Zhu Y (2023) The Optimization of Supply Chain Financing for Bank Green Credit Using Stackelberg Game Theory in Digital Economy Under Internet of Things Journal of Organizational and End User Computing 10.4018/JOEUC.318474 35 :3 (1-16) Online publication date: 24-Feb-2023 https://dl.acm.org/doi/10.4018/JOEUC.318474
• Li H Chen Y Li K Wang C Chen B (2023) Transportation Internet: A Sustainable Solution for Intelligent Transportation Systems IEEE Transactions on Intelligent Transportation Systems 10.1109/TITS.2023.3270749 24 :12 (15818-15829) Online publication date: 1-Dec-2023 https://dl.acm.org/doi/10.1109/TITS.2023.3270749
• Cruz M Abbade L Lorenz P Mafra S Rodrigues J (2023) Detecting Compromised IoT Devices Through XGBoost IEEE Transactions on Intelligent Transportation Systems 10.1109/TITS.2022.3187252 24 :12 (15392-15399) Online publication date: 1-Dec-2023 https://dl.acm.org/doi/10.1109/TITS.2022.3187252
• Liu R Liang J Cai C Zhou W (2023) Adaptive multimode transmission in wireless sensor networks with energy harvesting Physical Communication 10.1016/j.phycom.2023.102044 58 :C Online publication date: 1-Jun-2023 https://dl.acm.org/doi/10.1016/j.phycom.2023.102044
• Rodriguez-Conde I Campos C Fdez-Riverola F (2023) Cloud-assisted collaborative inference of convolutional neural networks for vision tasks on resource-constrained devices Neurocomputing 10.1016/j.neucom.2023.126835 560 :C Online publication date: 1-Dec-2023 https://dl.acm.org/doi/10.1016/j.neucom.2023.126835

View options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

• Survey Paper
• Open access
• Published: 09 December 2019

Internet of Things is a revolutionary approach for future technology enhancement: a review

• Sachin Kumar   ORCID: orcid.org/0000-0003-3949-0302 1 ,
• Prayag Tiwari 2 &
• Mikhail Zymbler 1  

Journal of Big Data volume  6 , Article number:  111 ( 2019 ) Cite this article

264k Accesses

572 Citations

44 Altmetric

Metrics details

Internet of Things (IoT) is a new paradigm that has changed the traditional way of living into a high tech life style. Smart city, smart homes, pollution control, energy saving, smart transportation, smart industries are such transformations due to IoT. A lot of crucial research studies and investigations have been done in order to enhance the technology through IoT. However, there are still a lot of challenges and issues that need to be addressed to achieve the full potential of IoT. These challenges and issues must be considered from various aspects of IoT such as applications, challenges, enabling technologies, social and environmental impacts etc. The main goal of this review article is to provide a detailed discussion from both technological and social perspective. The article discusses different challenges and key issues of IoT, architecture and important application domains. Also, the article bring into light the existing literature and illustrated their contribution in different aspects of IoT. Moreover, the importance of big data and its analysis with respect to IoT has been discussed. This article would help the readers and researcher to understand the IoT and its applicability to the real world.

Introduction

The Internet of Things (IoT) is an emerging paradigm that enables the communication between electronic devices and sensors through the internet in order to facilitate our lives. IoT use smart devices and internet to provide innovative solutions to various challenges and issues related to various business, governmental and public/private industries across the world [ 1 ]. IoT is progressively becoming an important aspect of our life that can be sensed everywhere around us. In whole, IoT is an innovation that puts together extensive variety of smart systems, frameworks and intelligent devices and sensors (Fig.  1 ). Moreover, it takes advantage of quantum and nanotechnology in terms of storage, sensing and processing speed which were not conceivable beforehand [ 2 ]. Extensive research studies have been done and available in terms of scientific articles, press reports both on internet and in the form of printed materials to illustrate the potential effectiveness and applicability of IoT transformations. It could be utilized as a preparatory work before making novel innovative business plans while considering the security, assurance and interoperability.

General architecture of IoT

A great transformation can be observed in our daily routine life along with the increasing involvement of IoT devices and technology. One such development of IoT is the concept of Smart Home Systems (SHS) and appliances that consist of internet based devices, automation system for homes and reliable energy management system [ 3 ]. Besides, another important achievement of IoT is Smart Health Sensing system (SHSS). SHSS incorporates small intelligent equipment and devices to support the health of the human being. These devices can be used both indoors and outdoors to check and monitor the different health issues and fitness level or the amount of calories burned in the fitness center etc. Also, it is being used to monitor the critical health conditions in the hospitals and trauma centers as well. Hence, it has changed the entire scenario of the medical domain by facilitating it with high technology and smart devices [ 4 , 5 ]. Moreover, IoT developers and researchers are actively involved to uplift the life style of the disabled and senior age group people. IoT has shown a drastic performance in this area and has provided a new direction for the normal life of such people. As these devices and equipment are very cost effective in terms of development cost and easily available within a normal price range, hence most of the people are availing them [ 6 ]. Thanks to IoT, as they can live a normal life. Another important aspect of our life is transportation. IoT has brought up some new advancements to make it more efficient, comfortable and reliable. Intelligent sensors, drone devices are now controlling the traffic at different signalized intersections across major cities. In addition, vehicles are being launched in markets with pre-installed sensing devices that are able to sense the upcoming heavy traffic congestions on the map and may suggest you another route with low traffic congestion [ 7 ]. Therefore IoT has a lot to serve in various aspects of life and technology. We may conclude that IoT has a lot of scope both in terms of technology enhancement and facilitate the humankind.

IoT has also shown its importance and potential in the economic and industrial growth of a developing region. Also, in trade and stock exchange market, it is being considered as a revolutionary step. However, security of data and information is an important concern and highly desirable, which is a major challenging issue to deal with [ 5 ]. Internet being a largest source of security threats and cyber-attacks has opened the various doors for hackers and thus made the data and information insecure. However, IoT is committed to provide the best possible solutions to deal with security issues of data and information. Hence, the most important concern of IoT in trade and economy is security. Therefore, the development of a secure path for collaboration between social networks and privacy concerns is a hot topic in IoT and IoT developers are working hard for this.

The remaining part of the article is organized as follows: “ Literature survey ” section will provide state of art on important studies that addressed various challenges and issues in IoT. “ IoT architecture and technologies ” section discussed the IoT functional blocks, architecture in detail. In “ Major key issues and challenges of IoT ” section, important key issues and challenges of IoT is discussed. “ Major IoT applications ” section provides emerging application domains of IoT. In “ Importance of big data analytics in IoT ” section, the role and importance of big data and its analysis is discussed. Finally, the article concluded in “ Conclusions ” section.

Literature survey

IoT has a multidisciplinary vision to provide its benefit to several domains such as environmental, industrial, public/private, medical, transportation etc. Different researchers have explained the IoT differently with respect to specific interests and aspects. The potential and power of IoT can be seen in several application domains. Figure  2 illustrates few of the application domains of IoTs potentials.

Some of the potential application domains of IoT

Various important IoT projects have taken charge over the market in last few years. Some of the important IoT projects that have captured most of the market are shown in Fig.  3 . In Fig.  3 , a global distribution of these IoT projects is shown among American, European and Asia/Pacific region. It can be seen that American continent are contributing more in the health care and smart supply chain projects whereas contribution of European continent is more in the smart city projects [ 8 ].

Global distribution of IoT projects among America (USA, South America and Canada), Europe and APAC (Asia and Pacific region) [ 8 ]

Figure  4 , illustrates the global market share of IoT projects worldwide [ 8 ]. It is evident that industry, smart city, smart energy and smart vehicle based IoT projects have a big market share in comparison to others.

Global share of IoT projects across the world [ 8 ]

Smart city is one of the trendy application areas of IoT that incorporates smart homes as well. Smart home consists of IoT enabled home appliances, air-conditioning/heating system, television, audio/video streaming devices, and security systems which are communicating with each other in order to provide best comfort, security and reduced energy consumption. All this communication takes place through IoT based central control unit using Internet. The concept of smart city gained popularity in the last decade and attracted a lot of research activities [ 9 ]. The smart home business economy is about to cross the 100 billion dollars by 2022 [ 10 ]. Smart home does not only provide the in-house comfort but also benefits the house owner in cost cutting in several aspects i.e. low energy consumption will results in comparatively lower electricity bill. Besides smart homes, another category that comes within smart city is smart vehicles. Modern cars are equipped with intelligent devices and sensors that control most of the components from the headlights of the car to the engine [ 11 ]. The IoT is committed towards developing a new smart car systems that incorporates wireless communication between car-to-car and car-to-driver to ensure predictive maintenance with comfortable and safe driving experience [ 12 ].

Khajenasiri et al. [ 10 ] performed a survey on the IoT solutions for smart energy control to benefit the smart city applications. They stated that at present IoT has been deployed in very few application areas to serve the technology and people. The scope of IoT is very wide and in near future IoT is able to capture almost all application areas. They mentioned that energy saving is one of the important part of the society and IoT can assist in developing a smart energy control system that will save both energy and money. They described an IoT architecture with respect to smart city concept. The authors also discussed that one of the challenging task in achieving this is the immaturity of IoT hardware and software. They suggested that these issues must be resolved to ensure a reliable, efficient and user friendly IoT system.

Alavi et al. [ 13 ] addressed the urbanization issue in the cities. The movement of people from rural to urban atmosphere resulting in growing population of the cities. Therefore, there is a need to provide smart solutions for mobility, energy, healthcare and infrastructure. Smart city is one of the important application areas for IoT developers. It explores several issues such as traffic management, air quality management, public safety solutions, smart parking, smart lightning and smart waste collection (Fig.  5 ). They mentioned that IoT is working hard to tackle these challenging issues. The need for improved smart city infrastructure with growing urbanization has opened the doors for entrepreneurs in the field of smart city technologies. The authors concluded that IoT enabled technology is very important for the development of sustainable smart cities.

Potential IoT application areas for smart cities

Another important issue of IoT that requires attention and a lot of research is security and privacy. Weber [ 14 ] focused on these issues and suggested that a private organization availing IoT must incorporate data authentication, access control, resilience to attacks and client privacy into their business activities that would be an additional advantage. Weber suggested that in order to define global security and privacy issues, IoT developers must take into account the geographical limitations of the different countries. A generic framework needs to be designed to fit the global needs in terms of privacy and security. It is highly recommended to investigate and recognize the issues and challenges in privacy and security before developing the full fledge working IoT framework.

Later, Heer et al. [ 15 ] came up with a security issue in IP based IoT system. They mentioned that internet is backbone for the communication among devices that takes place in an IoT system. Therefore, security issues in IP based IoT systems are an important concern. In addition, security architecture should be designed considering the life cycle and capabilities of any object in the IoT system. It also includes the involvement of the trusted third party and the security protocols. The security architecture with scalability potential to serve the small-scale to large-scale things in IoT is highly desirable. The study pointed out that IoT gave rise to a new way of communication among several things across the network therefore traditional end to end internet protocol are not able to provide required support to this communication. Therefore, new protocols must be designed considering the translations at the gateways to ensure end-to-end security. Moreover, all the layers responsible for communication has their own security issues and requirements. Therefore, satisfying the requirements for one particular layers will leave the system into a vulnerable state and security should be ensured for all the layers.

Authentication and access control is another issue in IoT that needs promising solutions to strengthen the security. Liu et al. [ 16 ] brought up a solution to handle authentication and access control. Authentication is very important to verify the communicating parties to prevent the loss of confidential information. Liu et al. [ 16 ] provided an authentication scheme based on Elliptic Curve Cryptosystem and verified it on different security threats i.e. eavesdropping, man-in-the-middle attack, key control and replay attack. They claimed that there proposed schemes are able to provide better authentication and access control in IoT based communication. Later, Kothmayr et al. [ 17 ] proposed a two-way authentication scheme based of datagram transport layer security (DTLS) for IoT. The attackers over the internet are always active to steal the secured information. The proposed approach are able to provide message security, integrity, authenticity and confidentiality, memory overhead and end-to-end latency in the IoT based communication network.

Li et al. [ 18 ] proposed a dynamic approach for data centric IoT applications with respect to cloud platforms. The need of an appropriate device, software configuration and infrastructure requires efficient solutions to support massive amount of IoT applications that are running on cloud platforms. IoT developers and researchers are actively engaged in developing solutions considering both massive platforms and heterogeneous nature of IoT objects and devices. Olivier et al. [ 19 ] explained the concept of software defined networking (SDN) based architecture that performs well even if a well-defined architecture is not available. They proposed that SDN based security architecture is more flexible and efficient for IoT.

Luk et al. [ 20 ] stated that the main task of a secure sensor network (SSN) is to provide data privacy, protection from replay attacks and authentication. They discussed two popular SSN services namely TinySec [ 21 ] and ZigBee [ 22 ]. They mentioned that although both the SSN services are efficient and reliable, however, ZigBee is comparatively provides higher security but consumes high energy whereas TinySec consumes low energy but not as highly secured as ZigBee. They proposed another architecture MiniSec to support high security and low energy consumption and demonstrated its performance for the Telos platform. Yan et al. [ 23 ] stated that trust management is an important issue in IoT. Trust management helps people to understand and trust IoT services and applications without worrying about uncertainty issues and risks [ 24 ]. They investigated different issues in trust management and discussed its importance with respect to IoT developers and users.

Noura et al. [ 25 ] stated the importance of interoperability in IoT as it allows integration of devices, services from different heterogeneous platforms to provide the efficient and reliable service. Several other studies focused on the importance of interoperability and discussed several challenges that interoperability issue is facing in IoT [ 26 , 27 , 28 ]. Kim et al. [ 29 ] addressed the issue of climate change and proposed an IoT based ecological monitoring system. They mentioned that existing approaches are time consuming and required a lot of human intervention. Also, a routine visit is required to collect the information from the sensors installed at the site under investigation. Also, some information remained missing which leads to not highly accurate analysis. Therefore, IoT based framework is able to solve this problem and can provide high accuracy in analysis and prediction. Later, Wang et al. [ 30 ] shows their concern for domestic waste water treatment. They discussed several deficiencies in the process of waste water treatment and dynamic monitoring system and suggested effective solutions based on IoT. They stated that IoT can be very effective in the waste water treatment and process monitoring.

Agriculture is one of the important domain around the world. Agriculture depends on several factors i.e. geographical, ecological etc. Qiu et al. [ 31 ] stated that technology that is being used for ecosystem control is immature with low intelligence level. They mentioned that it could be a good application area for IoT developers and researchers.

Qiu et al. [ 31 ] proposed an intelligent monitoring platform framework for facility agriculture ecosystem based on IoT that consists of four layer mechanism to manage the agriculture ecosystem. Each layer is responsible for specific task and together the framework is able to achieve a better ecosystem with reduced human intervention.

Another important concern around the world is climate change due to global warming. Fang et al. [ 32 ] introduced an integrated information system (IIS) that integrates IoT, geo-informatics, cloud computing, global positioning system (GPS), geographical information system (GIS) and e-science in order to provide an effective environmental monitoring and control system. They mentioned that the proposed IIS provides improved data collection, analysis and decision making for climate control. Air pollution is another important concern worldwide. Various tools and techniques are available to air quality measures and control. Cheng et al. [ 33 ] proposed AirCloud which is a cloud based air quality and monitoring system. They deployed AirCloud and evaluated its performance using 5 months data for the continuous duration of 2 months.

Temglit et al. [ 34 ] considered Quality of Service (QoS) as an important challenge and a complex task in evaluation and selection of IoT devices, protocols and services. QoS is very important criteria to attract and gain trust of users towards IoT services and devices. They came up with an interesting distributed QoS selection approach. This approach was based on distributed constraint optimization problem and multi-agent paradigm. Further, the approach was evaluated based on several experiments under realistic distributed environments. Another important aspect of IoT is its applicability to the environmental and agriculture standards. Talavera et al. [ 35 ] focused in this direction and presented the fundamental efforts of IoT for agro-industrial and environmental aspects in a survey study. They mentioned that the efforts of IoT in these areas are noticeable. IoT is strengthening the current technology and benefiting the farmers and society. Jara et al. [ 36 ] discussed the importance of IoT based monitoring of patients health. They suggested that IoT devices and sensors with the help of internet can assist health monitoring of patients. They also proposed a framework and protocol to achieve their objective. Table 1 provides a summary of the important studies and the direction of research with a comparison of studies on certain evaluation parameters.

IoT architecture and technologies

The IoT architecture consists of five important layers that defines all the functionalities of IoT systems. These layers are perception layer, network layer, middleware layer, application layer, business layer. At the bottom of IoT architecture, perception layer exists that consists of physical devices i.e. sensors, RFID chips, barcodes etc. and other physical objects connected in IoT network. These devices collects information in order to deliver it to the network layer. Network layer works as a transmission medium to deliver the information from perception layer to the information processing system. This transmission of information may use any wired/wireless medium along with 3G/4G, Wi-Fi, Bluetooth etc. Next level layer is known as middleware layer. The main task of this layer is to process the information received from the network layer and make decisions based on the results achieved from ubiquitous computing. Next, this processed information is used by application layer for global device management. On the top of the architecture, there is a business layer which control the overall IoT system, its applications and services. The business layer visualizes the information and statistics received from the application layer and further used this knowledge to plan future targets and strategies. Furthermore, the IoT architectures can be modified according to the need and application domain [ 19 , 20 , 37 ]. Besides layered framework, IoT system consists of several functional blocks that supports various IoT activities such as sensing mechanism, authentication and identification, control and management [ 38 ]. Figure  6 illustrates such functional blocks of IoT architecture.

A generic function module of IoT system

There are several important functional blocks responsible for I/O operations, connectivity issues, processing, audio/video monitoring and storage management. All these functional block together incorporates an efficient IoT system which are important for optimum performance. Although, there are several reference architectures proposed with the technical specifications, but these are still far from the standard architecture that is suitable for global IoT [ 39 ]. Therefore, a suitable architecture is still needsvk to be designed that could satisfy the global IoT needs. The generic working structure of IoT system is shown in Fig.  7 . Figure  7 shows a dependency of IoT on particular application parameters. IoT gateways have an important role in IoT communication as it allows connectivity between IoT servers and IoT devices related to several applications [ 40 ].

Working structure of IoT

Scalability, modularity, interoperability and openness are the key design issues for an efficient IoT architecture in a heterogenous environment. The IoT architecture must be designed with an objective to fulfil the requirements of cross domain interactions, multi-system integration with the potential of simple and scalable management functionalities, big data analytics and storage, and user friendly applications. Also, the architecture should be able to scaleup the functionality and add some intelligence and automation among the IoT devices in the system.

Moreover, increasing amount of massive data being generated through the communication between IoT sensors and devices is a new challenge. Therefore, an efficient architecture is required to deal with massive amount of streaming data in IoT system. Two popular IoT system architectures are cloud and fog/edge computing that supports with the handling, monitoring and analysis of huge amount of data in IoT systems. Therefore, a modern IoT architecture can be defined as a 4 stage architecture as shown in Fig.  8 .

Four stage IoT architecture to deal with massive data

In stage 1 of the architecture, sensors and actuators plays an important role. Real world is comprised of environment, humans, animals, electronic gadgets, smart vehicles, and buildings etc. Sensors detect the signals and data flow from these real world entities and transforms into data which could further be used for analysis. Moreover, actuators is able to intervene the reality i.e. to control the temperature of the room, to slow down the vehicle speed, to turn off the music and light etc. Therefore, stage 1 assist in collecting data from real world which could be useful for further analysis. Stage 2 is responsible to collaborate with sensors and actuators along with gateways and data acquisition systems. In this stage, massive amount of data generated in stage 1 is aggregated and optimized in a structured way suitable for processing. Once the massive amount of data is aggregated and structured then it is ready to be passed to stage 3 which is edge computing. Edge computing can be defined as an open architecture in distributed fashion which allows use of IoT technologies and massive computing power from different locations worldwide. It is very powerful approach for streaming data processing and thus suitable for IoT systems. In stage 3, edge computing technologies deals with massive amount of data and provides various functionalities such as visualization, integration of data from other sources, analysis using machine learning methods etc. The last stage comprises of several important activities such as in depth processing and analysis, sending feedback to improve the precision and accuracy of the entire system. Everything at this stage will be performed on cloud server or data centre. Big data framework such as Hadoop and Spark may be utilized to handle this large streaming data and machine learning approaches can be used to develop better prediction models which could help in a more accurate and reliable IoT system to meet the demand of present time.

Major key issues and challenges of IoT

The involvement of IoT based systems in all aspects of human lives and various technologies involved in data transfer between embedded devices made it complex and gave rise to several issues and challenges. These issues are also a challenge for the IoT developers in the advanced smart tech society. As technology is growing, challenges and need for advanced IoT system is also growing. Therefore, IoT developers need to think of new issues arising and should provide solutions for them.

Security and privacy issues

One of the most important and challenging issues in the IoT is the security and privacy due to several threats, cyber attacks, risks and vulnerabilities [ 41 ]. The issues that give rise to device level privacy are insufficient authorization and authentication, insecure software, firmware, web interface and poor transport layer encryption [ 42 ]. Security and privacy issues are very important parameters to develop confidence in IoT Systems with respect to various aspects [ 43 ]. Security mechanisms must be embedded at every layer of IoT architecture to prevent security threats and attacks [ 23 ]. Several protocols are developed and efficiently deployed on every layer of communication channel to ensure the security and privacy in IoT based systems [ 44 , 45 ]. Secure Socket Layer (SSL) and Datagram Transport Layer Security (DTLS) are one of the cryptographic protocols that are implemented between transport and application layer to provide security solutions in various IoT systems [ 44 ]. However, some IoT applications require different methods to ensure the security in communication between IoT devices. Besides this, if communication takes place using wireless technologies within the IoT system, it becomes more vulnerable to security risks. Therefore, certain methods should be deployed to detect malicious actions and for self healing or recovery. Privacy on the other hand is another important concern which allows users to feel secure and comfortable while using IoT solutions. Therefore, it is required to maintain the authorization and authentication over a secure network to establish the communication between trusted parties [ 46 ]. Another issue is the different privacy policies for different objects communicating within the IoT system. Therefore, each object should be able to verify the privacy policies of other objects in IoT system before transmitting the data.

Interoperability/standard issues

Interoperability is the feasibility to exchange the information among different IoT devices and systems. This exchange of information does not rely on the deployed software and hardware. The interoperability issue arises due to the heterogeneous nature of different technology and solutions used for IoT development. The four interoperability levels are technical, semantic, syntactic and organizational [ 47 ]. Various functionalities are being provided by IoT systems to improve the interoperability that ensures communication between different objects in a heterogeneous environment. Additionally, it is possible to merge different IoT platforms based on their functionalities to provide various solutions for IoT users [ 48 ]. Considering interoperability an important issue, researchers approved several solutions that are also know as interoperability handling approaches [ 49 ]. These solutions could be adapaters/gateways based, virtual networks/overlay based, service oriented architecture based etc. Although interoperability handling approaches ease some pressure on IoT systems but there are still certain challenges remain with interoperability that could be a scope for future studies [ 25 ].

Ethics, law and regulatory rights

Another issue for IoT developers is the ethics, law and regulatory rights. There are certain rules and regulations to maintain the standard, moral values and to prevent the people from violating them. Ethics and law are very similar term with the only difference is that ethics are standards that people believes and laws are certain restrictions decided by the government. However, both ethics and laws are designed to maintain the standard, quality and prevent people from illegal use. With the development of IoT, several real life problems are solved but it has also given rise to critical ethical and legal challenges [ 50 ]. Data security, privacy protection, trust and safety, data usability are some of those challenges. It has also been observed that majority of IoT users are supporting government norms and regulations with respect to data protection, privacy and safety due to the lack of trust in IoT devices. Therefore, this issue must be taken into consideration to maintain and improve the trust among people for the use of IoT devices and systems.

Scalability, availability and reliability

A system is scalable if it is possible to add new services, equipments and devices without degrading its performance. The main issue with IoT is to support a large number of devices with different memory, processing, storage power and bandwidth [ 28 ]. Another important issue that must be taken into consideration is the availability. Scalability and availability both should be deployed together in the layered framework of IoT. A great example of scalability is cloud based IoT systems which provide sufficient support to scale the IoT network by adding up new devices, storage and processing power as required.

However, this global distributed IoT network gives rise to a new research paradigm to develop a smooth IoT framework that satisfy global needs [ 51 ]. Another key challenge is the availability of resources to the authentic objects regardless of their location and time of the requirement. In a distributed fashion, several small IoT networks are timely attached to the global IoT platforms to utilize their resources and services. Therefore, availability is an important concern [ 52 ]. Due to the use of different data transmission channels i.e. satellite communication, some services and availability of resources may be interrupted. Therefore, an independent and reliable data transmission channel is required for uninterrupted availability of resources and services.

Quality of Service (QoS)

Quality of Service (QoS) is another important factor for IoT. QoS can be defined as a measure to evaluate the quality, efficiency and performance of IoT devices, systems and architecture [ 34 ]. The important and required QoS metrics for IoT applications are reliability, cost, energy consumption, security, availability and service time [ 53 ]. A smarter IoT ecosystem must fulfill the requirements of QoS standards. Also, to ensure the reliability of any IoT service and device, its QoS metrics must be defined first. Further, users may also be able to specifiy their needs and requirements accordingly. Several approaches can be deployed for QoS assessment, however as mentioned by White et al. [ 54 ] there is a trade-off between quality factors and approaches. Therefore, good quality models must be deployed to overcome this trade-off. There are certain good quality models available in literature such as ISO/IEC25010 [ 55 ] and OASIS-WSQM [ 56 ] which can be used to evaluate the approaches used for QoS assessment. These models provides a wide range of quality factors that is quite sufficient for QoS assessment for IoT services. Table  2 summarizes the different studies with respect to IoT key challenges and issues discussed above.

Major IoT applications

Emerging economy, environmental and health-care.

IoT is completely devoted to provide emerging public and financial benefits and development to the society and people. This includes a wide range of public facilities i.e. economic development, water quality maintenance, well-being, industrialization etc. Overall, IoT is working hard to accomplish the social, health and economic goals of United Nations advancement step. Environmental sustainability is another important concern. IoT developers must be concerned about environmental impact of the IoT systems and devices to overcome the negative impact [ 48 ]. Energy consumption by IoT devices is one of the challenges related to environmental impact. Energy consumption is increasing at a high rate due to internet enabled services and edge cutting devices. This area needs research for the development of high quality materials in order to create new IoT devices with lower energy consumption rate. Also, green technologies can be adopted to create efficient energy efficient devices for future use. It is not only environmental friendly but also advantageous for human health. Researchers and engineers are engaged in developing highly efficient IoT devices to monitor several health issues such as diabetes, obesity or depression [ 57 ]. Several issues related to environment, energy and healthcare are considered by several studies.

Smart city, transport and vehicles

IoT is transforming the traditional civil structure of the society into high tech structure with the concept of smart city, smart home and smart vehicles and transport. Rapid improvements are being done with the help of supporting technologies such as machine learning, natural language processing to understand the need and use of technology at home [ 58 ]. Various technologies such as cloud server technology, wireless sensor networks that must be used with IoT servers to provide an efficient smart city. Another important issue is to think about environmental aspect of smart city. Therefore, energy efficient technologies and Green technologies should also be considered for the design and planning of smart city infrastructure. Further, smart devices which are being incorporated into newly launched vehicles are able to detect traffic congestions on the road and thus can suggest an optimum alternate route to the driver. This can help to lower down the congestion in the city. Furthermore, smart devices with optimum cost should be designed to be incorporated in all range vehicles to monitor the activity of engine. IoT is also very effective in maintaining the vehicle’s health. Self driving cars have the potential to communicate with other self driving vehicles by the means of intelligent sensors. This would make the traffic flow smoother than human-driven cars who used to drive in a stop and go manner. This procedure will take time to be implemented all over the world. Till the time, IoT devices can help by sensing traffic congestion ahead and can take appropriate actions. Therefore, a transport manufacturing company should incorporate IoT devices into their manufactured vehicles to provide its advantage to the society.

Agriculture and industry automation

The world’s growing population is estimated to reach approximate 10 billion by 2050. Agriculture plays an important role in our lives. In order to feed such a massive population, we need to advance the current agriculture approaches. Therefore, there is a need to combine agriculture with technology so that the production can be improved in an efficient way. Greenhouse technology is one of the possible approaches in this direction. It provides a way to control the environmental parameters in order to improve the production. However, manual control of this technology is less effective, need manual efforts and cost, and results in energy loss and less production. With the advancement of IoT, smart devices and sensors makes it easier to control the climate inside the chamber and monitor the process which results in energy saving and improved production (Fig.  9 ). Automatization of industries is another advantage of IoT. IoT has been providing game changing solutions for factory digitalization, inventory management, quality control, logistics and supply chain optimization and management.

A working structure of IoT system in agriculture production

Importance of big data analytics in IoT

An IoT system comprises of a huge number of devices and sensors that communicates with each other. With the extensive growth and expansion of IoT network, the number of these sensors and devices are increasing rapidly. These devices communicate with each other and transfer a massive amount of data over internet. This data is very huge and streaming every second and thus qualified to be called as big data. Continuous expansion of IoT based networks gives rise to complex issue such as management and collection of data, storage and processing and analytics. IoT big data framework for smart buildings is very useful to deal with several issues of smart buildings such as managing oxygen level, to measure the smoke/hazardous gases and luminosity [ 59 ]. Such framework is capable to collect the data from the sensors installed in the buildings and performs data analytics for decision making. Moreover, industrial production can be improved using an IoT based cyber physical system that is equipped with an information analysis and knowledge acquisition techniques [ 60 ]. Traffic congestion is an important issue with smart cities. The real time traffic information can be collected through IoT devices and sensors installed in traffic signals and this information can be analyzed in an IoT based traffic management system [ 61 ]. In healthcare analysis, the IoT sensors used with patients generate a lot of information about the health condition of patients every second. This large amount of information needs to be integrated at one database and must be processed in real time to take quick decision with high accuracy and big data technology is the best solution for this job [ 62 ]. IoT along with big data analytics can also help to transform the traditional approaches used in manufacturing industries into the modern one [ 63 ]. The sensing devices generates information which can be analyzed using big data approaches and may help in various decision making tasks. Furthermore, use of cloud computing and analytics can benefit the energy development and conservation with reduced cost and customer satisfaction [ 64 ]. IoT devices generate a huge amount of streaming data which needs to be stored effectively and needs further analysis for decision making in real time. Deep learning is very effective to deal with such a large information and can provide results with high accuracy [ 65 ]. Therefore, IoT, Big data analytics and Deep learning together is very important to develop a high tech society.

Conclusions

Recent advancements in IoT have drawn attention of researchers and developers worldwide. IoT developers and researchers are working together to extend the technology on large scale and to benefit the society to the highest possible level. However, improvements are possible only if we consider the various issues and shortcomings in the present technical approaches. In this survey article, we presented several issues and challenges that IoT developer must take into account to develop an improved model. Also, important application areas of IoT is also discussed where IoT developers and researchers are engaged. As IoT is not only providing services but also generates a huge amount of data. Hence, the importance of big data analytics is also discussed which can provide accurate decisions that could be utilized to develop an improved IoT system.

Availability of data and materials

Not applicable.

Abbreviations

Internet of Things

Quality of Service

Web of Things

Cloud of Things

Smart Home System

Smart Health Sensing System

Sfar AR, Zied C, Challal Y. A systematic and cognitive vision for IoT security: a case study of military live simulation and security challenges. In: Proc. 2017 international conference on smart, monitored and controlled cities (SM2C), Sfax, Tunisia, 17–19 Feb. 2017. https://doi.org/10.1109/sm2c.2017.8071828 .

Gatsis K, Pappas GJ. Wireless control for the IoT: power spectrum and security challenges. In: Proc. 2017 IEEE/ACM second international conference on internet-of-things design and implementation (IoTDI), Pittsburg, PA, USA, 18–21 April 2017. INSPEC Accession Number: 16964293.

Zhou J, Cap Z, Dong X, Vasilakos AV. Security and privacy for cloud-based IoT: challenges. IEEE Commun Mag. 2017;55(1):26–33. https://doi.org/10.1109/MCOM.2017.1600363CM .

Article   Google Scholar  

Sfar AR, Natalizio E, Challal Y, Chtourou Z. A roadmap for security challenges in the internet of things. Digit Commun Netw. 2018;4(1):118–37.

Minoli D, Sohraby K, Kouns J. IoT security (IoTSec) considerations, requirements, and architectures. In: Proc. 14th IEEE annual consumer communications & networking conference (CCNC), Las Vegas, NV, USA, 8–11 January 2017. https://doi.org/10.1109/ccnc.2017.7983271 .

Gaona-Garcia P, Montenegro-Marin CE, Prieto JD, Nieto YV. Analysis of security mechanisms based on clusters IoT environments. Int J Interact Multimed Artif Intell. 2017;4(3):55–60.

Behrendt F. Cycling the smart and sustainable city: analyzing EC policy documents on internet of things, mobility and transport, and smart cities. Sustainability. 2019;11(3):763.

IoT application areas. https://iot-analytics.com/top-10-iot-project-application-areas-q3-2016/ . Accessed 05 Apr 2019.

Zanella A, Bui N, Castellani A, Vangelista L, Zorgi M. Internet of things for smart cities. IEEE IoT-J. 2014;1(1):22–32.

Google Scholar  

Khajenasiri I, Estebsari A, Verhelst M, Gielen G. A review on internet of things for intelligent energy control in buildings for smart city applications. Energy Procedia. 2017;111:770–9.

Internet of Things. http://www.ti.com/technologies/internet-of-things/overview.html . Accessed 01 Apr 2019.

Liu T, Yuan R, Chang H. Research on the internet of things in the automotive industry. In: ICMeCG 2012 international conference on management of e-commerce and e-Government, Beijing, China. 20–21 Oct 2012. p. 230–3.

Alavi AH, Jiao P, Buttlar WG, Lajnef N. Internet of things-enabled smart cities: state-of-the-art and future trends. Measurement. 2018;129:589–606.

Weber RH. Internet of things-new security and privacy challenges. Comput Law Secur Rev. 2010;26(1):23–30.

Article   MathSciNet   Google Scholar  

Heer T, Garcia-Morchon O, Hummen R, Keoh SL, Kumar SS, Wehrle K. Security challenges in the IP based internet of things. Wirel Pers Commun. 2011;61(3):527–42.

Liu J, Xiao Y, Philip-Chen CL. Authentication and access control in the internet of things. In: 32nd international conference on distributed computing systems workshops, Macau, China. IEEE xplore; 2012. https://doi.org/10.1109/icdcsw.2012.23 .

Kothmayr T, Schmitt C, Hu W, Brunig M, Carle G. DTLS based security and two-way authentication for the internet of things. Ad Hoc Netw. 2013;11:2710–23.

Li Y, et al. IoT-CANE: a unified knowledge management system for data centric internet of things application systems. J Parallel Distrib Comput. 2019;131:161–72.

Olivier F, Carlos G, Florent N. New security architecture for IoT network. In: International workshop on big data and data mining challenges on IoT and pervasive systems (BigD2M 2015), procedia computer science, vol. 52; 2015. p. 1028–33.

Luk M, Mezzour G, Perrig A, Gligor V. MiniSec: a secure sensor netowrk communication architecture. In: Proc: 6th international symposium on information processing in sensor networks, Cambridge, MA, USA, 25–27 April 2007.

Karlof C, Sastry N, Wagner D. TinySec: a link layer security architecture for wireless sensor networks. In: Proceedings of the second ACM conference on embedded networked sensor systems (SenSys 2004), November 2004.

ZigBee Alliance. Zigbee specification. Technical Report Document 053474r06, Version 1.0, ZigBee Alliance, June 2005.

Yan Z, Zhang P, Vasilakos AV. A survey on trust management for internet of things. J Netw Comput Appl. 2014;42:120–34.

Bao F, Chen I-R, Guo J. Scalable, adaptive and survivable trust management for community of interest based internet of things systems. In: Proc. IEEE 11th international symposium on autonomous decentralized systems (ISADS); 2013. p. 1–7.

Noura M, Atiquazzaman M, Gaedke M. Interoperability in internet of things: taxonomies and open challenges. Mob Netw Appl. 2019;24(3):796–809.

Al-Fuqaha A, Guizani M, Mohammadi M, Aledhari M, Ayyash M. Internet of things: a survey, on enabling technologies, protocols, and applications. IEEE Commun Surv Tutor. 2015;17(June):2347–76.

Palattella MR, Dohler M, Grieco A, Rizzo G, Torsner J, Engel T, Ladid L. Internet of things in the 5G era: enablers, architecture and business models. IEEE J Sel Areas Commun. 2016;34(3):510–27.

Pereira C, Aguiar A. Towards efficient mobile M2M communications: survey and open challenges. Sensors. 2014;14(10):19582–608.

Kim NS, Lee K, Ryu JH. Study on IoT based wild vegetation community ecological monitoring system. In: Proc. 2015 7th international conference on ubiquitous and future networks, Sapporo, Japan, 7–10 July 2015. IEEE.

Wang JY, Cao Y, Yu GP, Yuan M. Research on applications of IoT in domestic waste treatment and disposal. In: Proc. 11th World congress on intelligent control and automation, Shenyang, China, 2014. IEEE.

Qiu T, Xiao H, Zhou P. Framework and case studies of intelligent monitoring platform in facility agriculture ecosystem. In: Proc. 2013 second international conference on agro-geoinformatics (agro-geoinformatics), Fairfax, VA, USA, 12–16 Aug 2013. IEEE.

Fang S, et al. An integrated system for regional environmental monitoring and management based on internet of things. IEEE Trans Ind Inf. 2014;10(2):1596–605.

Cheng Y, et al. AirCloud: a cloud based air-quality monitoring system for everyone. In: Proceedings of the 12th ACM conference on embedded network sensor systems, ACM, Memphis, Tennessee, 03–06 Nov 2014. p. 251–65.

Temglit N, Chibani A, Djouani K, Nacer MA. A distributed agent-based approach for optimal QoS selection in web of object choreography. IEEE Syst J. 2018;12(2):1655–66.

Talavera JM, et al. Review of IoT applications in agro-industrial and environmental fields. Comput Electron Agric. 2017;142(7):283–97.

Jara AJ, Zamora-Izquierdo MA, Skarmeta AF. Interconnection framework for mHealth and remote monitoring based in the internet of things. IEEE J Sel Areas Commun. 2013;31(9):47–65.

Gubbi J, Buyya R, Marusic S, Palaniswami M. Internet of things (IoT): a vision, architectural elements, and future directions. Future Gener Comput Syst. 2013;29(7):1645–60.

Sebastian S, Ray PP. Development of IoT invasive architecture for complying with health of home. In: Proc: I3CS, Shillong; 2015. p. 79–83.

Nicolescu R, Huth M, Radanliev P, Roure DD. Mapping the values of IoT. J Inf Technol. 2018;33(4):345–60.

Hu P, Ning H, Qiu T, Xu Y, Luo X, Sangaiah AK. A unified face identification and resolutions scheme using cloud computing in internet of things. Future Gener Comput Syst. 2018;81:582–92.

Babovic ZB, Protic V, Milutinovic V. Web performance evaluation for internet of things applications. IEEE Access. 2016;4:6974–92.

Internet of Things research study: Hewlett Packard Enterprise Report. 2015. http://www8.hp.com/us/en/hp-news/press-release.html?id=1909050#.WPoNH6KxWUk .

Xu LD, He W, Li S. Internet of things in industries: a survey. IEEE Trans Ind Inf. 2014;10(4):2233–43.

Dierks T, Allen C. The TLS protocol version 1.0, IETF RFC, 2246; 1999. https://www.ietf.org/rfc/rfc2246.txt .

Pei M, Cook N, Yoo M, Atyeo A, Tschofenig H. The open trust protocol (OTrP). IETF 2016. https://tools.ietf.org/html/draft-pei-opentrustprotocol-00 .

Roman R, Najera P, Lopez J. Securing the internet of things. Computer. 2011;44(9):51–8.

Van-der-Veer H, Wiles A. Achieving technical, interoperability-the ETSI approach, ETSI White Paper No. 3. 2008. http://www.etsi.org/images/files/ETSIWhitePapers/IOP%20whitepaper%20Edition%203%20final.pdf .

Colacovic A, Hadzialic M. Internet of things (IoT): a review of enabling technologies, challenges and open research issues. Comput Netw. 2018;144:17–39.

Noura M, Atiquazzaman M, Gaedke M. Interoperability in internet of things infrastructure: classification, challenges and future work. In: Third international conference, IoTaaS 2017, Taichung, Taiwan. 20–22 September 2017.

Tzafestad SG. Ethics and law in the internet of things world. Smart Cities. 2018;1(1):98–120.

Mosko M, Solis I, Uzun E, Wood C. CCNx 1.0 protocol architecture. A Xerox company, computing science laboratory PARC; 2017.

Wu Y, Li J, Stankovic J, Whitehouse K, Son S, Kapitanova K. Run time assurance of application-level requirements in wireless sensor networks. In: Proc. 9th ACM/IEEE international conference on information processing in sensor networks, Stockholm, Sweden, 21–16 April 2010. p. 197–208.

Huo L, Wang Z. Service composition instantiation based on cross-modified artificial Bee Colony algorithm. Chin Commun. 2016;13(10):233–44.

White G, Nallur V, Clarke S. Quality of service approaches in IoT: a systematic mapping. J Syst Softw. 2017;132:186–203.

ISO/IEC 25010—Systems and software engineering—systems and software quality requirements and evaluation (SQuaRE)—system and software quality models, Technical Report; 2010.

Oasis. Web services quality factors version 1.0. 2012. http://docs.oasis-open.org/wsqm/wsqf/v1.0/WS-Quality-Factors.pdf .

Fafoutis X, et al. A residential maintenance-free long-term activity monitoring system for healthcare applications. EURASIP J Wireless Commun Netw. 2016. https://doi.org/10.1186/s13638-016-0534-3 .

Park E, Pobil AP, Kwon SJ. The role of internet of things (IoT) in smart cities: technology roadmap-oriented approaches. Sustainability. 2018;10:1388.

Bashir MR, Gill AQ. Towards an IoT big data analytics framework: smart buildings system. In: IEEE 18th international conference on high performance computing and communications; IEEE 14th international conference on smart city; IEEE 2nd international conference on data science and systems; 2016. p. 1325–32.

Lee C, Yeung C, Cheng M. Research on IoT based cyber physical system for industrial big data analytics. In: 2015 IEEE international conference on industrial engineering and engineering management (IEEM). New York: IEEE; 2015. p. 1855–9.

Rizwan P, Suresh K, Babu MR. Real-time smart traffic management system for smart cities by using internet of things and big data. In: International conference on emerging techno-logical trends (ICETT). New York: IEEE; 2016. p. 1–7.

Vuppalapati C, Ilapakurti A, Kedari S. The role of big data in creating sense EHR, an integrated approach to create next generation mobile sensor and wear-able data driven electronic health record (EHR). In: 2016 IEEE second international conference on big data computing service and applications (BigDataService). New York: IEEE; 2016. p. 293–6.

Mourtzis D, Vlachou E, Milas N. Industrial big data as a result of IoT adoption in manufacturing. Procedia CIRP. 2016;55:290–5.

Ramakrishnan R, Gaur L. Smart electricity distribution in residential areas: Internet of things (IoT) based advanced metering infrastructure and cloud analytics. In: International Conference on internet of things and applications (IOTA). New York: IEEE; 2016. p. 46–51.

Mohammadi M, Al-Fuqaha A, Sorour S, Guizani M. Deep learning for IoT big data and streaming analytics: a survey. IEEE Commun Surv Tutor. 2018;20(4):2923–60.

Clausen T, Herberg U, Philipp M. A critical evaluation of the IPv6 routing protocol for low power and lossy networks (RPL). In: 2011 IEEE 7th international conference on wireless and mobile computing, networking and communications (WiMob), Wuhan, China, 10–12 Oct 2011.

Li H, Wang H, Yin W, Li Y, Qian Y, Hu F. Development of remote monitoring system for henhouse based on IoT technology. Future Internet. 2015;7(3):329–41.

Zhang L. An IoT system for environmental monitoring and protecting with heterogeneous communication networks. In: Proc. 2011 6th international ICST conference on communications and networking in China (CHINACOM), Harbin, China, 17–19 Aug 2011. IEEE.

Montori F, Bedogni L, Bononi L. A collaborative internet of things architecture for smart cities and environmental monitoring. IEEE Internet Things J. 2018;5(2):592–605.

Distefano S, Longo F, Scarpa M. QoS assessment of mobile crowd sensing services. J Grid Comput. 2015;13(4):629–50.

Stankovic JA. Research directions for the internet of things. IEEE Internet Things J. 2014;1(1):3–9.

Al-Fuqaha A, Khreishah A, Guizani M, Rayes A, Mohammadi M. Toward better horizontal integration among IoT services. IEEE Commun Mag. 2015;53(9):72–9.

Chen IR, Guo J, Bao F. Trust management for SOA-based IoT and its application to service composition. IEEE Trans Serv Comput. 2016;9(3):482–95.

Sarkar C, et al. DIAT: a scalable distributed architecture for IoT. IEEE Internet Things J. 2014;2(3):230–9.

Chen S, Xu H, Liu D, Hu B, Wang H. A vision of IoT: applications, challenges, and opportunities with China perspective. IEEE Internet Things J. 2014;1(4):349–59.

Kang K, Pang J, Xu LD, Ma L, Wang C. An interactive trust model for application market of the internet of things. IEEE Trans Ind Inf. 2014;10(2):1516–26.

Gupta A, Jha RK. A survey of 5G network: architecture and emerging technologies. IEEE Access. 2015;3:1206–32.

Vlacheas P, et al. Enabling smart cities through a cognitive management framework for the internet of things. IEEE Commun Mag. 2013;51(6):102–11.

Bizanis N, Kuipers FA. SDN and virtualization solutions for the internet of things: a survey. IEEE Access. 2016;4:5591–606.

Zeng X, et al. IOTSim: a simulator for analyzing IoT applications. J Syst Architect. 2017;72:93–107.

Fantacci R, Pecorella T, Viti R, Carlini C. A network architecture solutions for efficient IOT WSN backhauling: challenges and opportunities. IEEE Wirel Commun. 2014;21(4):113–9.

Kim M, Ahn H, Kim KP. Process-aware internet of things: a conceptual extension of the internet of things framework and architecture. KSII Trans Internet Inf Syst. 2016;10(8):4008–22.

Hsieh H-C, Chang K-D, Wang L-F, Chen J-L, Chao H-C. ScriptIoT: a script framework for and internet of things applications. IEEE Internet Things J. 2015;3(4):628–36.

Kiljander J, et al. Semantic interoperability architecture for pervasive computing and internet of things. IEEE Access. 2014;2:856–73.

Ye J, Chen B, Liu Q, Fang Y. A precision agriculture management system based on internet of things and WebGIS. In: Proc. 2013 21st international conference on geoinformatics, Kaifeng, China, 20–22 June 2013. IEEE.

Jara AJ, Martinez-Julia P, Skarmeta A. Light-weight multicast DNS and DNS-SD (ImDNS-SD): IPv6-based resource and service discovery for web of things. In: Proc. sixth international conference on innovative mobile and internet services in ubiquitous computing, Palermo, Italy, 4–6 July 2012.

Diaz M, Martin C, Rubio B. State-of-the-art, challenges, and open issues in the integration of internet of things and cloud computing. J Netw Comput Appl. 2016;67:99–117.

Lo A, Law YW, Jacobsson M. A cellular-centric service architecture for machine to machine (M2M) communications. IEEE Wirel Commun. 2013;20(5):143–51.

Kecskemeti G, Casale G, Jha DN, Lyon J, Ranjan R. Modeling and simulation challenges in internet of things. IEEE Cloud Comput. 2017;4(1):62–9.

Cuomo S, Somma VD, Sica F. An application of the one-factor HullWhite model in an IoT financial scenario. Sustain Cities Soc. 2018;38:18–20.

Liu J, et al. A cooperative evolution for QoS-driven IOT service composition. Autom J Control Meas Electron Comput Commun. 2013;54(4):438–47.

Huo Y, et al. Multi-objective service composition model based on cost-effective optimization. Appl Intell. 2017;48(3):651–69.

Han SN, Crespi N. Semantic service provisioning for smart objects: integrating IoT applications into the web. Future Gener Comput Syst. 2017;76:180–97.

Alodib M. QoS-aware approach to monitor violations of SLAs in the IoT. J Innov Digit Ecosyst. 2016;3(2):197–207.

Rizzardi A, Sicari S, Miorandi D, Coen-Porisini A. AUPS: an open source authenticated publish/subscribe system for internet of things. Inf Syst. 2016;62:29–41.

Fenye B, Ing-Ray C, Jia G. Scalable, adaptive and survivable trust management for community of interest based internet of things systems. In: Proc. IEEE eleventh international symposium on autonomous decentralized systems (ISADS), Mexico City, Mexico, 6–8 March 2013.

Tehrani MN, Uysal M, Yanikomeroglu H. Device to device communication in 5G cellular networks: challenges, solutions, and future directions. IEEE Commun Mag. 2014;52(5):86–92.

Zhu C, Leung VCM, Shu L, Ngai ECH. Green internet of things for smart world. IEEE Access. 2015;3:2151–62.

Adame T, Bel A, Bellalta B, Barcelo J, Oliver M. IEEE 802.11AH: the WiFi approach for M2M communications. IEEE Wirel Commun. 2014;21(6):144–52.

Shaikh FK, Zeadally S, Exposito E. Enabling technologies for green internet of things. IEEE Syst J. 2015;99:1–12.

Palattella MR, et al. Standardized protocol stack for the internet of (important) things. IEEE Commun Surv Tutor. 2012;15(3):1389–406.

Vatari S, Bakshi A, Thakur T. Green house by using IoT and cloud computing. In: Proc. 2016 IEEE international conference on recent trends in electronic, information & communication technology (RTEICT), Bangalore, India, 20–21 May 2016.

Chiang M, Zhang T. Fog and IoT: an overview of research opportunities. IEEE Internet Things J. 2016;3(6):854–64.

Elkhodr M, Shahrestani S, Cheung H. A smart home application based on the internet of things management platform. In: Proc. 2015 IEEE international conference on data science and data intensive systems, Sydney, Australia, 11–13 Dec 2015.

Talari S, et al. A review of smart cities based on the internet of things concept. Energies. 2017;10(4):421–43.

Burange AW, Misalkar HD. Review of internet of things in development of smart cities with data management & privacy. In: Proc. 2015 international conference on advances in computer engineering and applications, Ghaziabad, India, 19–20 March 2015.

Zia T, Liu P, Han W. Application-specific digital forensics investigative model in internet of things (IoT). In: Proc. 12th international conference on availability, reliability and security, Reggio Calabria, Italy; 2017.

Lingling H, Haifeng L, Xu X, Jian L. An intelligent vehicle monitoring system based on internet of things. In: Proc. 7th international conference on computational intelligence and security, Hainan, China, 3–4 Dec 2011. IEEE.

Duttagupta S, Kumar M, Ranjan R, Nambiar M. Performance prediction of IoT application: an experimental analysis. In: Proc. 6th international conference on the internet of things, Stuttgart, Germany, 07–09 Nov 2016. p. 43–51.

Chen S, Liu B, Chen X, Zhang Y, Huang G. Framework for adaptive computation offloading in IoT applications. In: Proc. 9th Asia-Pacific symposium on internetware, Shanghai, China, 23 Sep 2017. ACM.

Li Q, Dou R, Chen F, Nan G. A QoS-oriented web service composition approach based on multi-population genetic algorithm for internet of things. Int J Comput Intell Syst. 2014;7(Sup2):26–34.

Urbieta A, Gonzalez-Beltran A, Mokhtar SB, Hossain MA, Capra L. Adaptive and context-aware service composition for IoT-based smart cities. Future Gener Comput Syst. 2017;76:262–74.

Krishna GG, Krishna G, Bhalaji N. Analysis of routing protocol for low-power and lossy networks in IoT real time applications. Procedia Comput Sci. 2016;87:270–4.

Singh D, Tripathi G, Jara AJ. A survey of internet of things: future vision, architecture, challenge and services. In: Proc. IEEE world forum on internet of things, Seoul, South Korea; 2014. p. 287–92.

Jara AJ, Ladid L, Skarmeta A. The internet of everything through Ipv6: an analysis of challenges, solutions and opportunities. J Wirel Mob Netw Ubiquitous Comput Dependable Appl. 2013;4(3):97–118.

Madsen H, Burtschy B, Albeanu G, Popentiu-Vladicescu Fl. Reliability in the utility computing era: towards reliable Fog computing. In: Proc. 20th international conference on systems, signals, and image processing (IWSSIP); 2013. p. 43–6.

Soret B, Pedersen KI, Jorgensen NTK, Fernandez-Lopez V. Interference coordination for dense wireless networks. IEEE Commun Mag. 2015;53(1):102–9.

Andrews JG. Seven ways that HetNets are a cellular paradigm shift. IEEE Commun Mag. 2013;51(3):136–44.

Jaber M, Imran MA, Tafazolli R, Tukmanov A. 5G Backhaul challenges and emerging research directions: a survey. IEEE Access. 2016;4:1743–66.

Choi S, Koh S-J. Use of proxy mobile IPv6 for mobility management in CoAP-based internet of things networks. IEEE Commun Lett. 2016;20(11):2284–7.

Maier M, Chowdhury M, Rimal BP, Van DP. The tactile internet: vision, recent progress, and open challenges. IEEE Commun Mag. 2016;54(5):138–45.

Fernandes JL, Lopes IC, Rodrigues JJPC, Ullah S. Performance evaluations of RESTful web services and AMQP protocol. In: 5th international conference on ubiquitous and future networks (ICUFN), Da Nang, Vietnam, 2–5 July 2013.

Download references

Acknowledgements

This work was financially supported by the Ministry of Education and Science of Russian Federation (government order 2.7905.2017/8.9).

The research received no external funding.

Author information

Authors and affiliations.

Department of Computer Science, South Ural State University, Chelyabinsk, Russian Federation

Sachin Kumar & Mikhail Zymbler

Department of Information Engineering, University of Padova, Padua, Italy

Prayag Tiwari

You can also search for this author in PubMed   Google Scholar

Contributions

SK and PT prepared the draft and Idea. SK wrote the manuscript. MZ prepared the tables, references and checked the English. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sachin Kumar .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Kumar, S., Tiwari, P. & Zymbler, M. Internet of Things is a revolutionary approach for future technology enhancement: a review. J Big Data 6 , 111 (2019). https://doi.org/10.1186/s40537-019-0268-2

Download citation

Received : 24 July 2019

Accepted : 10 November 2019

Published : 09 December 2019

DOI : https://doi.org/10.1186/s40537-019-0268-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Internet of Things (IoT)
• IoT architecture
• IoT challenges
• IoT applications

Advertisement

A comprehensive and systematic literature review on the big data management techniques in the internet of things

• Original Paper
• Published: 15 November 2022
• Volume 29 , pages 1085–1144, ( 2023 )

Cite this article

• Arezou Naghib   nAff1 ,
• Nima Jafari Navimipour 2 , 3 ,
• Mehdi Hosseinzadeh 4 , 5 , 6 &
• Arash Sharifi 1  

11k Accesses

16 Citations

Explore all metrics

The Internet of Things (IoT) is a communication paradigm and a collection of heterogeneous interconnected devices. It produces large-scale distributed, and diverse data called big data. Big Data Management (BDM) in IoT is used for knowledge discovery and intelligent decision-making and is one of the most significant research challenges today. There are several mechanisms and technologies for BDM in IoT. This paper aims to study the important mechanisms in this area systematically. This paper studies articles published between 2016 and August 2022. Initially, 751 articles were identified, but a paper selection process reduced the number of articles to 110 significant studies. Four categories to study BDM mechanisms in IoT include BDM processes, BDM architectures/frameworks, quality attributes, and big data analytics types. Also, this paper represents a detailed comparison of the mechanisms in each category. Finally, the development challenges and open issues of BDM in IoT are discussed. As a result, predictive analysis and classification methods are used in many articles. On the other hand, some quality attributes such as confidentiality, accessibility, and sustainability are less considered. Also, none of the articles use key-value databases for data storage. This study can help researchers develop more effective BDM in IoT methods in a complex environment.

Similar content being viewed by others

Systematic Literature Review on Data Provenance in Internet of Things

Data quality and the Internet of Things

Recent Research on Data Analytics Techniques for Internet of Things

Explore related subjects.

• Artificial Intelligence

Avoid common mistakes on your manuscript.

1 Introduction

The Internet of Things (IoT) is an emerging information technology model and a dynamic network that enables interaction between self-configuring, smart, and interconnected devices and humans [ 1 ]. The IoT's ubiquitous data collection devices (such as Radio-Frequency Identification (RFID) tags, sensors, Global Positioning Systems (GPS), Geographical Information Systems (GIS), drives, Near-Field Communication (NFC), actuators, and mobile phones) collect and share real-time, mobile, and environmental data for automatic monitoring, identification, processing, maintenance, and control in real-time [ 2 , 3 , 4 ]. The IoT ecosystem has five main components generally: IoT devices, including sensors and actuators that collect data and perform actions on things; IoT connectivity, including protocols and gateways, that is responsible for creating communication in the IoT ecosystem between smart devices, gateways, and the cloud; an IoT cloud that is responsible for data storage, processing, analysis, and decision-making; IoT analytics and data management are responsible for processing the data; and end-user devices and user interfaces help to control and configure the system [ 5 ]. The most important applications of IoT include environmental monitoring, disaster management, smart homes/buildings, smart farms, healthcare, smart cities, urban, smart manufacturing, intelligent transport systems, smart floods, financial risk management, supply chain management, water management, enterprise culture, cultural heritage, smart surveillance, military tracking and environment, digital forensics, underwater environments, and understanding social phenomena [ 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 ]. The IoT devices and sensors in the Wireless Sensor Networks (WSN) generate large data. According to the international data corporation Footnote 1 forecast, the number of IoT devices will be 41.6 billion and generate 79.4 zettabytes of data in 2025. This massive structured, semi-structured, and unstructured data, which is expanding rapidly with time, results in "Big Data" [ 23 ]. "Big data" technologies are a new generation of distributed architectures and technologies that provide distributed data mining capabilities to inexpensively, valuable, and effectively extract value from a huge dataset with characteristics such as volume, velocity, variety, variability, veracity, and value [ 24 ]. Big data provides both opportunities and problems for organizations and enterprises. Big data can improve data precision, be used for forecasting and decision-making, and give stakeholders more in-depth analytical findings [ 2 ]. Traditional data processing systems cannot collect, process, manage, and interpret data effectively using conventional mechanisms. Therefore, it requires a scalable architecture or framework for effective capture, storage, management, and analysis [ 25 ].

A major challenge in implementing IoT in real and complex environments is analyzing heterogeneous data volumes that contain a wide variety of knowledge content [ 26 ]. Various platforms, tools, and technologies have been developed for big data monitoring, collecting, ingesting, storing, processing, analysis, and visualization [ 10 , 27 ]. These platforms and tools are Apache Hadoop, MapReduce, 1010data, Apache Storm, Cloudera, Cassandra, HP-HAVEn, SAP-Hana, Hortonworks, MongoDB, Apache Kafka, Apache Spark, Infobright, etc. Industries and enterprises use Big Data Analytics (BDA) with IoT technologies to handle the timely analysis of information streams and intelligent decision-making [ 28 , 29 , 30 ]. BDM in the IoT involves different analytic types [ 31 ]. Marjani et al. [ 29 ] discussed analytical types in real-time, offline, memory, business intelligence, and at massive levels. Singh and Yassine [ 28 ] divided analytical types into preprocessing, pattern mining, and classification. Gandomi and Haider [ 32 ] divided big data processing into two major phases: data management and data analytics. Also, Ahmed et al. [ 33 ] provided five aspects of big data: acquisition and storage; programming model; benchmark process; analysis; and application. Finally, ur Rehman et al. [ 34 ] divided BDA into five main steps: data ingestion, cleaning, conformation, transformation, and shaping.

However, despite the importance of BDM in the IoT and the rising challenges in this area, as far as we know, there is not any complete and detailed systematic review in this field. Hence, this paper tries to analyze the mechanisms of BDM in the IoT. The main contributions of this paper are as follows:

Presenting a study of the existing methods for BDM in the IoT.

Dividing BDM methods in the IoT are divided into four main categories: BDM processes, BDM architectures/frameworks, quality attributes, and big data analytics types.

Dividing the BDM process in the IoT into six main steps, including data collection, communication, data ingestion, data storage, processing and analysis, and post-processing.

Dividing the BDM architecture/framework in the IoT into two main subcategories: BDM architectures/frameworks in IoT-based applications and BDM architectures/frameworks in the IoT paradigms.

Exploring the primary challenges, issues, and future works for BDM in the IoT.

The following subsection discusses related work to show the main differences between this review and similar studies. Also, the abbreviations used in this paper are presented in Table 1 .

1.1 Related work and contributions of this review

This section studies some reviews and survey articles that work on BDM in the IoT to highlight the need for reviewing them. In addition, this section describes the main advantages and disadvantages of this article to distinguish this one.

Ahmed et al. [ 27 ] analyzed several techniques for IoT-based big data. This article categorizes the literature based on parameters, including big data sources, system components, big data enabling technologies, functional elements, and analytics types. The authors also discussed connectivity, storage, quality of services, real-time analytics, and benchmarking as the critical requirements for big data processing and analytics.

Constante Nicolalde et al. [ 35 ] overviewed the technical tools used to process big data and discussed the relationship between BDA and IoT. The big data challenges are divided into four general categories: data storage and analysis; the discovery of knowledge and computational complexities; information security; and scalability and data visualization.

Talebkhah et al. [ 36 ] investigated the architecture, challenges, and opportunities of big data systems in smart cities. This article suggested a 4-layer architecture for BDM in smart cities. The layers of this architecture are data acquisition, data preprocessing, data storage, and data analytics. This article also considered the opportunities and challenges for smart cities, such as heterogeneity, design and maintenance costs, failure management, throughout, etc.

Bansal et al. [ 37 ] investigated state-of-the-art research on IoT and BDM. This article proposed a taxonomy based on BDM in the IoT applications, including smart transport, smart cities, smart buildings, and smart living. BDM steps are considered as data acquisition, communication, storage, processing, and retrieval. Also, the related surveys on BDM were divided into three general categories: surveys on IoT BDA, domain-specific surveys on IoT big data, and surveys on challenges in IoT big data. The authors classified the articles based on four major vendor services (Google, Amazon, Microsoft, and IBM) to integrate IoT and IoT big data with case studies. The big data management challenges in the IoT are considered based on 13 V’s challenges.

Marjani et al. [ 29 ] investigated state-of-the-art research efforts directed toward big IoT data analytics and proposed a new architecture for big IoT data analytics. This article discusses big IoT data analytic types under real-time, offline, memory-level, business intelligence, and massive level analytics categories.

Simmhan and Perera [ 38 ] presented the analytics requirements of IoT applications. They defined the relationship between data volume capacity and processing latency of new big data platforms. This article divided decision systems into visual analytics, alerts and warnings, reactive systems, control and optimization, complex systems, knowledge-driven intelligent systems, and behavioral and probabilistic systems.

Shoumy et al. [ 39 ] discussed frameworks and techniques for multimodal big data analytics. They divided multimodal big data analytics techniques into four topics: affective framework; multimodal framework; big data and analytics framework; and fusion techniques. Furthermore, Ge et al. [ 40 ] discussed the similarities and differences among big data technologies used in IoT domains and developed a conceptual framework. This article interpreted big data research and application opportunities in eight IoT domains (healthcare, energy, transportation, building automation, smart cities, agriculture, industry, and military) and discussed the advantages and disadvantages of big data technologies. In addition, it examined four aspects of big data processes: storage, cleaning/cleansing, analysis/analytics, and visualization.

Siow et al. [ 41 ] considered the analytics infrastructure from data generation, collection, integration, storage, and computing. This article presented a comprehensive classification of analytical capabilities consisting of five categories: descriptive, diagnostic, discovery, predictive, and prescriptive analytics. In addition, a 3-layered taxonomy of data analytics was presented, including data, analytics, and applications.

Fawzy et al. [ 42 ] investigated the techniques and technologies of IoT systems from BDA architectures and software engineering perspectives. This article proposed a taxonomy based on BDA systems in the IoT, including smart environments, human, network, energy, and environmental analytics. The BDA target, approach, technology, challenges, software architecture and design, model-driven engineering, separation of concerns, and system validation and verification. The authors presented the IoT data features as multidimensional, massive, timely, heterogeneous, inconsistent, traded, valuable, and spatially correlated. The proposed domain-independent BDA-based IoT architecture has six layers. The layers of architecture are data manager, system resources controller, system recovery manager, BDA handler, software engineering handler, and security manager.

Zhong et al. [ 43 ] investigated using BDA and data mining techniques in the IoT. This article divided the review articles into four categories: architecture and platform, framework, applications, and security. The data mining methods for BDA in the IoT were discussed in these four categories. The challenges investigated in the article are as follows: data volume, data diversity, speed, data value, security, data visualization, knowledge extraction, and real-time analysis.

Hajjaji et al. [ 44 ] discussed applications, tools, technologies, architectures, current developments, challenges, and opportunities in big data and IoT-based applications in smart environments. This article divided the benefits of combining the IoT and big data into six categories: multi-source and heterogeneous data; connectivity; data storage; data analysis; and cost-effectiveness.

Ahmadova et al. [ 45 ] discussed big data applications in the IoT. They proposed a taxonomy of big data in the IoT that includes healthcare, smart cities, security, big data algorithms, industry, and general view. In the article, the authors discussed big data technologies' advantages and disadvantages for IoT domains. Also, the evaluation factors that are considered in the article are security, throughput, cost, energy consumption, reliability, response time, and availability.

Table 2 shows the summary contributions of related survey articles. The publication year, methodology, discussion, and other disadvantages are shown for each article in this table. Due to the existing weaknesses in the review articles, this paper presents a systematic literature review and a proper categorization of BDM mechanisms in the IoT that addresses the shortcomings as follows:

This paper provides a complete research methodology that includes research questions and the article selection process.

This paper discusses the newly proposed mechanisms for BDM in the IoT between 2016 and August 2022.

This paper considers the architectures/frameworks of IoT-based applications, including healthcare, smart cities, smart homes/buildings, intelligent transport, traffic control and energy, urban planning, and other IoT applications (smart IoT systems, smart flood, smart farms, disaster management, laundry, digital manufacturing, and smart factory).

This paper investigates the quality attributes and categorizes the review articles based on the quality attributes used and the reference model of standard software quality attributes, i.e., ISO 25010.

This paper classifies the review articles based on BDA types in the IoT and their tactics.

This paper considers the big data storage systems and tools in the IoT based on relational databases, NoSQL databases, distributed file systems, and cloud/edge/fog/mist storage.

This paper discusses the BDM process in six steps: data collection, communication, data ingestion, data storage, processing and analysis, and post-processing, and proposes the main tools in each step.

This paper presents open issues and challenges on BDM in the IoT and divides challenges into two categories: BDM in the IoT and quality attributes challenges.

The rest of the paper is structured as follows: Sect.  2 explains the research methodology and the article selection process. The categories of the BDM methods in the IoT and their comparison are described in Sect.  3 . Section  4 discusses the challenges and some open issues. Finally, Sect.  5 represents the conclusion and the paper’s limitations.

2 Research methodology

Systematic literature review (SLR) is a research methodology that examines data and findings of the researchers relative to specified questions [ 46 , 47 ]. It aims to find as much relevant research on the defined questions as possible and to use explicit methods to identify what can reliably be said based on these studies [ 48 , 49 ]. This section provides an SLR to understand the BDM techniques in the IoT. The following subsection will explain the research questions and the article selection process.

2.1 Research questions

This study focuses more explicitly on the articles related to BDM in the IoT, focusing on their advantages and disadvantages, architectures, processing and analysis methods, storage systems, evaluation metrics, and tools. To achieve the goals mentioned above, the following research questions are presented.

RQ1: What is BDM in IoT?

Section  1 answered this question.

RQ2: What is the importance of BDM in the IoT?

This question aims to show the number of published articles about BDM in IoT between 2016 and August 2022.

Section  2 answers this question.

RQ3: How are the articles searched and chosen to be assessed?

Section 2.2 discusses the question.

RQ4: What are the classifications of BDM methods in the IoT?

This question aims to show the existing methods of BDM in the IoT environment. Section  3 will discuss this answer.

RQ5: What are the challenges and technical issues of BDM in the IoT?

This question identifies the challenges for BDM in the IoT and provides open issues for future research. Section  4 will discuss this answer.

2.2 Article selection process

In this study, the article’s search and selection process consists of three stages. These stages are shown in Fig.  1 . In the first stage, the articles between 2016 and August 2022 were searched based on the keywords and terms (presented in Table 3 ). These articles are the results of searching popular electronic databases. These electronic databases include Google Scholar, Elsevier, ACM, IEEE Explore, Emerald Insight, MDPI, Springer Link, Taylor and Francis, Wiley, JST, Dblp, DOAJ, and ProQuest. The articles include journals, chapters, conference papers, books, notes, technical reports, and special issues. 751 articles were found in Stage 1. In Stage 2, there are two steps to select the final number of articles to review. First, the articles are considered based on the inclusion criteria in Fig.  2 . There are 314 articles left at this stage. Next, the review articles are removed; of the remaining 314 articles in the previous stage, 85 (27.07%) were review articles. Elsevier has the highest number of review articles (31.76%, 27 articles). EMERALD and Taylor and Francis have the lowest number of reviewed articles (2.35%, one article). The highest number of published review articles is in 2019 (24.71%), and the lowest is in 2022 (8.24%). The number of remaining articles at this stage is 229. In Stage 3, the title and abstract of the articles are reviewed. Also, to ensure that the articles are relevant to the study, we reviewed the methodology, evaluation, discussion, and conclusion sections. The number of selected articles retained at this stage is 110. Elsevier publishes most of the selected articles (30.91%, 34 articles). The lowest number is related to ACM (0.91%, one article). 2018 has the highest number of published articles (26.36%, 29 articles). The Future Generation Computer Systems journal publishes the highest number of articles (11.82%, 13 articles).

Articles search and selection process stage

Inclusion criteria in the articles selection process

3 Big data management approaches in the IoT

This section presents four different categories for the reviewed articles. These categories include the BDM process in the IoT (Sect. 3.1 ), BDM architectures/frameworks for IoT applications (Sect. 3.2 ), quality attributes (Sect. 3.3 ), and big data analytics types (Sect. 3.4 ). Each category has subcategories that will be considered in its relevant section. Figure  3 shows this taxonomy.

Taxonomy of the selected articles

3.1 Big data management process in the IoT

This section categorizes articles based on BDM process mechanisms and presents a comprehensive framework for BDM in the IoT. The comprehensive framework for BDM in the IoT is shown in Fig.  4 . The steps of BDM in IoT include data collection, communication, data ingestion, storage, processing and analysis, and post-processing.

Big data management framework in IoT

3.1.1 Data collection

A variety of sources generates IoT data. There are different mechanisms for IoT data collection, but there is still no fully efficient and adaptive mechanism for IoT data collection [ 50 ]. This paper divides IoT sources into sensors, applications, devices, and other resources. Figure  5 shows the classification of the sources based on these four categories.

Big data sources categories in IoT

3.1.2 Communication

The data sources are located on various networks, such as IoT sensor networks, wired and wireless sensor networks, fiber-optic sensor networks, and machine-to-machine communications. Communication technologies are required to process and analyze these data sources [ 51 , 52 ]. There are several communication technologies and protocols in the IoT. The communication protocols used in the articles are IPV6, RPL, MQTT, CoAP, SSL, AMQP, Websocket, 6LowPANIPV6, Alljoyn, TCP/IP, HTTP/IP. Communication technologies are compared based on frequency, data rate, range, power usage, cost, latency, etc. There are several categories of these communication technologies. This paper divides big data communication technologies in the IoT based on distance criteria into three categories: pan, local, and WAN. Table 4 shows the articles' classification based on these three categories. Wi-Fi, ZigBee, Bluetooth, and 4G LTE are of the utmost importance in communication technology, with a total number of 29, 19, 17, and 17 articles, respectively.

3.1.3 Data ingestion

Data ingestion is the process of importing and transporting data in different formats from various sources (shown in Fig.  4 ) to a storage medium, processing and analyzing platform, and decision support engines [ 93 , 94 ]. The quality of the dataset used by ML-based prediction models (classification) plays a vital role in BDM in the IoT. A prediction model requires a lot of correctly labeled data for correct construction, assessment, and accurate result generation [ 95 ]. Therefore, the data ingestion layer should handle the enormous volume, high speed (velocity), variety, value, variable, and validated data for the processing and analysis step. In different articles, this layer has multiple tasks. The data ingestion layer in [ 96 ] includes identification, filtration, validation, noise reduction, integration, transformation, and compression. The data ingestion layer in [ 97 ] provides data synchronization, data slicing, data splitting, and data indexing. Also, the data ingestion layer in [ 98 ] includes data stream acquisition, data stream extraction, enrichment, integration, and data stream distribution. Finally, the data ingestion layer in [ 99 ] includes data cleaning, data integration, and data compression.

There are three categories of data ingestion technologies: real-time data ingestion, batch data ingestion, and both. Real-time data ingestion is used for time-sensitive data and real-time intelligent decision-making. Batch data ingestion is used for data collection from sources at regular intervals (daily reports and schedules) [ 100 ]. There are many tools and platforms for data ingestion, such as Apache Kafka, Apache NIFI, Apache Storm, Apache Flume, Apache Sqoop, Apache Samza, Apache Minifi, Confluent Platform, and Elastic Logstash. These tools can be compared based on throughput, latency, scalability, and security [ 98 ]. The data ingestion layer in this paper includes data cleaning, data integration, data transformation/ discretization, and data reduction. Each of these steps uses special tools, methods, and algorithms. Table 5 shows the categorization of articles based on the tools that are used for data ingestion. Data ingestion tools have been compared based on ingestion type, throughput, reliability, latency, scalability, security, and fault tolerance. Platforms in some articles use a combination of these tools, such as the Horton data flow platform in [ 101 ], including Apache NiFi/MiNiFi, Apache Kafka, Apache Storm, and Druid tools. As you can see in Table 5 , Apache Kafka is of utmost importance to the data ingestion tool, with a total of 8 articles. Also, Table 6 shows the categorization of articles based on the big data preprocessing stage in the IoT.

3.1.4 Data storage

This subsection categorizes articles based on storage mechanisms. The articles use various methods and tools to store big data. This study divides these mechanisms into four categories: relational, NoSQL, Distributed File Systems (DFS), and cloud/edge/fog/mist storage. Each of these categories has subcategories. One of the most critical big data challenges is the categorization and scalability that traditional relational databases such as MySQL, SQL Server, and Postgres cannot overcome. Therefore, NoSQL databases are used to store big data. NoSQL technologies are divided into four categories: key-value, column-oriented, document-oriented, and graph-oriented [ 102 ]. These NoSQL technologies have many platforms to support their operations. Key-value storage is the most straightforward and highly flexible type of NoSQL database and stores all the data as a pair of keys and values. A document-oriented database stores data as a set of columns. In a relational database, data is stored in rows and read row-by-row. A graph database focuses on the relationships between data elements, and each element is stored as a node. Tables 7 and 8 show the types of storage methods used in articles. Table 7 shows the classification of articles based on relational databases, NoSQL databases, and DFS. As you can see, any of the 110 selected articles do not use the key-value databases. In relational databases, Hive, NoSQL databases, Hbase, and distributed file systems, HDFS is most commonly used. Table 7 compares these storage tools and platforms based on in-memory database/storage or disk-based, data type, scalability, security, availability, flexibility, performance, fault-tolerant, easy to use, and replication.

Table 8 shows the classification of articles based on cloud/edge/fog/mist storage. Cloud computing provides scalable computing, high data storage, processing power, and ensures the quality of the applications. However, it has main challenges such as latency, network overhead, bandwidth, data privacy, lower real-time responsiveness, location awareness, security, reliability, data availability, and accessibility [ 103 ]. Network architectures came into existence to overcome these challenges, such as fog, edge, and mist computing, that move the data and computation closer to the consumer and reduce some of the workloads from the cloud [ 104 ].

Fog computing is a type of decentralized computing that is between cloud storage and IoT devices. Fog computing reduces service latency, bandwidth, energy consumption, storage, and computing costs and improves the QoS [ 149 ]. The fog computing for the IoT model supports real-time services, mobility, and geographic distribution [ 150 ]. Another alternative approach to cloud computing is edge computing. Data storage and processing in edge computing occur closer to the device or data source to improve data locality, performance, and decision-making [ 151 ]. Edge computing is less scalable than fog computing but provides near real-time analytics and high-speed data access and reduces data leakage during transmission [ 104 , 152 ]. Mist computing is an intermediate layer between fog/cloud and edge computing. It can improve the fog/cloud challenges, such as response time, location awareness, data privacy, local decision-making, network overhead, latency, and computing and storage costs. Mist nodes had low processing power and storage [ 153 ]. In some articles, in addition to using cloud/edge/fog/mist storage, HDFS and NoSQL databases are used alongside these technologies. The goal is to overcome the disadvantages of these technologies by using them together.

3.1.5 Processing and analysis

Big data processing and analysis in the IoT are techniques or programming models for extracting knowledge from large amounts of data for supporting and providing intelligent decisions [ 154 ]. Efficient big data processing and analysis in IoT can help mitigate many challenges in event management, action management, control and monitoring, improved customer service, cost savings, improve business relationships [ 155 ], etc. This paper divides the big data processing and analysis step in IoT into a set of sub-steps: batch and stream processing, query processing, statistical and numerical analysis, graph processing, ML, resource management, and infrastructure/containers. Table 9 shows the articles' classification and comparison of the tools based on criteria: throughput, reliability, availability, latency, scalability, security, flexibility, ease of use, and cost-effectiveness. Big data processing in the IoT is generally done at both batch and stream levels. Many tools, platforms, and frameworks exist for batch and stream processing. The tools used in the articles are Apache Hadoop, Apache Spark, Map Reduce, Apache Storm, Apache Flink, Anaconda, Apache S4, Weka, streaming analytics manager, and CEP.

As you can see in Table 9 , Apache Hadoop, MapReduce, and Apache Spark are the most critical quality attributes, with a total number of 45, 32, and 31 articles, respectively. Some of these tools include a set of libraries and procedures for efficient processing and analysis. In the study, the libraries and functions used by the articles are Hadooppcap-lib, Hadoop-pcap-serde, Hadoop-pcap-input (Apache Hadoop), MLlib, GraphX, Spark Streaming, Spark SQL, Spark Core (Apache Spark), Map, FlatMap, Filter, Reduce, Shuffle (Map Reduce), Gelly, FlinkML, Table and SQL, FlinkCEP (Apache Flink), NumPy [ 132 ], Keras [ 108 ], Pandas [ 59 ], and Scikit-Learn, Paho-MQTT (Anaconda). Also, various algorithms and methods are used to process and analyze data, such as classification, clustering, regression, optimization algorithms, and SVM. Most of these tools have these algorithms.

3.1.6 Post-processing

The post-processing step is another vital task in knowledge discovery from big data in the IoT. This paper divides the post-processing step into evaluation and selection (data governance), virtualization/dashboard, intelligent decision, and service and application. The evaluation and selection stage evaluates results obtained using test methods on different types of datasets. There are various criteria for assessing the results. In this section, the articles are categorized based on the methods they used for the test. These methods are divided into four categories, including test methods, classification, clustering, and regression. Each of them uses various criteria for evaluation. Table 10 shows the articles’ classification based on these four categories. The virtualization/dashboard stage uses tools, graphs, tables [ 75 ], graphical user interface [ 59 ], and charts [ 92 ] to display the results. Intelligent decisions can be made using stochastic binary decisions [ 156 ], ML, pattern recognition, soft computing, and decision models [ 51 , 53 , 74 ]. These tools are Kibana, Plotly, Tableau, Microsoft Power BI, Grafana, vSphere, NodeJS, and Matplotlib [ 59 , 105 , 106 , 109 , 110 , 113 , 140 ].

Tables 11 and 12 show the relevant datasets that the articles used for investigating/numerically assessing techniques for BDM in the IoT. These datasets are divided into two categories: 1) categorized based on characteristics including dataset name, repository, dataset characteristics, attribute characteristics, number of instances/size, and number of attributes 2) categorized based on characteristics including dataset name, website address, and size. As you can see, the UCI machine learning repository has been repeatedly used in articles as a repository to access techniques for BDM in the IoT.

3.2 Big data management architectures/frameworks in the IoT

This subsection investigates and analyzes the articles that (71 articles) presented the frameworks and architectures for BDM techniques in the IoT. These articles are divided into two categories: BDM architectures/frameworks in the IoT-based applications (63 articles) and BDM architectures/frameworks in the IoT paradigms (8 articles).

3.2.1 Big data management architectures/frameworks in the IoT applications

The architectural models used in the selected articles are layered, component-based, and cloud/fog-based architecture. A layered architecture is organized hierarchically, and each layer performs a service. The layered architecture ensures the system is more adaptable to emerging technologies at each layer and improves the acquisition and integration of data processes [ 167 ]. Component-based architecture is a framework that decomposes the system into reusable and logical components. The advantages of component-based architecture are increased quality, reliability, component reusability, and reduced time. Operations and components related to processing or storage in cloud-based or fog-based architectures are placed in the cloud or fog. Most of the proposed architectures are layered, and the most common types of BDM architectures in the IoT are 3-layer and 4-layer (22 and 20 articles). Also, most of the proposed architectures are in IoT-based healthcare, equivalent to 33.33%, followed by IoT-based smart cities, which equals 22.22%. The selected articles in this study used nine different OS for BDM in the IoT. Ubuntu is the most important OS, with 18 articles. Articles used programming languages to analyze and process big data in the IoT. Java, Python, and MATLAB are the major programming languages. In the following, these architectures and frameworks will be examined. For a better presentation, we have divided these architectures and frameworks into seven categories in terms of IoT applications (healthcare, smart cities, smart home/building, intelligent transport, traffic control and energy, urban planning, and other IoT applications (smart IoT systems, smart flood, smart farms, disaster management, laundry, digital manufacturing, and smart factory)). Then we review the attributes of the architectures and frameworks, including layers, the functions of the layers, the operating system, the programming language, and the advantages and disadvantages of each.

3.2.1.1 BDM architectural/framework for IoT-based healthcare

Predicting health and disease and preventing deaths are essential in our modern world [ 168 , 169 ]. Healthcare IoT (e.g., electronic and mobile health) uses wireless body sensor networks for monitoring the patients’ environmental, physiological, and behavioral parameters [ 170 ]. Wearables and other IoT devices within the healthcare industry generate a large amount of data. The health data must be collected, stored, processed, and analyzed for future intelligent decision-making. BDA plays a vital role in minimizing computation time, predicting the future status of individuals, providing reliable health services, prevention, healthy living, population health, early detection, and optimal management [ 133 , 158 , 171 ]. There are the BDM mechanisms’ objectives and requirements for different types of medical data [ 172 ]. Various research has presented many mechanisms for BDM in IoT-based healthcare that have advantages and disadvantages. Therefore, this subsection examines the articles (21 articles; 33.33%) that discussed the architectures or frameworks of BDM in IoT-based healthcare.

Rathore et al. [ 58 ] proposed Hadoop-based intelligent healthcare using a BDA approach. This system collected the big data and directed them to a 3-unit smart building for storing and processing. The units of this system are big data collection, Hadoop processing, and analysis and decision. This system used the 5-layer architecture for parallel, real-time, and offline processing. The layers of this architecture are the data collection, communication, processing, management, and service. The data collection layer includes data sensing, acquisition, buffering, and filtration. The big data are divided into small pieces in the processing layer, processed in parallel using HDFS and MapReduce, and stored. The management layer uses medical expert systems for processing the results and recommending corresponding actions.

Chui et al. [ 126 ] proposed a 6-layer architecture for patient behavior monitoring based on big data and IoT. Message queue, Apache Hadoop, behavior analytics, Mongo database, distributed stream processing, and exposer are the layers of this architecture. This architecture uses Hadoop for processing (descriptive, diagnostic, predictive, and prescriptive analytics), MongoDB for storing, Spark/Flink/Storm for stream processing, and Apache Kafka for breaking up the data stream into several partitions. Also, the authors have discussed the challenges of trust, security, privacy, and interoperability in the healthcare research field.

Ullah et al. [ 140 ] proposed a lightweight Semantic Interoperability Model for Big-Data in IoT (SIMB-IoT). The SIMB-IoT model has two main components: user interface and semantic interoperability. The semantic interoperability component is divided into three subcomponents: semantic interoperability, cloud services, and big data analytics. IoT data is collected and directed into an intelligent health cloud for online storage and processing. After processing, it sends suitable medicines to the patient’s IoT devices. This article used the SPARQL query to find hidden patterns.

Elhoseny et al. [ 173 ] presented a Parallel Particle Swarm Optimization (PPSO) algorithm for IoT big data analysis in cloud computing healthcare applications. This article aims are: optimize virtual machine selection and storage by using GA, PSO, and PPSO algorithms; real-time processing; and reducing the execution time. This architecture has four components: stakeholders’ devices; tasks; cloud broker; and network administrator. The cloud broker sends and receives requests to the cloud. The network administrator finds the optimal selection of virtual machines in the cloud for task scheduling.

Manogaran et al. [ 141 ] proposed a secured cloud-fog-based architecture for storing and processing real-time data for health care applications. This architecture has two sub-architectures: meta fog-redirection and grouping and choosing architectures. The meta fog-redirection architecture has three phases: data collection, data transfer, and big data storage. The data collection phase collected data from sensors in fog computing. The data transfer phase used the ‘s3cmd utility’ method for transferring data to Amazon S3.The big data storage phase used Apache Pig and Apache HBase for storage. The grouping and choosing architecture protects data and provides security services in fog and cloud environments. Also, this architecture used MapReduce to predict.

García-Magariño et al. [ 156 ] is an agent-based simulation framework for IoT BDA in smart beds. This framework has two layers: the primary mechanism for simulating sleepers' postures and the information's analyzer. The first layer provides the simulation of the poses of sleeper mechanisms. The second layer analysis collected data from the first layer. The agent types in this framework are sleeper agent, weight sensor agent, bed agent, observer agent, analyzer agent, stochastic sleeper agent, bed sleeper agent, restless sleeper agent, and healthy sleeper agent. This framework helps researchers to test different sleeper posture recognition algorithms, discusses other sleeper behaviors, and performs online or offline detection mechanisms.

Yacchirema et al. [ 59 ] proposed a 3-layer architecture for sleep monitoring based on IoT and big data at the network's edge. The layers of this architecture are the IoT layer, the fog layer, and the cloud layer. The IoT layer collected and aggregated the big data and directed them to the fog layer. The fog layer is responsible for connectivity and interoperability between heterogeneous devices, preprocessing the collected data, and sending notifications to react in real-time. The big data is stored, processed, and analyzed in the cloud layer for intelligent decision-making. This layer has three modules: data management, big data analyzer, and web application. This architecture used HDFS for data storage and Spark for offline and real-time processing.

BigReduce [ 137 ] is a cloud-based IoT framework for big data reduction for health monitoring in smart cities that focuses on reducing energy costs. This framework has two schemes: real-time big data reduction and intelligent big data decision-making. The big data reduction is made in two phases: at the time of acquisition and before transmission using an event-insensitive frequency content process.

Ma et al. [ 33 ] proposed a 3-layer architecture for the IoT big health system based on cloud-to-end fusion. The layers of this architecture are the big health perception layer, transport layer, and big health cloud service layer. In the big health perception layer, data are collected and preprocessed. The transport layer sends data to sensor nodes and receives data from the perception layer using network technologies. The big health cloud service layer has two sub-layers: the cloud service support and the cloud service application. The cloud service support sub-layer is responsible for compressing, storing, processing, and analyzing the real-time data. The cloud service application sub-layer is the interface between users and health networking. This sub-layer controls the sensor nodes and visualizes the big data.

Rathore et al. [ 61 ] proposed the 5-layer architecture for big data IoT analytics-based real-time medical emergency response systems. The data collection layer is responsible for data sensing, acquisition, buffering, filtration, and processing. This layer collected and aggregated data using a coordinator or relay node and transmitted them to a polarization mode dispersion. The communication layer provides device-to-device communication to various smart devices. The processing layer divides big data into small chunks. Each chunk is processed separately, aggregated, and stored. This article used MapReduce, HDFS, and Spark for data processing and analysis. The management layer is responsible for managing all types of outcomes using a medical expert system. The service layer is the interface between end-users and health networking. This architecture minimized the processing time and increased the throughput.

El‐Hasnony et al. [ 84 ] proposed a hybrid real-time remote patient monitoring framework based on mist, fog, and cloud computing. This article provided the 5-layer architecture for near real-time data analysis. The layers are the perception layer, the mist layer, the fog layer, the cloud layer, and the service provider layer. The mist layer is responsible for data filtering, data fusion, anomaly detection, and data transmission to the fog layer. The fog layer has done local monitoring and analysis, data aggregation, local storage, data pre-analysis, and data transmission to the cloud layer. The cloud layer implemented several data analytics techniques for intelligent decision-making and storage. This article presented a case study comparing traditional data mining techniques, including REPtree, MLP, Naive Bayes (NB), and sequential minimal optimization algorithms. The results showed that the REPtree algorithm achieved better accuracy, and the NB achieved the least time.

Harb et al. [ 106 ] proposed the 4-layer architecture for real-time BDA for patient monitoring and decision-making in healthcare applications. The layers of this platform are real-time patient monitoring, real-time decision and data storage, patient classification, and disease diagnosis, and data retrieval and visualization. The first layer is responsible for data ingestion using Kafka and Sqoop tools. The second layer processes and stores data using Spark and Hadoop HDFS. This layer preprocesses data and finds the missing records using MissRec (a script for Spark). The third layer is responsible for classification data using stability-based K-means, an adapted version of K-means clustering, and disease diagnosis using a modified version of the association rule mining algorithm. The last layer retrieves and visualizes data to understand the patient’s situation using Hive, SparkSQL, and Matplotlib.

Zhou et al. [ 62 ] proposed a data mining technology based on the IoT. The layers of the proposed functional architecture are the data acquisition layer, data transmission layer, data storage layer, and cloud service center layer. This article used the WIT120 system for data collection, the adaptive k-means clustering method based on the MapReduce framework for data preprocessing, HDFS for storing, and the GM (1,1) grey model for users’ health status prediction.

Hong-Tan et al. [ 90 ] proposed a real-time Ambient Intelligence assisted Student Health Monitoring System (AmIHMS). The data required by time ambient intelligence environments are collected from the WSN and sent to the cloud for handling. Their work developed a framework for real-time effective alerting of student health information. The AmIHMS architecture has three layers. The IoT layer collects health data from medical devices and sensors and saves it on one mobile computer or smartphone. The cloud layer receives the data through internet platforms such as 4G, 5G, LTE, etc., and executes the mining algorithms to extract relevant data for processing. The student health monitoring layer performs four stages to provide information and warnings about student health status. These stages include data retrieval, preprocessing, normalization, and classification/health status recognition.

Li [ 30 ] designed the fog-based Smart and Real-time Healthcare Information Processing (SRHIP) system. SRHIP architecture has three layers. IoT body sensor network layer performs data collection (health, environment, and locality), aggregation, compression, and encryption. Fog processing and computation layer use Spark and Hadoop ecosystem for information extraction, data normalization, rule engine, data filtration, and data processing. This layer performs the classification using the NB classifier. The cloud computation layer performs in-depth data analysis, storage, and decision-making. SRHIP minimizes the delay, transmission cost, and data size. This article uses hierarchical symmetric key data encryption to increase confidentiality.

The Improved Bayesian Convolution Network (IBCN) was proposed for human activity recognition [ 87 ]. The system architecture includes Wi-Fi and clouds onboard applications. The combination of a variable autoencoder with a standard deep net classifier is used to improve the performance of IBCN. This article used the convolution layers to extract the features and Enhanced Deep Learning (EDL) for security issues. IBCN provided the ability to download data via traditional radio frequency or low-power back-distribution communication. According to the experimental analysis, the proposed method allows the network to be continuously improved as new training sets are added and distinguishes between data-dependency and model-dependency. This architecture has high accuracy, versatility, flexibility, and reliability.

Sengupta and Bhunia [ 88 ] implemented a 3-layer IoT-enabled e-health framework for secure real-time data management using Cloudlet. The IoT layer uses IoT Hub for communicating with IoT devices. The Cloudlet layer is an intermediate layer between the IoT and cloud layers. This layer performs in-depth healthcare data analytics and processes. The cloud layer performs various analytics applications and processes queries. This framework uses SQLite for data storage in IoT Hub and Cassandra for future storing of sensed data. The result demonstrated that this framework has high efficiency, low data transmission time, low communication energy, data-packet loss, and query response time.

IBDAM [ 133 ] is an Intelligent BDA Model for efficient cardiac disease prediction in the IoT using multi-level fuzzy rules and valuable feature selection. This article used the open-source UCI database. First, it performs preprocessing on the UCI database, and the next step uses multi-level fuzzy rule generation for feature selection. IBDAM uses an optimized Recurrent Neural Network (RNN) to train the features. Finally, the features are classified into labeled classes according to the risk of evaluation by a medical practitioner. The results of this article demonstrate that this architecture has high performance and is quick and accurate.

Ahmed et al. [ 158 ] proposed an IoT-based health monitoring framework for pandemic disease analysis, prediction, and detection, such as COVID-19, using BDA. In this framework, the COVID-19 data set is collected from different data sources. Four data analysis techniques are performed on these data, including descriptive, diagnostic, predictive, and prescriptive. The experts opine on the results, and then users receive the results of these analyses through the internet and cloud servers. This article uses a neural network-based model for diagnosing and predicting the pandemic. The results of this article indicated that the accuracy, precision, F-score, and recall of the proposed architecture are better than AdaBoost, k-Nearest Neighbors (KNN), logistic regression, NB, and linear Support Vector Machine (SVM).

Ahanger et al. [ 71 ] proposed an IoT-based healthcare architecture for real-time COVID-19 data monitoring and predicting based on fog and cloud computing. This architecture has four layers. The data collection layer collects data from sensors and uses protocols to guarantee information security. The information classification layer classifies the information into four classes: health data, meteorological data, location data, and environmental data. The COVID-19-mining and extraction layer is responsible for splitting information into two groups using a fuzzy C-means procedure in the fog layer. The COVID-19 prediction and decision modeling layer use temporal RNN for estimating the results of the COVID-19 measure and a self-organization map-based technique to increase the perceived viability of the model. This article, in contrast to the existing methods, has high classification efficiency, viability, precision, and reliability.

Oğur et al. [ 109 ] proposed a real-time data analytics architecture for smart healthcare in IoT. This architecture has two domains. The software-defined networking-based WSN and RFID technology are used in the vertical domain, and data analytics tools, including Kafka, Spark, MongoDB, and NodeJS, are used in the horizontal domain. The collected data from WSN using RFID transmit to the Kafka platform using TCP sockets. The Kafka sends data to three consumers: The Apache Spark analysis engine that analyzes data in real-time; the NodeJS web application that visualizes patient data; and the MongoDB database that stores data. This article uses logistic regression and Apache spark MLlib for data classification. The result demonstrated this architecture has high performance and accuracy and is appropriate for a time-saving experimental environment.

Table 13 shows the result of the analysis of the articles. This table shows each article's architecture or framework name, OS name, programming language, advantages, and disadvantages. As you can see, layered architecture is the most important, with 14 articles.

3.2.1.2 BDM architectural/framework for IoT-based smart cities

According to the United Nations forecasting, about 67% of the world population will live in urban areas by 2050, resulting in environmental pollution, ecosystem destruction, energy shortage, emission reduction, and resource limitation [ 36 , 174 , 175 ]. Smart cities are large-scale distributed systems that could be a solution to overcoming these problems and improving intelligent services for residents [ 112 , 176 ]. Smart cities have many implemented sensing devices that generate large amounts of data. These data must be stored, processed, and analyzed to extract valuable information [ 177 ]. BDM plays a significant role in this context and facilitates better resource management and decision-making [ 176 ]. Many research focused on BDM mechanisms in IoT-based smart cities with different objectives, including improving monitoring and communication, real-time controlling, and increased quality attributes (such as reliability, throughput, energy conservation, accuracy, scalability, delay, bandwidth usage, etc.). Therefore, this subsection examines the articles (14 articles; 22.22%) that have discussed the architectures or frameworks of BDM in IoT-based smart cities.

Jindal et al. [ 85 ] propose a tensor-based big data processing technique for energy consumption in smart cities. This article aims to reduce the dimensionality of data and decrease the overall complexity. The proposed framework has two phases. The first phase is the 3-layer data gathering and processing architecture. The layers of this architecture are data acquisition, transmission, and processing. In the second phase, the collected data was represented in tensor form, and SVM was used to identify the loads to manage the demand response services in smart cities. The technique reduces data storage by 38%.

ESTemd [ 105 ] is a distributed stream processing middleware framework for real-time analysis using big data techniques on Apache Kafka. The layers of this framework are the data ingestion layer, the data broker layer (source), the stream data processing engine and services, the data broker layer (sink), and the event hub. The data broker layer is responsible for data processing and transformation, with the support of multiple transport protocols. The third layer does stream processing and consists of the predictive data analytics model and Kafka CEP operators. This framework helps with performance improvement through data integration and distributed applications' interoperability.

CPSO [ 115 ] is a self-adaptive preprocessing approach for big data stream classification. This approach handles four mechanisms: sub-window processing; feature extraction; feature selection; and optimization of the window size and feature picking. CPSO uses clustering-based PSO for data stream mining; the sliding window technique for data segmentation; statistical feature extraction for variable partitioning; correlation feature selection, and information gain for feature selection. The proposed approach improves its accuracy.

Rani and Chauhdary [ 72 ] proposed a novel approach for smart city applications based on BDA and a new protocol for mobile IoT. They presented the 5-layer architecture where the layers are: data source, technology, data management, application, and utility programs. The data source layer collects, compresses, and filters data. The technology layer is responsible for communication between sensor nodes, edge nodes, and base station. The management layer used MapReduce, SQL, and Hbase for analyzing, storing, and processing. The utility program layer used WSN and IoT protocols to work with the other layers. Also, this article presented a new protocol that reduces energy consumption, increases throughput, and reduces the delay and transmission time.

SCDAP [ 107 ] is the 3-layer BDA architecture for smart cities. The first layer is the platform that includes hardware clusters, the operating system, communication protocols, and other required computing nodes. The second layer is security. The last layer is the data processing layer that supports online and batch data processing. This layer has ten components: data acquisition; data preprocessing; online analytics; real-time analytics; batch data repository; batch data analytics; model management; model aggregation; smart application; and user interface. This architecture used Hadoop and Spark for data analysis. Also, this article presented a taxonomy of literature reviews based on six characteristics: focus, goal, organization, perspective, audience, and coverage.

Chilipirea et al. [ 80 ] proposed a data flow-based architecture for big data processing in smart cities. The architecture has seven steps: data sources, data normalization; data brokering; data storage; data analysis; data visualization; and decision support systems. This article used Extract, Transform, and Load (ETL) and Electronic Batchload Service (EBS) for normalizing the real-time and batch data. The data brokering step created the links between the collected data and the relevant context. This architecture used Hadoop for batch data processing and Storm for real-time data processing.

Gohar et al. [ 92 ] proposed a four-layer architecture for analyzing and storing data on the Internet of Small Things (IoST). The layers of this architecture are the small things layer, the infrastructure layer, the platform layer, and the application layer. The first layer collected data by using the LoRa gateway from LoRa devices. The infrastructure layer provides connectivity to devices by using the Internet. The platform layer is responsible for data preprocessing. For processing, this layer employs Max–Min normalization, the Kalman filter, the Round-Robin load balancing technique, the Least Slack Time algorithm (LST), the divide-and-conquer approach for aggregation, and NoSQL databases for storage. In the last layer, data is visualized for decision-making. This article implemented the architecture by using Hadoop, Spark, and GraphX. In this article, throughput has increased with the rise in data size.

Farmanbar and Rong [ 113 ] proposed an interactive cloud-based dashboard for online data visualization and a data analytics toolkit for smart city applications. The proposed architecture has three layers: the data layer, application and analysis layer, and presentation layer. The data layer is the core of the architecture and contains data acquisition units, data ingestion, data storage, and data access. This architecture used Logstash for data ingesting, Elasticsearch for storing, and Kibana for accessing and real-time monitoring. This platform has been tested on five datasets, including transportation data, electricity consumption, cargo e-bikes, parking, vacancies, and energy. The results showed this architecture is robust, scalable, and improves communication between users and urban service providers.

He et al. [ 116 ] proposed a big data architecture to achieve high Quality of Experience (QoE) performance in smart cities. This architecture has three plans: the data storage plane, the data processing plane, and the data application plane. This article used MongoDB and HDFS for data storing and Spark and the deep-learning-based greedy algorithm for data processing. The simulation result indicated that the proposed architecture's accuracy, precision, and recall are better than SVM and KNN.

Khan et al. [ 128 ] proposed an SDN-based 3-tier architecture that includes data collection, data processing and management, and an application layer for real-time big data processing in smart cities with two intermediate levels that work on SDN principles. This architecture uses Spark and GraphX with Hadoop for offline and real-time data analysis and processing. Also, this article proposed an adaptive job scheduling mechanism for load balancing and achieving high performance. The results showed that when clusters and processing time increase, the proposed system's performance also increases.

IoTDeM [ 73 ] is the IoT big data-oriented multiple edge-cloud architectures for MapReduce performance prediction with varying cluster scales. This architecture consists of three parts: multiple edge cloud redirectors, an edge cloud-based big data platform, and a centralized cloud-based big data platform. This architecture used historical job execution records and Locally Weighted Linear Regression (LWLR) techniques for predicting jobs' executing times and Ceph for storing them. Because of Ceph, there was no need to transfer data to the newly added slave node. This article validated the accuracy of the proposed model by using the TESTDFSIO and Sort benchmark applications in a general implementation scenario based on Hadoop2 and Ceph and achieved an average relative error of less than 10%.

Ahab [ 112 ] is a generic, scalable, fault-tolerant, and cloud-based framework for online and offline big data processing. This framework has four components: the user API, repositories, messaging infrastructure, and stream processing. The API directs the published data streams from different sources. Ahab uses the component, stream, policy, and action repositories for storing data streams, management policies, and actions. Ahab uses distributed messaging for handling data streams, minimizing unnecessary network traffic. Also, it allows the components to choose an appropriate communication point freely. The Ahab architecture has two layers: the streaming and service layers. The streaming layer is implemented as a lambda architecture. This layer has three sub-layers for data stream processing: the batch layer, the speed layer, and the serving layer. The HDFS and Apache Spark are used for data storing and stream processing. The service layer is responsible for analyzing, managing, and adapting components.

Mobi-Het [ 81 ] is a mobility-aware optimal resource allocation architecture for remote big data task execution in mobile cloud computing. This article uses the SMOOTH random mobility model to propound the free movement of mobile devices and estimate their speed and direction. Mobi-Het has three layers: mobile devices, cloudlets, and the master cloud. The mobile devices component has a decision-maker module that decides whether tasks should be executed remotely or locally. The master cloud component implements the resource allocation algorithm. This article has a low execution time, high execution reliability, and efficiency in timeliness.

Hossain et al. [ 132 ] proposed a knowledge-driven framework that automatically selects the suitable data mining and ML algorithms for a dynamic IoT smart city dataset. The system architecture has four units: data Knowledgeextraction, extactGoalKnowledge, extractAlgoKnowledge, and matchKnowledge. The framework's inputs are three key factors: datasets, goals, and data mining and ML algorithms. This article discussed both supervised and unsupervised data mining. The results show that this framework reduces computational time and complexity and increases performance and flexibility while dynamically choosing a high-accuracy solution.

Table 14 shows the result of the analysis of the articles. This table shows the architecture or framework name, OS name, programming language, advantages, and disadvantages of each article. As you can see, layered architecture is the most important, with 13 articles.

3.2.1.3 BDM architectural/framework for IoT-based smart home/ building

BDM mechanisms and IoT (architecture/ frameworks) have a crucial role in smart home/building, including processing data collected by the home sensors; analyzing, classifying, monitoring, and managing energy consumption and saving; intelligently identifying user behavior patterns and home activities; and increasing safety and comfort at home [ 76 ]. This subsection presents a review of the articles (8 articles; 12.70%) that have discussed the architectures or frameworks of BDM in the IoT-based smart home/ building.

Al-Ali et al. [ 68 ] proposed a smart home energy management architecture using IoT and BDA approaches. This architecture is divided into two sub-architectures: hardware architecture and software architecture. The hardware architecture includes sensors and actuators, high-end microcontrollers, and server blocks. The software architecture comprises the data acquisition module on the edge device, a middleware module, and a client application module. The first module monitors and collects data and transmits them to the middleware module. The second module uses several tools to provide different services, including facilitating communication between edge devices and middleware, data storage, data analysis, and sending results to the requester. The third module develops the front-end mobile user interface using a cross-platform integrated development environment. This article is evaluated using a prototype. The results showed the proposed architecture has high scalability, security, privacy, throughput, and speed.

Silva et al. [ 55 ] proposed a real-time BDA embedded architecture for the smart city with the RESTful web of things. This article integrated the web and smart control systems using a smart gateway system. The proposed architecture consists of four levels: data creation and collection; data processing and management; event and decision management; and application. The data processing and management level utilized HDFS for primary data storing, MapReduce for processing, Hbase to speed up the processing, and HIVE for data querying and managing. The event and decision management level classified two events as service and resource events based on the processed information. The application level remotely provides access to the smart city services and has three sub-layers: departmental layer, services layer, and sub-services layer. This article has high performance and throughput, low processing time, and minimizes energy consumption.

Khan et al. [ 57 ] proposed a scheduling algorithm, an IoT BDA architecture, and a real-time platform for managing sensors' energy consumption. This architecture has four steps: appliance discovery, sensor configuration and deployment, event management and scheduling, and information gathering and processing. Appliances are identified and classified in the first step based on user availability and usage time. The second step used Poisson distribution for sensor distribution in an IoT environment. In the third step, the appliance sleep-scheduling mechanism is presented for job scheduling. In the last step, the collected data from sensors were directed to Hadoop, Spark, and GraphX for processing and analysis. This step used HDFS for data storage. This article minimized total execution time and energy consumption.

HEMS-IoT [ 76 ] is a 7-layer architecture based on big data and ML for in-home energy management. The layers of this architecture are the presentation layer, IoT services layer, security layer, management layer, communication layer, data layer, and device layer. The management layer uses the J48 ML algorithm and the Weka API for energy consumption reduction and user behavior pattern extraction. This layer also classifies the data and houses based on energy consumption using the C4.5 algorithm. The IoT services layer provides different REST-based web services. The security layer guarantees data confidentiality. This layer has two components, namely authorization and authentication. This article uses RULEML and Apache Mahout to generate energy-saving recommendations.

Yassine et al. [ 56 ] proposed a platform for IoT smart homes based on fog and cloud computing. The components of the proposed platform are smart home components, IoT management and integration services, fog computing nodes, and cloud systems. The smart home component is divided into three tiers. The three tiers are: 1) the cyber-physical tier is responsible for interacting with the outside world through the second tier; 2) the connectivity tier is responsible for communicating with the smart home; and 3) the context-aware tier consists of user-defined rules and policies that create a privacy and security configuration. The IoT management and integration services component is in charge of providing interoperability, handling requests, authentication, and service registration. The fog computing nodes performed preprocessing, pattern mining, event detection, behavioral and predictive analytics, and visualization functions. The cloud system is responsible for storing and performing historical data analytics.

Luo et al. [ 131 ] proposed a 4-layer ML-based energy demand predictive model for smart building energy demands. Firstly, the sensitization layer collected data and transferred them to the storage layer. The storage layer performed data cleaning and storing. The model’s smart core is in the analytics support layer, where Artificial Neural Network (ANN) and k-means clustering are used for identifying features in weather profile patterns. The service layer is an interface between the proposed model and the smart building management system. The proposed model improved accuracy and decreased mean absolute percentage error.

Bashir et al. [ 110 ] proposed an Integrated Big Data Management and Analytics (IBDMA) framework for smart buildings. The reference architecture and the metamodel are two phases of this framework. The reference architecture has eight layers: data monitoring, sourcing, ingestion, storage, analysis, visualization, decision-making, and action. People, processes, technology, information, and facility are the components of the metamodel phase. The core component of the metamodel is people (IoT policymakers, developers, and residents of intelligent buildings). The process component includes data monitoring, sourcing, ingesting, storage, decision-making, analytics, and action/control. The technology component consists of the tools and software packages to implement the IBDMA. Some of these tools are Apache Flume for data ingesting; HDFS for data storing; Apache Spark for data analysis; Microsoft Power BI for static data visualization; and Elasticsearch and Kibana for near-real-time data visualization. The information element manages disasters and controls various facilities based on results obtained by using the technology stack. The last element is the facility that improves the comfort, safety, and living conditions for the people of the building.

Table 15 shows the result of the analysis of the articles. This table shows each article's architecture or framework name, OS name, programming language, advantages, and disadvantages. As you can see, layered architecture is the most important, with five articles.

3.2.1.4 BDM architectural/framework for IoT-based intelligent transport

Safety, reliability, fault diagnosis, data transmission, and early warning in the intelligent transport system are critical for decision-making [ 178 ]. The intelligent transport system uses digital technologies, sensor networks, ML, and BDA mechanisms to overcome the challenges, including accident prevention, road safety, pollution reduction, automated driving, traffic control, intelligent navigation, and parking systems [ 179 ]. This subsection presents a review of the articles (2 articles; 3.17%) that have discussed the architectures or frameworks of BDM in IoT-based intelligent transport.

SMART TSS [ 129 ] is a BDA modular architecture for intelligent transportation systems. This architecture has four units: a big data acquisition and preprocessing unit, a big data processing unit, a big data analytics unit, and a data visualization unit. The big data processing unit stored the offline data in the cloud system for future analysis. The online data is sent to the extraction and filtration unit for load balancing on NoSQL databases. The big data analytics unit uses the map-reduce mechanism for analysis. This article uses Hadoop, Spark, and GraphX for big data processing and analysis. The throughput of the proposed system increases with increasing data size and has low accuracy and security.

Babar and Arif [ 89 ] proposed a real-time IoT big data analytics architecture for the smart transportation system. This architecture has three phases: big data organization and management, big data processing and analysis, and big data service management. The first phase performed data preprocessing, including big data detection, logging, integration, reduction, transformation, and cleaning. This phase used the divide-and-conquer technique for data aggregation, the Min–Max method for data transformation, and the Kalman filter technique for data cleaning. The second phase used Hadoop for big data processing, HDFS, Hive, and Hbase for data storage, and Spark for data stream analysis. This phase performed load balancing that caused increased throughput, minimized processor use, and reduced response time. The third phase is responsible for intelligent decision-making and event management.

Table 16 shows the result of the analysis of the articles. This table shows the architecture or framework name, OS name, programming language, advantages, and disadvantages of each article. As you can see, layered architecture is the most important, with two articles.

3.2.1.5 BDM architectural/framework for IoT-based traffic control and energy

Two reviewed articles discussed the architectures or frameworks of BDM in IoT-based traffic control and energy and used the ML for this purpose. ML4IoT [ 108 ] is a container-based ML framework for IoT data analytics and coordinating ML workflows. This framework aims to define and automate the execution of ML workflows. The proposed framework uses several types of ML algorithms. The ML4IoT framework has two layers: ML4IoT data management and ML4IoT core. The ML4IoT core layer trains and deploys ML models and consists of five components: a workflow designer, a workflow orchestrator, a workflow scheduler, container-based components, and a distributed data processing engine. ML4IoT data management is responsible for data ingesting and storing and has three sub-components: a messaging system, a distributed file system, and a NoSQL database. The results of this article reveal that this framework has high elasticity, scalability, robustness, and performance. Furthermore, Chhabra et al. [ 111 ] proposed a scalable and flexible cyber-forensics framework for IoT BDA analytics with high precision and sensitivity. This framework consisted of four modules: the data collector and information generator; feature analytics and extraction; designing ML models; and analyzing models on various efficiency matrices. This article used Google’s programming model, MapReduce, as the core for traffic translation, extraction, and analysis of dynamic traffic features. Also, they presented a comparative study of globally accepted ML models for peer-to-peer malware analysis in mocked real-time.

Table 17 shows the result of the analysis of the articles. This table shows the architecture or framework name, OS name, programming language, advantages, and disadvantages for each article. As you can see, the component-based architecture is the most important, with two articles.

3.2.1.6 BDM architectural/framework for IoT-based urban planning

To improve the quality, plan, design, sustainability, living standards, dynamic organization, mobility of urban space and structure, and maintain the urban services, BDM is responsible for offline and online aggregation, managing, processing, and analyzing the large amounts of big data in urbanization [ 180 , 181 , 182 ]. Rathore et al. [ 51 ] proposed the 4-layer IoT-based BDA architecture for smart city development and urban planning. The first layer generated, aggregated, registered, and filtrated data from various IoT sources. Using communication technologies, the second layer created communication between sensors and the relay node. The third layer used HDFS, Hbase, Hive, and SQL for storage; MapReduce for offline analysis; and Spark, VoltDB, and Storm for real-time analysis. The last layer is responsible for showing the study results for intelligent and fast decision-making. The results show that the architecture provides efficient outcomes even on IoT big data sets. Throughput has increased with the rise in data size, and the processing time has decreased.

Silva et al. [ 63 ] proposed a reliable 3-layer BDA-embedded architecture for urban planning. The layers of this architecture are data aggregation, data management, and service management. The purpose of this article is to increase throughput and minimize processing time. The real-time data management layer is the main layer and performs data filtration, analysis, processing, and storing. This layer used data filtration and min–max normalization techniques to improve energy data. This architecture used MapReduce for offline data processing, Spark for online data processing, and Hbase for storing.

Table 18 shows the result of the analysis of the articles. This table shows the architecture or framework name, OS name, programming language, advantages, and disadvantages for each article. As you can see, layered architecture is the most important, with two articles.

3.2.1.7 BDM architectural/framework for other IoT-based applications

This subsection presents a review of the articles (14 articles) that have discussed the architectures or frameworks of BDM in other IoT-based applications. These IoT applications are smart IoT systems (4 articles), smart flood (1 article), smart farms (2 articles), disaster management (1 article), laundry (1 article), smart pipeline (1 article), network traffic (1 article), digital manufacturing (1 article), smart factory (2 articles).

Al-Osta et al. [ 121 ] proposed an event-driven and semantic rules-based approach for IoT data processing. The main levels of this system are sensor, edge, and cloud levels. This article has two purposes: reducing the required resources and the volume of data before transfer to the cloud for storage. The collected data is first aggregated, filtered, and classified at the gateway level. This causes a saving in bandwidth and minimizes the network traffic. This approach used semantic rules for data filtering. It also employed a complex event processing module to analyze input events and detect processing priority.

Wang et al. [ 148 ] proposed a 3-layer edge-based architecture and a dynamic switching algorithm for IoT big data analytics. The layers of this architecture are the cloud layer, edge layer, and IoT layer. The edge layer performed some functions, including identifying IoT applications, classifying them, and sending classification results to the cloud layer. The LibSVM method is used for IoT application identification and classification based on system status and requirements. Also, this article presented a new algorithm, namely the dynamic switching algorithm, for task offloading from cloud to edge based on the delay and network conditions. This algorithm performed task offloading based on classification results. The results showed the proposed architecture reduced delay, processing time, and energy consumption.

IODML-BDA [ 124 ] is a model for Intelligent Outlier Detection in Apache Spark using ML-powered BDA for mobile edge computing. This model performs four steps: data preprocessing, outlier detection, feature selection, and classification. This article employs an Adaptive Synthetic Sampling (ADASYN)-based technique for outlier detection, the Oppositional Swallow Swarm Optimization (OSSO) for feature selection, and a Long Short-Term Memory (LSTM) model for classification. This model has high performance and accuracy in BDA.

Kumar et al. [ 3 ] presented a novel 4-layer architecture for IoT big data management in cloud computing networks and a collaborative filtering recommender system. The information layer collects data and transmits them to the second layer. The transport layer uses GPRS/CDMA, wireless RFID, or Ethernet channels for communication and data uploading in the data mining layer. The data mining layer utilizes the ML method for data analysis. The application layer is responsible for data visualization based on extracted information from the data mining layer. The article also proposed a collaborative filtering algorithm to improve the prediction accuracy based on the time-weighted decay function and asymmetrical influence degree. The result of this article demonstrated that this architecture has high accuracy.

Sood et al. [ 75 ] proposed a 4-layer flood forecasting and monitoring architecture based on IoT, High-Performance Computing (HPC), and big data convergence. The IoT layer is responsible for IoT device installation and data collection. The fog computing layer reduces the latency of application execution when predicting the real-time flood. The data analysis layer received, stored, and analyzed the collected data. This layer used Singular Value Decomposition (SVD) for data reduction and a K-mean clustering algorithm to estimate the flood situation and rating. Also, Holt-Winter’s forecasting method is utilized to forecast the flood. The last layer is the presentation layer, which generates information for decision-making. The results showed the proposed architecture reduced latency, complexity, completion time, and energy consumption.

Muangprathub et al. [ 79 ] proposed a WSN system for agriculture data analysis based on the IoT for watering crops. This system consists of three components. The hardware component collected data and sent them to the web application for real-time analysis. This component is responsible for data preprocessing, data reduction by the equal-width histograms technique, data modeling/discovery by association rules mining technique, and solution analysis. The web application manages real-time information. The mobile application component controlled crop watering remotely. The architecture of this system has three layers: the environmental data acquisition layer, the data, and communication layer, and the application layer. This system can help to reduce costs and increase agricultural productivity.

Al-Qurabat et al. [ 65 ] proposed a two-level system for data traffic management in smart agriculture based on compression and Minimum Description Length (MDL) techniques. The first level is the sensor node level. This level monitors the features of the environment using a lightweight lossless compression algorithm based on Differential Encoding (DE) and Huffman techniques. The second level is the edge gateway level. This level is responsible for processing, analyzing, filtering, storing, and sending the data to the cloud, and minimizes the first level dataset using MDL and hierarchical clustering. The results demonstrated the suggested method has a high compression ratio and accuracy and decreases data and energy consumption.

Shah et al. [ 53 ] proposed the 5-layer architecture for IoT BDA in a disaster-resilient smart city. The purpose of this architecture is to store, mine, and process big data from IoT devices. This architecture's layers include data resource, transmission, aggregation, analytics and management, and application and support services. This architecture used Apache Flume and Apache Sqoop for unstructured and structured data collection; Hadoop and Spark for real-time and offline data analysis; and HDFS for data storage. The proposed implementation model comprises data harvesting, data aggregation, data preprocessing, and a big data analytics and service platform. This article used a variety of datasets for validation and evaluation based on processing time and throughput.

Liu et al. [ 14 ] proposed a cloud laundry business model based on the IoT and BDA. This model used big data analytics, intelligent logistics management, and ML techniques for big data analytics. This model minimized human interference and increased system efficiency.

Tang et al. [ 7 ] proposed the 4-layer distributed fog computing-based architecture for big data analysis in smart cities. The layers of this architecture are the data center on the cloud layer, intermediate computing nodes layer, edge devices layer, and sensing networks on the critical infrastructure layer. This architecture reduces the communication bandwidth and data size. First, data was collected from the fiber sensor network and transmitted to the edge computing nodes layer. This layer performed two tasks: identifying potential threat patterns and feature extraction using supervised and non-supervised ML algorithms. The intermediate computing nodes layer used the hidden Markov model for big data analysis and hazardous event detection. The results showed the proposed architecture reduced the service response time and the number of service requests submitted to the cloud.

Kotenko et al. [ 136 ] introduced a framework for security monitoring mobile IoT based on big data processing and ML. This framework consists of three layers: 1) extraction and decomposition of a data set using the heuristic approach; 2) compression of feature vectors using Principal Component Analysis (PCA); and 3) learning and classification using the SVM k-nearest neighbor’s method, Gaussian NB, artificial neural network, and decision tree. This framework has high performance and accuracy in the detection of attacks.

Bi et al. [ 157 ] proposed a new enterprise architecture that integrates IoT and BDA for managing the complexity and stability of the digital manufacturing system. This article used Shannon entropy to measure the complexity of a system based on the number of events and the probabilities of event occurrences. This architecture performs three processes: data acquisition, management, and utilization. The result of this article demonstrated that this architecture decreases the system complexity and increases flexibility, resilience, responsiveness, agility, and adaptability.

Yu et al. [ 118 ] presented a BDA and IoT-based framework for health state monitoring in a smart factory. This framework consists of four phases. The data ingestion phase is responsible for extracting different data types, managing data collection, data security, data transformation using a secure file transfer protocol, and data storage issues. The big data management phase uses optimized HDFS for data storage on the cloud nodes and processing using Apache Spark. The data preparation phase performs sensor selection and noise detection processing to produce high-quality data. This phase uses the high-variance feature removal method for feature selection and a novel method for noise detection. The predictive modeling phase has four stages: PCA model training, streaming anomaly detection, contribution analysis, and alarm sequence analysis.

Kahveci et al. [ 183 ] proposed a secure, interoperable, resilient, scalable, and real-time end-to-end BDA platform for IoT-based smart factories. The platform architecture has five layers and several components that perform data collection, data integration, data storing, data analytics, and data visualization. The layers of architecture are the control and sensing layer, the data collection layer, the data integration layer, the data storage and analytics layer, and the data presentation layer. All kinds of sensing and control activities are performed in the first layer. The data collection layer communicates with the first layer through a multi-node client/server architecture. The data integration layer uses the RESTful application program interface to transfer data collected to the data storage layer. The data storage layer uses InfluxDB for industrial metrics and events. Using this architecture, production line performance is improved, bottlenecks are identified, product quality is improved, and production costs are reduced.

Table 19 shows the result of the analysis of the articles. This table shows the architecture or framework name, OS name, programming language, advantages, and disadvantages for each article. As you can see, layered architecture is the most important, with 14 articles.

3.2.2 BDM architectural/framework for IoT paradigms

Another category presented in this article is BDM architectures and frameworks in two important IoT paradigms, i.e., Social Internet of Things (SIoT) and Multiple Internet of Things (MIoT). SIoT is the integration of the IoT with social networking that leads to improved scalability in information and service discovery, trustworthy relationships, security, performance, and high network navigability [ 91 , 184 ]. The SIoT establishes relationships and interactions between human-to-human, human-to-object, and object-to-object social networks in which humans are considered intellectual and relational objects [ 185 , 186 ]. The types of relationships.

between smart, complex, and social objects in SIoT are parental object relationships, co-location object relationships, co-work object relationships, ownership object relationships, social object relationships, stranger object relationships, guest object relationships, sibling object relationships, and service object relationships [ 187 , 188 ]. A MIoT is a collection of connected things that are different kinds of relationships and objects.

In contrast to SIoT, the number of relationships in MIoT is not predefined. Therefore, SIoT is a specific case of MIoT where the number of possible relationship types is limited [ 187 ]. The MIoT paradigm has advantages over the IoT and SIoT. IoT can be divided into multiple networks of interconnected smart objects through MIoT. The MIOT can handle situations where the same objects behave differently in different networks and allows objects from various networks to communicate without being directly connected [ 189 ]. Social objects in the SIoT and MIoT can perform tasks, including physical condition detection, data collection, information exchange, big data processing and analysis, and visualization for decision-making, predicting human behavior, and increasing efficiency and scalability. Due to the heterogeneous nature of communication and social networks, which generate high volume, multi-source, dynamic, and sparse data from SIoT and MIoT objects, the BDA is a vital issue in these paradigms. For BDA in SIoT and MIoT, a large amount of memory, power processing, and bandwidth are required to store, define, process, predict, and assist humans for a limited time [ 64 , 91 ]. Different researchers have examined BDA in these paradigms in various ways.

Paul et al. [ 91 ] proposed a system called SmartBuddy that performs the BDA for SIoT-based smart city data to define real-time human dynamics. This architecture has three domains: the object domain, the SIoT server domain, and the application domain. The object domain collects the data and sends them to the SIoT server for balancing, storing, querying, processing, defining, and predicting human behavior. The application domain has four main components: security, cloud server, results in storage devices, and data server. This domain compilation is the result of the SIoT server domain. This article uses MapReduce programming for offline data analysis and Apache Spark for real-time analysis. SmartBuddy has high throughput and applicability.

HABC [ 52 ] is a Hadoop-based architecture for social IoT big data feature selection and analysis. This architecture has four layers: data collection, communication, feature selection and processing, and service. The data collection layer collected, registered, and filtered data. The communication layer provided end-to-end connectivity to various devices and used the Kalman filter to remove noise. The feature selection and processing layer used MapReduce for data analysis and HDFS, HBSE, and HIVE for manipulation and storing. The Artificial Bee Colony (ABC) is used for feature selection. The results indicate that the architecture increases throughput and accuracy and is more scalable.

Lakshmanaprabu et al. [ 64 ] proposed a hierarchical framework for feature extraction in SIoT big data using the MapReduce framework and a supervised classifier model. This framework has five steps: SIoT data collection, filtering, database reduction, feature selection, and classification. This article used the Gabor filter to reduce the noisy data, Hadoop MapReduce for database reduction, Elephant Herd Optimization (EHO) for feature selection, and a linear kernel SVM-based classifier for data classification. The result showed the proposed architecture has high maximum accuracy, specificity, sensitivity, and throughput.

Socio-cyber network [ 66 ] is the 4-layer architecture that integrates the social network with the technical network for analyzing human behavior using big data. This architecture uses the user's geolocation information to make friendships and graph theory to examine the trust index. The data generation layer is responsible for data collection, aggregating, registration, and filtration. The communication layer provides end-to-end connectivity to various devices. This layer creates a graph of data, and when new data are added to the system, this graph is updated. The data storage and processing layer perform the load balancing algorithm and graph processing. This layer uses MapReduce for data processing, the Spark GraphX tool for real-time analysis, and HDFS for data storage. This article uses the Knowledge Pyramid for knowledge extraction. The service layer shows the result to users.

Shaji et al. [ 120 ] presented a 5-phase approach for big data classification in SIoT. The phases of this approach are the data acquisition phase, data filtering phase, reduction phase, feature selection phase, and classification phase. This article uses an adaptive Savitzky–Golay filter for filtering and eliminating noisy data; the Hadoop MapReduce framework for data reduction; a modified relief technique for optimal feature selection; and a deep neural network-based marine predator algorithm for classification. This article has high accuracy, precision, specificity, sensitivity, throughput, and low energy consumption.

Floris et al. [ 67 ] proposed a 4-layer architecture based on SIoT to deploy a full-stack smart parking solution. The layers of this architecture are the hardware layer, virtualization layer, aggregation layer, and application layer. The hardware layer collected data and consisted of a vehicle detection board, Bluetooth beacon, data transmission board, and concentrator. The SIoT paradigm is implements in the virtualization layer using device virtualization. ML algorithms are implemented in the aggregation layer for data aggregation and data processing. The application layer includes the management platform that supports the control dashboard for smart parking management and the Android App for the citizens.

Cauteruccio et al. [ 166 ] presented a framework for anomaly detection and classification in MIoT scenarios. This framework investigated two problems: the anomaly effects analysis on the MIoT and the source of the anomaly detection. The anomalies in MIoT are divided into three categories: presence anomalies versus success anomalies, hard anomalies versus soft anomalies, and contact anomalies versus content anomalies.

Lo Giudice et al. [ 189 ] proposed a definition of a thing’s profile and topic-guided virtual IoT. The profile of a thing has two components: a content-based component (past behavior) and a collaborative filtering component (principal characteristics of those things it has previously interacted with the most). This article uses a supervised and unsupervised approach to build topic-guided virtual IoTs in a MIoT scenario. Table 20 shows the result of the analysis of the articles. The architecture or framework name, the OS name, programming language, advantages, and disadvantages are shown for each article in this table. As you can see, layered architecture is the most important, with five articles.

3.3 Categories based on quality attributes

Systems have different attributes generally divided into qualitative or functional attributes and non-qualitative or non-functional attributes. This section considers the quality attributes of the selected articles. Quality attributes indicate the system’s characteristics, operating conditions, and constraints. There are different software quality models, such as McCall [ 190 ], Bohem [ 191 ], ISO/IEC9126, and FURPS [ 192 ]. As far as we know, no systematic article has completely categorized articles based on qualitative characteristics. Therefore, this paper categorized the selected articles based on 18 qualitative attributes presented in Table 21 . In this table, the first column shows the names of these 18 quality attributes. The reviewed articles used these quality attributes to show the characteristics, quality attribute analysis, and performance analysis of the proposed approaches, architectures, and frameworks and comparison with other works. Performance attributes have been analyzed in different articles based on different criteria. The reviewed articles utilized 12 quality attributes for performance attribute analysis. These quality attributes are load balancing, energy conservation, network lifetime, processing/execution time, response time, delay, CPU usage, memory usage, bandwidth usage, throughput, latency, and concurrency. In Table 21 , ↓ indicates the reduction of that quality characteristic and ↑ indicates the increase of that quality characteristic. The second column in this table shows the articles that have used these features. The performance, efficiency, accuracy, and scalability attributes are the most critical quality attributes, with 79, 62, 58, and 47 articles, respectively. From another point of view, the reference model of standard software quality attributes, i.e., ISO 25010, has been used to classify articles based on quality attributes. Table 22 shows the articles' classification according to this standard. In the following, some quality attributes and their importance will be defined.

Performance: Performance refers to the ability of BDM techniques in the IoT to provide results and services with high load balancing, energy conservation, throughput, concurrency, low processing/execution time, delay, CPU/memory/ bandwidth usage, and latency.

Feasibility: Feasibility refers to the ability to perform successfully or study the current mode of operation, evaluate alternatives, and develop BDM techniques in the IoT.

Scalability: Scalability refers to the ability of BDM techniques in the IoT to exploit increasing computing resources effectively to maintain service quality when the real data volumes increase. BDM techniques in IoT must be scalable in performance and data storage. Some methods and advanced systems are used to improve the scalability of big data analysis, like parallel implementation, HPC systems, and clouds [ 193 ].

Accuracy: Accuracy refers to the ability to describe data and represent a real-world object or event correctly [ 194 ]. In the reviewed articles, various definitions of accuracy are provided, including clustering accuracy, classification accuracy, the accuracy of features selecting/extracting, and the accuracy of the prediction model. Each of these cases is evaluated in different ways.

Efficiency: Efficiency refers to BDM techniques in IoT with minimum energy and response time and high throughput, accuracy, and performance.

Reliability: Reliability refers to the ability of BDM techniques in the IoT to apply the specified functions under specified conditions and within the expected duration.

Availability: The main goal of many researchers is the availability of information and their analysis from heterogeneous data sources. Availability is one of the components of service trust and is part of reliability.

Interoperability: Interoperability refers to the ability to interconnect and communicate among smart objects, heterogeneous IoT devices, and different operating systems. Low-cost device interoperability is a vital issue in IoT [ 53 , 54 , 195 ].

Flexibility: Flexibility refers to the capacity of BDM techniques in the IoT to be adapted for different environments and situations to face external changes [ 196 ].

Robustness: Robustness refers to a stable BDM system in the IoT that can function despite erroneous, exceptional, or unexpected inputs and unexpected events.

3.4 Big data analytics types in IoT

There are different types of analytics. This study uses Gartner’s classification, Footnote 2 which includes four types of analysis: descriptive analysis (“what happened?”), diagnostic analysis (“why did it happen?”), predictive analysis (“What could happen?”), and prescriptive analysis (“What should we do?”). In descriptive analytics, historical business data is analyzed to describe what happened in the past. Diagnostic analytics investigates and identifies the causes of trends and why they occurred. The goal of predictive analytics is to forecast the future using a variety of statistical and ML techniques. Prescriptive analytics proposes the best action to take to accomplish a business’s objective using the data collected from descriptive and predictive analytics for decision-making based on future situations [ 197 ].

This paper investigates the applied methods for data analysis and categorizes them based on the type of analysis these methods provide. Organizations need statistics, AI, deep learning, data mining, prediction mechanisms, etc., for BDA and to evaluate the data [ 198 ]. The articles used ML algorithms to perform various analyses in the steps of BDA. ML algorithm is an appropriate approach or tool for BDA; decision-making; meaningful, precise, and valuable information extraction; and detecting hidden patterns in big datasets [ 199 , 200 ]. Utilizing the ML algorithms in BDA has advantages such as improving and optimizing BDM processes; heterogeneous big data analysis; sustainability; fault detection, prediction, and prevention; accurate and reliable real-time processing; resource management and reduction; and increased quality prediction, visual inspection, and productivity in IoT applications [ 83 , 201 ]. These algorithms are divided into four types: supervised, semi-supervised, unsupervised, and reinforcement ML algorithms [ 53 , 202 ]. Table 23 shows the categorization of articles based on BDA types. The most common tactics that the selected articles use for BDM in the IoT include classification (51 articles), simulation (38 articles), optimization (30 articles), and clustering (25 articles).

The reason for using more classification algorithms is that they help to categorize unstructured and high-volume data. Therefore, BDM in the IoT is faster and more efficient. Before classification begins, it must optimize the classification algorithm's inputs. Data reduction strategies extract optimal and required data from a large amount of data. These strategies include dimensionality reduction, numerosity reduction, and data compression. Some reviewed articles used Principal Components Analysis (PCA) to standardize, reduce the data redundancy and dimensionality, reduce the cost and processing time, and maintain the original data [ 69 , 114 , 118 , 135 , 136 ]. Also, the authors in [ 160 ] used the fuzzy C-means algorithm to reduce the amount of data. Feature selection methods improve classification accuracy and reduce the number of features in BDA. The collected data from IoT applications and monitoring systems are usually anomalous, and it is difficult to distinguish between the original data and the anomaly [ 201 ]. The anomaly and outlier data reduce the accuracy of the classification and prediction models. For instance, NRDD-DBSCAN [ 114 ], DBSCAN-based outlier detection [ 83 ], GA, and One-Class Support Tucker Machine (OCSTuM) [ 122 , 124 ] are some of the high-robust, high-performance, and anti-noisy methods for anomaly detection that are presented in reviewed articles.

SVM is the most common method based on classification (10 articles) for BDM in the IoT in supervised classification. SVM is a non-parametric, memory-efficient, error-reduction classification method that performs well in theoretical analysis and real-world applications. It can model non-linear, complex, and real-world problems in high-dimensional feature space [ 2 , 69 , 203 ]. However, SVM is difficult to interpret, has a high computational cost, and is not scalable [ 204 ]. In unsupervised classification, the k-means clustering algorithm is the most common strategy (6 articles). The standard k-mean clustering algorithm is a simple partitioning method that works well for small and structured datasets. It is sensitive to the number of clusters, initial input, and noise data. The standard k-means clustering must be modified to be used in BDA. Some research focuses on the MapReduce/Spark implementation of traditional k-means clustering that improves the accuracy and reduces the time complexity [ 205 ]. Also, articles used the k-means clustering algorithm to predict floods [ 75 ], security monitoring [ 136 ], energy management and improve the prediction accuracy [ 56 , 131 ], the data access and resource utilization [ 144 ] in IoT. Association rules are an unsupervised learning approach used to discover interesting and hidden relationships and correlations between variables and objects in large databases and for data modeling in IoT [ 79 ]. Association rule mining uses various algorithms to identify frequent item sets, such as the apriori algorithm, FP growth algorithm, and maximal frequent itemset algorithm [ 79 , 106 ]. Neural networks (NN) perform big data processing and analysis efficiently. NN has self-learning ability and plays a significant role in BDA in IoT. NN is used for classification, big data mining, hidden pattern recognition, correlation recognition in big data raw, and decision-making in IoT applications. There are several different kinds of neural network algorithms, including LSTM [ 108 ], radial basis functions network [ 69 ], Deep NN [ 101 , 162 ], convolutional NN [ 163 ], etc.

Deep learning is a modern machine learning model that employs supervised or unsupervised methods to learn and extract multiple-level, high-level, and hierarchical features for big data classification tasks and pattern recognition [ 163 , 206 ]. Deep learning is a BDA tool that can speed up big data decision-making and feature extraction, improve the extracted information QoE level, resolve security issues, data dimensionality, and unlabeled and un-categorized big data processing in IoT applications [ 116 , 207 ]. In the reviewed articles, deep learning methods are used for human activity recognition [ 87 ], flood detection [ 130 ], smart cities [ 116 ], and feature learning on big data in the IoT [ 163 ]. Optimization refers to selecting the best solution from a set of alternatives by minimizing or maximizing a specified objective function [ 208 ]. Bio-inspired algorithms are stochastic search techniques used by many researchers to solve optimization problems in BDM processes in the IoT, including data ingestion, processing, analytics, and virtualization [ 209 ]. The features of these algorithms are good applicability, simplicity, robustness, flexibility, self-organization, and the possibility of dealing with real-world problems [ 210 ]. There are different types of categories for these algorithms in various articles. For instance, in [ 211 ], these algorithms are categorized into six categories: local search-based and global search-based; single-solution based and population-based; memory-based and memoryless; greedy and iterative; parallel; and nature-inspired and hybridized. In the reviewed articles, GA and NN are used more for BDM in the IoT (6 articles). GA has been used for feature extraction and selection, outlier detection, scheduling, optimizing energy consumption, reducing execution time and delay, and optimizing the predictive model in IoT applications [ 69 , 86 , 115 , 122 , 146 , 173 ].

4 Open issues and challenge

This section offers a variety of vital issues and challenges that require future work. IoT faces many challenges and open issues, including security, privacy, hardware, heterogeneity, data analysis, and virtualization challenges. IoT devices produce big data that must be monitored and managed using particular data patterns. For efficient decision-making, BDA in the IoT is applied to large datasets to reveal unseen patterns and correlations. So the key challenge in big data in the IoT is analyzing that data for knowledge discovery and virtualization. Various types of research have presented different categories for challenges and open issues for BDM in the IoT. Romero et al. [ 212 ] divided challenges into principal worries, security and monitoring, technological development, standardization, and privacy. Santana et al. [ 213 ] divided challenges into privacy, data management, heterogeneity, energy management, communication, scalability, security, lack of testbed, city models, and platform maintenance. Ahmed et al. [ 27 ] divided challenges into four categories: diversity, security, data provenance, data management, and data governance and regulation. This study divides challenges into BDM in the IoT and quality attributes challenges.

4.1 Big data management in the IoT challenges

In many reviewed articles, IoT big data management depends on centralized centers, including cloud-based servers, and has technical limitations. These architectures are platform-centric and have costly customized access mechanisms. A centralized architecture can have a single point of failure, which is very inefficient in terms of scalability and reliability. Also, in these architectures, unauthorized access to the server might easily result in the modification, leak, or manipulation of critical data [ 215 ]. In some research, authors used blockchain technology to overcome these problems [ 215 , 216 ]. But this technology has some challenges. For example, blockchain platforms can consume IoT devices' computational resources extensively. During the review in Sect.  3.1 , the process of BDM in the IoT includes data collection, communication, data ingestion, data storage, processing and analysis, and post-processing, each of which faces a variety of challenges and problems. This section examines the challenges involved in each of these steps.

4.1.1 Data collection

Big data in the IoT is generated from different, distributed, and multisource heterogeneous unsupervised domain [ 217 , 218 ]. Collecting this large amount of diverse data faces challenges such as energy consumption, limited battery life in sensors and other data collection devices, different hardware and operating systems, multiple and disparate resources, and combining them. It can be difficult to obtain complete, accurate, and maintain quality data. IoT and WSN encompass a large number of distributed mobile nodes. Mobile nodes [ 219 ] must increase the amount of data collected while minimizing the power consumption of both the mobile node and IoT devices. Therefore, the main challenge is mobile data collection management, determining and planning mobile sink trajectories for collecting data from nodes. Most existing mobile data collection approaches are static and only find a solution for a scenario with fixed parameters [ 220 ]. These solutions do not consider the change in the amount of data generated by the IoT nodes or devices when an IoT device can move from one situation to another. For future work, we propose using AI techniques, including ML or deep learning, for intelligent management of mobile data collection.

4.1.2 Communication

Transferring data from different sources to the data processing and analysis stage is one of the steps in BDM in the IoT. Communication protocols and technologies must share data at high speeds and on time. The connectivity challenges include interoperability, bandwidth, reducing traffic, energy consumption, security, network, transport protocols, delivery of services, network congestion, and communication cost. Another connectivity challenge is nodes accessing other nodes' information under different network topologies with different channel fading [ 221 ]. Concerning advances in mobile information infrastructure, integration of the 6G technologies, mobile satellite communications, and AI can increase frequency band, network speed, and network coverage and improve the number of connections [ 222 ]. Different approaches are proposed for data transmission optimizing and overcoming these limitations, such as parsimonious/compressive sensing [ 223 , 224 ]. Compressive sensing technology is a theory of acquiring and compressing signals that use the sparsity behavior of natural signals at the sensing stage to minimize power consummation and data dimensionality reduction [ 225 ]. In compressive sensing technology, the collected data from different sensors are first compressed and then transmitted. Therefore, the complexity is transferred to the receiver side from the sensors, which are usually resource-constrained and self-powered [ 226 ]. For future work, we propose combining compressive sensing with AI technologies to present a lightweight, real-time, and dynamic compressive sensing method for overcoming the communication challenges in BDM in the IoT.

4.1.3 Data ingestion

Big data in the IoT have various features such as: enormous, high-speed, heterogeneity of data formats, complexity, different data resolutions, abnormal and incorrect, ambiguity, unbalanced, massive redundancy, multidimensional, granularity, continuously, inconsistencies, probabilistic, sparse, sequential, dynamical, timeliness, non-randomly distributed, and misplaced [ 56 , 63 , 89 , 117 , 119 , 125 , 135 , 137 , 173 , 227 ]. Each data ingestion step discussed in Sect.  3.1.3 has challenges. These issues are anomaly detection, missing data, outlier detection, feature selection/extraction, dimensionality reduction, redundancy, standardization, rule discovery, computational cost, and normalization that different mechanisms use for these challenges. Missing data could lead to the loss of a large amount of valuable and reliable information and bad decision-making. Many articles utilize the delete, ignore, mean/median value, or constant global methods for handling missing data. These dangerous methods may yield biased and untrustworthy results [ 228 ]. Therefore, adding new techniques by considering more efficiency, high accuracy, minimal computational complexity, and less time consumption is interesting in the future. For this purpose, we can use ML and nature-inspired optimization algorithms or a combination thereof. The parallel technology has made data ingestion and processing more efficient in recent years, and it saves space and time by eliminating the need to decompress data [ 229 ]. Also, BDA types in the IoT are used in this stage, which is discussed in Sect.  3.4 . Each of these methods has challenges. For example, clustering has challenges such as real-time clustering, local optima, determining the number of clusters, updating the clustering centers, and determining the initial clustering centers. ANN faces many issues, including how to determine the number of layers, the training, and test samples, the number of nodes, choosing an operable objective function, and how to improve the training speed of the network in a big data environment. Various articles solve these problems using meta-heuristic algorithms. However, these algorithms cannot handle big IoT data sets within the specified time due to high computation costs, limited memory, and processing units, and premature convergence [ 145 , 230 ]. For future work, we propose using new optimization meta-heuristic algorithms and AI methods based on these techniques by utilizing the strengths of MapReduce and Apache Spark.

4.1.4 Data storage

Data storage is another major challenge in BDM in the IoT. The big data storage mechanisms in the IoT were discussed in Sect.  3.1.4 . The challenges in this regard can be categorized as IoT-based big data storage systems in cloud computing and complex environments such as industry 4.0 applications and data storage architecture. The main data storage challenges are IoT data replication and consistency management. Many researchers have proposed strategies for determining the best location for copy storage in geo-distributed storage systems based on cloud and fog computing. But many of them, due to the geographical distance between distributed storage systems, cannot handle the problems of high data access latencies and replica synchronization costs [ 231 ]. Also, data consistency management strategies must manage the massive amounts of data with different data consistency requirements and system heterogeneity.

4.1.5 Processing and analysis

The big data processing and analysis in BDM in the IoT has different challenges, including task scheduling, real-time data analysis, developing the IoT data analysis infrastructure, data management in the cloud-IoT environments, and query optimization. The authors used data mining and AI algorithms to overcome these challenges. The challenges of using AI technologies for data analytics in the IoT are to balance the computational costs (or response time) and improve the accuracy of the prediction and analysis results [ 232 ]. Also, many multi-objective optimization problems have more than three objective functions, which present challenges, including the diversity and convergence speed of the algorithm [ 152 ]. However, determining an algorithm to process a dynamic IoT dataset based on some application-specific goals for better accuracy remains a challenge. Also, most current methods cannot meet user demands for the fundamental features of cloud-IoT environments, including heterogeneity, dynamism, reliability, flexibility, responsiveness, and elasticity. For future work, we propose studies of various optimization algorithms, including metaheuristic algorithms (many-objective) and ML algorithms, and combined versions of these algorithms for big data processing and analysis in the IoT. Regarding the limitations of wireless nodes (low power and computational) and cloud servers (high latency, privacy, performance bottleneck, context unawareness, etc.) for processing and analysis computing tasks, using mobile edge or fog computing to overcome these problems is helpful.

4.1.6 Post-processing

Providing insight from processed and analyzed data in the IoT requires selecting appropriate visualization techniques. Most of the reviewed methods use simulator tools such as CloudSim [ 143 , 173 ], TRNSYS [ 131 ], Cooja [ 82 ], and Extend-Sim [ 8 ] for evaluation. Additional studies are needed to evaluate the mentioned approaches in real-world systems and datasets.

4.2 QoS management

QoS is one of the critical factors in BDM in the IoT and needs research, management, and optimization (discussed in Sect.  3.3 ). The reviewed articles used these parameters and metrics for evaluation. No article considers these parameters thoroughly for its proposed architecture. Therefore, it is exciting to compare various architectures by considering the different QoS parameters and quality attributes in the future. Security, privacy, and trust are critical issues in IoT BDA that most reviewed articles did not address, and the proposed architectures or frameworks did not involve the data perception layer. The security frame generally consists of confidentiality, integrity, authentication, non-repudiation, availability, and privacy [ 233 ]. We concede that no comprehensive and highly secure scheme or platform for all types of data collection, analysis, and sharing meets all security requirements. The other main challenges are integrating privacy protection methods with data sharing platforms and selecting the best privacy protection algorithms to use during data processing [ 172 ]. Therefore, it is suggested for the future to utilize cryptographic mechanisms in different layers of architectures or frameworks, add a data perception layer, and develop security protocols specifically for IoT devices because of their heterogeneity and resource limitations.

The blockchain framework is widely used in IoT to improve protection, trust, reputation, management, control, and security. The blockchain framework provides decentralized security, authentication rules, and privacy for IoT devices. However, there are major challenges, such as high energy consumption, delay, and computational overhead, because of the resource constraints in IoT devices. Many types of research have been suggested as solutions to these problems. For instance, Corradini et al. [ 234 ] proposed a two-tier Blockchain framework for increasing the security and autonomy of smart objects in the IoT by implementing a trust-based protection mechanism. The tiers of this framework are a point-to-point local tier and a community-oriented global tier. Pincheira et al. [ 235 ] proposed a cost-effective blockchain-based architecture for ensuring data integrity, auditability, and traceability and increasing trust and trustworthiness in IoT devices. This architecture has four components: the cloud module, mobile app, connected tool, and blockchain module. Tchagna Kouanou et al. [ 236 ] proposed a 4-layer blockchain-based architecture to secure data in the IoT to increase security, integrity, scalability, flexibility, and throughput. The layers of this architecture are tokens, smart contracts, blockchain, and peers. In future research, we suggest using AI techniques and a lightweight blockchain framework to increase protection, trust, reputation, and security in the IoT.

Trust and reputation management are vital issues in the SIoT and MIoT scenarios. In [ 237 ], the authors defined trust and reputation in the MIoT as the trust of an instance in another one of the same IoT; the trust of an object in another one of the MIoT; the reputation of an instance in an IoT; the reputation of an object in a MIoT; the reputation of an IoT in a MIoT; the trust of an IoT in another IoT; and the trust of an object in an IoT. Security in the SIoT aims to differentiate between secure and malicious things and increase the safety and protection of SIoT networks [ 185 ]. Investigating trust and reputation in SIoT and MIoT has many benefits, such as identifying, isolating, managing malicious objects, supporting collaboration, and identifying and evaluating the objects’ QoS parameters. Also, the lack of trust and reputation management in SIoT and MIoT causes problems such as loss of accessibility, privacy, and security [ 237 ]. To overcome these issues, we suggest utilizing trust and reputation with AI methods to develop detection techniques for anomalous and malicious behaviors of things in the MIoT and SIoT in future works.

5 Conclusion

This paper presented a systematic review of the BDM mechanisms in the IoT. First, we discussed the advantages and disadvantages of some systematic and review articles about BDM in the IoT and then explained the purpose of this paper. Then, the research methodology and details of 110 selected articles were presented. These articles were divided into four main categories, including BDM processes, big BDM architectures/frameworks, quality attributes, and data analytics types in IoT. Some of these categories have been divided into some subcategories: BDM process in IoT was divided into data collection, communication, data ingestion, data storage, processing and analysis, and post-processing; big data architectures/frameworks in the IoT were divided into BDM architectures/frameworks in the IoT-based applications and BDM architectures/frameworks in the IoT paradigms; big data analytics-types were divided into the descriptive, diagnostic, predictive, and prescriptive analysis; and big data storage systems in the IoT were divided into relational databases, NoSQL databases, DFS, and cloud/edge/fog/mist storage. Also, the advantages and disadvantages of each of the BDM mechanisms in the IoT were discussed. The tools and platforms used for BDM in the IoT in the articles were reviewed and compared based on criteria. The most common type of analysis that articles use is predictive analysis, with 57.27%, which uses ML algorithms. The classification, optimization, and clustering algorithms are the most widely used for big data analysis in the IoT. Some articles present architectures mostly in IoT-based healthcare, with 33.33%, and IoT-based smart cities, with 22.22%. These architectures have two to eight layers, each performing a set of functions. In the review of qualitative characteristics, we observed that most articles evaluated their evaluations based on criteria, including performance, efficiency, accuracy, and scalability. Meanwhile, some features are less used, including confidentiality, sustainability, accessibility, portability, generality, and maintainability. The NoSQL database and DFS are used more to store data than other databases. The BDM process in the IoT uses different algorithms and tools with various features. Various programming languages and operating systems are used to evaluate and implement the proposed mechanisms. The Java and python programming languages and the UBUNTU operating system are used more.

This paper tries to review the BDM mechanisms in the IoT. Specifically, it considers studies published in high-quality international journals. The most recent works on BDM mechanisms in the IoT have been compared and analyzed in this paper. We hope that this study will be helpful for the next generation of studies for developing BDM mechanisms in real-complex environments.

https://www.idc.com/ .

http://www.gartner.com/it-glossary/predictive-analytics/ .

Cao, B., Zhang, Y., Zhao, J., Liu, X., Skonieczny, Ł, & Lv, Z. (2021). Recommendation based on large-scale many-objective optimization for the intelligent internet of things system. IEEE Internet of Things Journal . https://doi.org/10.1109/JIOT.2021.3104661

Article   Google Scholar  

Hou, R., Kong, Y., Cai, B., & Liu, H. (2020). Unstructured big data analysis algorithm and simulation of internet of things based on machine learning. Neural Computing and Applications, 32 , 5399–5407.

Kumar, M., Kumar, S., & Kashyap, P. K. (2021). Towards data mining in IoT cloud computing networks: Collaborative filtering based recommended system. Journal of Discrete Mathematical Sciences and Cryptography, 24 , 1309–1326.

Article   MathSciNet   MATH   Google Scholar  

Cao, B., Zhao, J., Lv, Z., & Yang, P. (2020). Diversified personalized recommendation optimization based on mobile data. IEEE Transactions on Intelligent Transportation Systems, 22 , 2133–2139.

Sanislav, T., Mois, G. D., Zeadally, S., & Folea, S. C. (2021). Energy harvesting techniques for internet of things (IoT). IEEE Access, 9 , 39530–39549.

Zhou, H., Sun, G., Fu, S., Liu, J., Zhou, X., & Zhou, J. (2019). A Big data mining approach of PSO-based BP Neural network for financial risk management with IoT. IEEE Access, 7 , 154035–154043.

Tang, B., Chen, Z., Hefferman, G., Pei, S., Wei, T., He, H., et al. (2017). Incorporating intelligence in fog computing for big data analysis in smart cities. IEEE Transactions on Industrial informatics, 13 , 2140–2150.

Jiang, W. (2019). An intelligent supply chain information collaboration model based on internet of things and big data. IEEE Access, 7 , 58324–58335.

Xiao, S., Yu, H., Wu, Y., Peng, Z., & Zhang, Y. (2017). Self-evolving trading strategy integrating internet of things and big data. IEEE Internet of Things Journal, 5 , 2518–2525.

Sowe, S. K., Kimata, T., Dong, M., & Zettsu K. (2014). Managing heterogeneous sensor data on a big data platform: IoT services for data-intensive science. In 2014 IEEE 38th International Computer Software and Applications Conference Workshops , Vasteras, Sweden, pp. 295-300

Nie, X., Fan, T., Wang, B., Li, Z., Shankar, A., & Manickam, A. (2020). Big data analytics and IoT in operation safety management in under water management. Computer Communications, 154 , 188–196.

Liu, H., & Liu, X. (2019). A novel research on the influence of enterprise culture on internal control in big data and internet of things. Mobile Networks and Applications, 24 , 365–374.

Piccialli, F., Benedusi, P., Carratore, L., & Colecchia, G. (2020). An IoT data analytics approach for cultural heritage. Personal and Ubiquitous Computing . https://doi.org/10.1007/s00779-019-01323-z

Liu, C., Feng, Y., Lin, D., Wu, L., & Guo, M. (2020). Iot based laundry services: an application of big data analytics, intelligent logistics management, and machine learning techniques. International Journal of Production Research . https://doi.org/10.1080/00207543.2019.1677961

Wang, J., Wu, Y., Yen, N., Guo, S., & Cheng, Z. (2016). Big data analytics for emergency communication networks: A survey. IEEE Communications Surveys & Tutorials, 18 , 1758–1778.

Jahanbakht, M., Xiang, W., Hanzo, L., & Azghadi, M. R. (2020) Internet of underwater things and big marine data analytics--a comprehensive survey. arXiv preprint arXiv:2012.06712 .

Stoyanova, M., Nikoloudakis, Y., Panagiotakis, S., Pallis, E., & Markakis, E. K. (2020). A survey on the internet of things (IoT) forensics: Challenges, approaches, and open issues. IEEE Communications Surveys & Tutorials, 22 , 1191–1221.

Aldalahmeh, S. A., & Ciuonzo, D. (2022). Distributed detection fusion in clustered sensor networks over multiple access fading channels. IEEE Transactions on Signal and Information Processing over Networks, 8 , 317–329.

Article   MathSciNet   Google Scholar  

Rajavel, R., Ravichandran, S. K., Harimoorthy, K., Nagappan, P., & Gobichettipalayam, K. R. (2022). IoT-based smart healthcare video surveillance system using edge computing. Journal of Ambient Intelligence and Humanized Computing, 13 , 3195–3207.

Shahid, H., Shah, M. A., Almogren, A., Khattak, H. A., Din, I. U., Kumar, N., et al. (2021). Machine learning-based mist computing enabled internet of battlefield things. ACM Transactions on Internet Technology (TOIT), 21 , 1–26.

Thomas, D., Orgun, M., Hitchens, M., Shankaran, R., Mukhopadhyay, S. C., & Ni, W. (2020). A graph-based fault-tolerant approach to modeling QoS for IoT-based surveillance applications. IEEE Internet of Things Journal, 8 , 3587–3604.

S. Vahdat (2020) The role of IT-based technologies on the management of human resources in the COVID-19 era. Kybernetes .

Hassan, M., Awan, F. M., Naz, A., deAndrés-Galiana, E. J., Alvarez, O., Cernea, A., et al. (2022). Innovations in genomics and big data analytics for personalized medicine and health care: A review. International Journal of Molecular Sciences, 23 , 4645.

Honar Pajooh, H., Rashid, M. A., Alam, F., & Demidenko, S. (2021). IoT big data provenance scheme using blockchain on Hadoop ecosystem. Journal of Big Data, 8 , 1–26.

Priyadarshini, S. B. B., Bhusan Bagjadab, A., & Mishra B. K. (2019). The role of IoT and big data in modern technological arena: A comprehensive study. In Internet of Things and Big Data Analytics for Smart Generation. Springer, pp. 13–25.

Zheng, W., Yin, L., Chen, X., Ma, Z., Liu, S., & Yang, B. (2021). Knowledge base graph embedding module design for Visual question answering model. Pattern Recognition, 120 , 108153.

Ahmed, E., Yaqoob, I., Hashem, I. A. T., Khan, I., Ahmed, A. I. A., Imran, M., et al. (2017). The role of big data analytics in internet of things. Computer Networks, 129 , 459–471.

Singh, S., & Yassine, A. (2018). IoT big data analytics with fog computing for household energy management in smart grids. In International Conference on Smart Grid and Internet of Things . pp. 13–22.

Marjani, M., Nasaruddin, F., Gani, A., Karim, A., Hashem, I. A. T., Siddiqa A., et al. (2017). Big IoT data analytics: architecture, opportunities, and open research challenges. ieee access , 5, 5247–5261.

Li, C. (2020). Information processing in internet of things using big data analytics. Computer Communications, 160 , 718–729.

Kwon, O., Lee, N., & Shin, B. (2014). Data quality management, data usage experience and acquisition intention of big data analytics. International journal of information management, 34 , 387–394.

Gandomi, A., & Haider, M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management, 35 , 137–144.

Ahmed, M., Choudhury, S., & Al-Turjman, F. (2019). Big data analytics for intelligent internet of things. In Artificial Intelligence in IoT . Springer, pp. 107–127.

Urrehman, M. H., Ahmed, E., Yaqoob, I., Hashem, I. A. T., Imran, M., & Ahmad, S. (2018). Big data analytics in industrial IoT using a concentric computing model. IEEE Communications Magazine, 56 , 37–43.

Constante Nicolalde, F., Silva, F., Herrera, B., & Pereira, A. (2018). Big data analytics in IOT: challenges, open research issues and tools. In World conference on information systems and technologies , pp. 775–788.

Talebkhah, M., Sali, A., Marjani, M., Gordan, M., Hashim, S. J., & Rokhani, F. Z. (2021). IoT and big data applications in smart cities: Recent advances, challenges, and critical issues. IEEE Access, 9 , 55465–55484.

Bansal, M., Chana, I., & Clarke, S. (2020). A survey on iot big data: Current status, 13 v’s challenges, and future directions. ACM Computing Surveys (CSUR), 53 , 1–59.

Simmhan, Y., & Perera, S. (2016). Big data analytics platforms for real-time applications in IoT. In Big data analytics . Springer, pp. 115–135.

Shoumy, N. J., Ang, L.-M., Seng, K. P., Rahaman, D. M., & Zia, T. (2020). Multimodal big data affective analytics: A comprehensive survey using text, audio, visual and physiological signals. Journal of Network and Computer Applications, 149 , 102447.

Ge, M., Bangui, H., & Buhnova, B. (2018). Big data for internet of things: A survey. Future Generation Computer Systems, 87 , 601–614.

Siow, E., Tiropanis, T., & Hall, W. (2018). Analytics for the internet of things: A survey. ACM Computing Surveys (CSUR), 51 , 1–36.

Fawzy, D., Moussa, S. M., & Badr, N. L. (2022). The internet of things and architectures of big data analytics: Challenges of intersection at different domains. IEEE Access, 10 , 4969–4992.

Zhong, Y., Chen, L., Dan, C., & Rezaeipanah, A. (2022). A systematic survey of data mining and big data analysis in internet of things. The Journal of Supercomputing . https://doi.org/10.1007/s11227-022-04594-1

Hajjaji, Y., Boulila, W., Farah, I. R., Romdhani, I., & Hussain, A. (2021). Big data and IoT-based applications in smart environments: A systematic review. Computer Science Review, 39 , 100318.

Ahmadova, U., Mustafayev, M., Kiani Kalejahi, B., Saeedvand, S., & Rahmani, A. M. (2021). Big data applications on the internet of things: A systematic literature review. International Journal of Communication Systems, 34 , e5004.

Doewes, R. I., Gharibian, G., Zadeh, F. A., Zaman, B. A., Vahdat, S., & Akhavan-Sigari, R. (2022). An updated systematic review on the effects of aerobic exercise on human blood lipid profile. Current Problems in Cardiology . https://doi.org/10.1016/j.cpcardiol.2022.101108

Zadeh, F. A., Bokov, D. O., Yasin, G., Vahdat, S., & Abbasalizad-Farhangi, M. (2021). Central obesity accelerates leukocyte telomere length (LTL) shortening in apparently healthy adults: A systematic review and meta-analysis. Critical Reviews in Food Science and Nutrition . https://doi.org/10.1080/10408398.2021.1971155

Esmailiyan, M., Amerizadeh, A., Vahdat, S., Ghodsi, M., Doewes, R. I., & Sundram, Y. (2021). Effect of different types of aerobic exercise on individuals with and without hypertension: An updated systematic review. Current Problems in Cardiology . https://doi.org/10.1016/j.cpcardiol.2021.101034

Vahdat, S., & Shahidi, S. (2020). D-dimer levels in chronic kidney illness: a comprehensive and systematic literature review. Proceedings of the National Academy of Sciences, India Section b: Biological Sciences . https://doi.org/10.1007/s40011-020-01172-4

Zhou, D., Yan, Z., Fu, Y., & Yao, Z. (2018). A survey on network data collection. Journal of Network and Computer Applications, 116 , 9–23.

Rathore, M. M., Ahmad, A., Paul, A., & Rho, S. (2016). Urban planning and building smart cities based on the internet of things using big data analytics. Computer Networks, 101 , 63–80.

Ahmad, A., Khan, M., Paul, A., Din, S., Rathore, M. M., Jeon, G., et al. (2018). Toward modeling and optimization of features selection in big data based social Internet of Things. Future Generation Computer Systems, 82 , 715–726.

Shah, S. A., Seker, D. Z., Rathore, M. M., Hameed, S., Yahia, S. B., & Draheim, D. (2019). Towards disaster resilient smart cities: Can internet of things and big data analytics be the game changers? IEEE Access, 7 , 91885–91903.

Celesti, A., & Fazio, M. (2019). A framework for real time end to end monitoring and big data oriented management of smart environments. Journal of Parallel and Distributed Computing, 132 , 262–273.

Silva, B. N., Khan, M., & Han, K. (2017). Integration of big data analytics embedded smart city architecture with RESTful web of things for efficient service provision and energy management. Future generation computer systems . https://doi.org/10.1016/j.future.2017.06.024

Yassine, A., Singh, S., Hossain, M. S., & Muhammad, G. (2019). IoT big data analytics for smart homes with fog and cloud computing. Future Generation Computer Systems, 91 , 563–573.

Khan, M., Han, K., & Karthik, S. (2018). Designing smart control systems based on internet of things and big data analytics. Wireless Personal Communications, 99 , 1683–1697.

Rathore, M. M., Paul, A., Ahmad, A., Anisetti, M., & Jeon, G. (2017). Hadoop-based intelligent care system (HICS) analytical approach for big data in IoT. ACM Transactions on Internet Technology (TOIT), 18 , 1–24.

Yacchirema, D. C., Sarabia-Jácome, D., Palau, C. E., & Esteve, M. (2018). A smart system for sleep monitoring by integrating IoT with big data analytics. IEEE Access, 6 , 35988–36001.

Ma, Y., Wang, Y., Yang, J., Miao, Y., & Li, W. (2016). Big health application system based on health internet of things and big data. IEEE Access, 5 , 7885–7897.

Rathore, M. M., Ahmad, A., Paul, A., Wan, J., & Zhang, D. (2016). Real-time medical emergency response system: Exploiting IoT and big data for public health. Journal of medical systems, 40 , 283.

Zhou, Q., Zhang, Z., & Wang, Y. (2019). WIT120 data mining technology based on internet of things. Health Care Management Science . https://doi.org/10.1007/s10729-019-09497-x

Silva, B. N., Khan, M., Jung, C., Seo, J., Muhammad, D., Han, J., et al. (2018). Urban planning and smart city decision management empowered by real-time data processing using big data analytics. Sensors, 18 , 2994.

Lakshmanaprabu, S., Shankar, K., Khanna, A., Gupta, D., Rodrigues, J. J., Pinheiro, P. R., et al. (2018). Effective features to classify big data using social internet of things. IEEE access, 6 , 24196–24204.

Al-Qurabat, A. K. M., Mohammed, Z. A., & Hussein, Z. J. (2021). Data traffic management based on compression and MDL techniques for smart agriculture in IoT. Wireless Personal Communications, 120 , 2227–2258.

Ahmad, A., Babar, M., Din, S., Khalid, S., Ullah, M. M., Paul, A., et al. (2019). Socio-cyber network: The potential of cyber-physical system to define human behaviors using big data analytics. Future generation computer systems, 92 , 868–878.

Floris, A., Porcu, S., Atzori, L., & Girau, R. (2022). A Social IoT-based platform for the deployment of a smart parking solution. Computer Networks, 205 , 108756.

Al-Ali, A.-R., Zualkernan, I. A., Rashid, M., Gupta, R., & AliKarar, M. (2017). A smart home energy management system using IoT and big data analytics approach. IEEE Transactions on Consumer Electronics, 63 , 426–434.

Moreno, M. V., Terroso-Sáenz, F., González-Vidal, A., Valdés-Vela, M., Skarmeta, A. F., Zamora, M. A., et al. (2016). Applicability of big data techniques to smart cities deployments. IEEE Transactions on Industrial Informatics, 13 , 800–809.

Nasiri, H., Nasehi, S., & Goudarzi, M. (2019). Evaluation of distributed stream processing frameworks for IoT applications in smart cities. Journal of Big Data, 6 , 52.

Ahanger, T. A., Tariq, U., Nusir, M., Aldaej, A., Ullah, I., & Sulman, A. (2022). A novel IoT–fog–cloud-based healthcare system for monitoring and predicting COVID-19 outspread. The Journal of Supercomputing, 78 , 1783–1806.

Rani, S., & Chauhdary, S. H. (2018). A novel framework and enhanced QoS big data protocol for smart city applications. Sensors, 18 , 3980.

Lu, Z., Wang, N., Wu, J., & Qiu, M. (2018). IoTDeM: An IoT big data-oriented MapReduce performance prediction extended model in multiple edge clouds. Journal of Parallel and Distributed Computing, 118 , 316–327.

Rathore, M. M., Paul, A., Hong, W.-H., Seo, H., Awan, I., & Saeed, S. (2018). Exploiting IoT and big data analytics: Defining smart digital city using real-time urban data. Sustainable cities and society, 40 , 600–610.

Sood, S. K., Sandhu, R., Singla, K., & Chang, V. (2018). IoT, big data and HPC based smart flood management framework. Sustainable Computing: Informatics and Systems, 20 , 102–117.

Google Scholar  

Machorro-Cano, I., Alor-Hernández, G., Paredes-Valverde, M. A., Rodríguez-Mazahua, L., Sánchez-Cervantes, J. L., & Olmedo-Aguirre, J. O. (2020). HEMS-IoT: A big data and machine learning-based smart home system for energy saving. Energies, 13 , 1097.

Raptis, T. P., Passarella, A., & Conti, M. (2018). Performance analysis of latency-aware data management in industrial IoT networks. Sensors, 18 , 2611.

Seng, K. P., & Ang, L.-M. (2018). A big data layered architecture and functional units for the multimedia Internet of Things. IEEE Transactions on Multi-Scale Computing Systems, 4 , 500–512.

Muangprathub, J., Boonnam, N., Kajornkasirat, S., Lekbangpong, N., Wanichsombat, A., & Nillaor, P. (2019). IoT and agriculture data analysis for smart farm. Computers and electronics in agriculture, 156 , 467–474.

Chilipirea, C., Petre, A.-C., Groza, L.-M., Dobre, C., & Pop, F. (2017). An integrated architecture for future studies in data processing for smart cities. Microprocessors and Microsystems, 52 , 335–342.

Enayet, A., Razzaque, M. A., Hassan, M. M., Alamri, A., & Fortino, G. (2018). A mobility-aware optimal resource allocation architecture for big data task execution on mobile cloud in smart cities. IEEE Communications Magazine, 56 , 110–117.

Plageras, A. P., Psannis, K. E., Stergiou, C., Wang, H., & Gupta, B. B. (2018). Efficient IoT-based sensor BIG data collection–processing and analysis in smart buildings. Future Generation Computer Systems, 82 , 349–357.

Syafrudin, M., Alfian, G., Fitriyani, N. L., & Rhee, J. (2018). Performance analysis of IoT-based sensor, big data processing, and machine learning model for real-time monitoring system in automotive manufacturing. Sensors, 18 , 2946.

El-Hasnony, I. M., Mostafa, R. R., Elhoseny, M., & Barakat, S. I. (2021). Leveraging mist and fog for big data analytics in IoT environment. Transactions on Emerging Telecommunications Technologies . https://doi.org/10.1002/ett.4057

Jindal, A., Kumar, N., & Singh, M. (2020). A unified framework for big data acquisition, storage, and analytics for demand response management in smart cities. Future Generation Computer Systems, 108 , 921–934.

Hussain, M. M., Beg, M. S., & Alam, M. S. (2020). Fog computing for big data analytics in IoT aided smart grid networks. Wireless Personal Communications . https://doi.org/10.1007/s11277-020-07538-1

Zhou, Z., Yu, H., & Shi, H. (2020). Human activity recognition based on improved Bayesian convolution network to analyze health care data using wearable IoT device. IEEE Access, 8 , 86411–86418.

Sengupta, S., & Bhunia, S. S. (2020). Secure data management in cloudlet assisted IoT enabled e-health framework in smart city. IEEE Sensors Journal, 20 , 9581–9588.

Babar, M., & Arif, F. (2019). Real-time data processing scheme using big data analytics in internet of things based smart transportation environment. Journal of Ambient Intelligence and Humanized Computing, 10 , 4167–4177.

Hong-Tan, L., Cui-hua, K., Muthu, B., & Sivaparthipan, C. (2021). Big data and ambient intelligence in IoT-based wireless student health monitoring system. Aggression and Violent Behavior . https://doi.org/10.1016/j.avb.2021.101601

Paul, A., Ahmad, A., Rathore, M. M., & Jabbar, S. (2016). Smartbuddy: Defining human behaviors using big data analytics in social internet of things. IEEE Wireless communications, 23 , 68–74.

Gohar, M., Ahmed, S. H., Khan, M., Guizani, N., Ahmed, A., & Rahman, A. U. (2018). A big data analytics architecture for the internet of small things. IEEE Communications Magazine, 56 , 128–133.

Armoogum, S., & Li, X. (2019). Big data analytics and deep learning in bioinformatics with hadoop. In Deep Learning and Parallel Computing Environment for Bioengineering Systems . Elsevier, pp. 17–36.

Demchenko, Y., Turkmen, F., de Laat, C., Hsu, C. H., Blanchet, C., & Loomis, C. (2017). Cloud computing infrastructure for data intensive applications. In Big Data Analytics for Sensor-Network Collected Intelligence . Elsevier, pp. 21–62.

Wu, X., Zheng, W., Xia, X., & Lo, D. (2021). Data quality matters: A case study on data label correctness for security bug report prediction. IEEE Transactions on Software Engineering . https://doi.org/10.1109/TSE.2021.3063727

Erraissi, A., & Belangour, A. (2018). Data sources and ingestion big data layers: Meta-modeling of key concepts and features. International Journal of Engineering & Technology, 7 , 3607–3612.

Ji, C., Shao, Q., Sun, J., Liu, S., Pan, L., Wu, L., et al. (2016). Device data ingestion for industrial big data platforms with a case study. Sensors, 16 , 279.

Isah, H., & Zulkernine F (2018) A scalable and robust framework for data stream ingestion. In 2018 IEEE International Conference on Big Data (Big Data) . pp. 2900-2905

Dai, H.-N., Wong, R.C.-W., Wang, H., Zheng, Z., & Vasilakos, A. V. (2019). Big data analytics for large-scale wireless networks: Challenges and opportunities. ACM Computing Surveys (CSUR), 52 , 1–36.

Chawla, H., & Khattar, P., (2020). Data ingestion. In Data Lake Analytics on Microsoft Azure . Springer, pp. 43–85.

Sankaranarayanan, S., Rodrigues, J. J., Sugumaran, V., & Kozlov, S. (2020). Data flow and distributed deep neural network based low latency IoT-edge computation model for big data environment. Engineering Applications of Artificial Intelligence, 94 , 103785.

Davoudian, A., Chen, L., & Liu, M. (2018). A survey on NoSQL stores. ACM Computing Surveys (CSUR), 51 , 1–43.

Cao, B., Sun, Z., Zhang, J., & Gu, Y. (2021). Resource allocation in 5G IoV architecture based on SDN and fog-cloud computing. IEEE Transactions on Intelligent Transportation Systems, 22 , 3832–3840.

Sonbol, K., Özkasap, Ö., Al-Oqily, I., & Aloqaily, M. (2020). EdgeKV: Decentralized, scalable, and consistent storage for the edge. Journal of Parallel and Distributed Computing, 144 , 28–40.

Akanbi, A., & Masinde, M. (2020). A distributed stream processing middleware framework for real-time analysis of heterogeneous data on big data platform: case of environmental monitoring. Sensors, 20 , 3166.

Harb, H., Mroue, H., Mansour, A., Nasser, A., & Motta Cruz, E. (2020). A hadoop-based platform for patient classification and disease diagnosis in healthcare applications. Sensors, 20 , 1931.

Osman, A. M. S. (2019). A novel big data analytics framework for smart cities. Future Generation Computer Systems, 91 , 620–633.

Alves, J. M., Honório, L. M., & Capretz, M. A. (2019). ML4IoT: A framework to orchestrate machine learning workflows on internet of things data. IEEE Access, 7 , 152953–152967.

Oğur, N. B., Al-Hubaishi, M., & Çeken, C. (2022). IoT data analytics architecture for smart healthcare using RFID and WSN. ETRI Journal, 44 , 135–146.

Bashir, M. R., Gill, A. Q., Beydoun, G., & Mccusker, B. (2020). Big data management and analytics metamodel for IoT-enabled smart buildings. IEEE Access, 8 , 169740–169758.

Chhabra, G. S., Singh, V. P., & Singh, M. (2018). Cyber forensics framework for big data analytics in IoT environment using machine learning. Multimedia Tools and Applications . https://doi.org/10.1007/s11042-018-6338-1

Vögler, M., Schleicher, J. M., Inzinger, C., & Dustdar, S. (2017). Ahab: A cloud-based distributed big data analytics framework for the internet of things. Software: Practice and Experience, 47 , 443–454.

Farmanbar, M., & Rong, C. (2020). Triangulum city dashboard: An interactive data analytic platform for visualizing smart city performance. Processes, 8 , 250.

Ghallab, H., Fahmy, H., & Nasr, M. (2020). Detection outliers on internet of things using big data technology. Egyptian Informatics Journal, 21 , 131–138.

Lan, K., Fong, S., Song, W., Vasilakos, A. V., & Millham, R. C. (2017). Self-adaptive pre-processing methodology for big data stream mining in internet of things environmental sensor monitoring. Symmetry, 9 , 244.

He, X., Wang, K., Huang, H., & Liu, B. (2018). QoE-driven big data architecture for smart city. IEEE Communications Magazine, 56 , 88–93.

Singh, A., Garg, S., Batra, S., Kumar, N., & Rodrigues, J. J. (2018). Bloom filter based optimization scheme for massive data handling in IoT environment. Future Generation Computer Systems, 82 , 440–449.

Yu, W., Liu, Y., Dillon, T., Rahayu, W., & Mostafa, F. (2021). An integrated framework for health state monitoring in a smart factory employing IoT and big data techniques. IEEE Internet of Things Journal, 9 , 2443–2454.

Zhang, Q., Zhu, C., Yang, L. T., Chen, Z., Zhao, L., & Li, P. (2017). An incremental CFS algorithm for clustering large data in industrial Internet of Things. IEEE Transactions on Industrial Informatics, 13 , 1193–1201.

Shaji, B., Lal Raja Singh, R., & Nisha, K. (2022). A novel deep neural network based marine predator model for effective classification of big data from social internet of things. Concurrency and Computation: Practice and Experience . https://doi.org/10.1002/cpe.7244

Al-Osta, M., Bali, A., & Gherbi, A. (2019). Event driven and semantic based approach for data processing on IoT gateway devices. Journal of Ambient Intelligence and Humanized Computing, 10 , 4663–4678.

Deng, X., Jiang, P., Peng, X., & Mi, C. (2018). An intelligent outlier detection method with one class support tucker machine and genetic algorithm toward big sensor data in Internet of Things. IEEE Transactions on Industrial Electronics, 66 , 4672–4683.

Yao, X., Wang, J., Shen, M., Kong, H., & Ning, H. (2019). An improved clustering algorithm and its application in IoT data analysis. Computer Networks, 159 , 63–72.

Mansour, R. F., Abdel-Khalek, S., Hilali-Jaghdam, I., Nebhen, J., Cho, W., & Joshi, G. P. (2021). An intelligent outlier detection with machine learning empowered big data analytics for mobile edge computing. Cluster Computing . https://doi.org/10.1007/s10586-021-03472-4

Karyotis, V., Tsitseklis, K., Sotiropoulos, K., & Papavassiliou, S. (2018). Big data clustering via community detection and hyperbolic network embedding in IoT applications. Sensors, 18 , 1205.

Chui, K. T., Liu, R. W., Lytras, M. D., & Zhao, M. (2019). Big data and IoT solution for patient behaviour monitoring. Behaviour & Information Technology, 38 , 940–949.

Song, C.-W., Jung, H., & Chung, K. (2019). Development of a medical big-data mining process using topic modeling. Cluster Computing, 22 , 1949–1958.

Khan, M., Iqbal, J., Talha, M., Arshad, M., Diyan, M., & Han, K. (2018). Big data processing using internet of software defined things in smart cities. International Journal of Parallel Programming . https://doi.org/10.1007/s10766-018-0573-y

Gohar, M., Muzammal, M., & Rahman, A. U. (2018). SMART TSS: Defining transportation system behavior using big data analytics in smart cities. Sustainable cities and society, 41 , 114–119.

Anbarasan, M., Muthu, B., Sivaparthipan, C., Sundarasekar, R., Kadry, S., Krishnamoorthy, S., et al. (2020). Detection of flood disaster system based on IoT, big data and convolutional deep neural network. Computer Communications, 150 , 150–157.

Luo, X., Oyedele, L. O., Ajayi, A. O., Monyei, C. G., Akinade, O. O., & Akanbi, L. A. (2019). Development of an IoT-based big data platform for day-ahead prediction of building heating and cooling demands. Advanced Engineering Informatics, 41 , 100926.

Hossain, M. A., Ferdousi, R., Hossain, S. A., Alhamid, M. F., & El Saddik, A. (2020). A novel framework for recommending data mining algorithm in dynamic iot environment. IEEE Access, 8 , 157333–157345.

Safa, M., & Pandian, A. (2021). Intelligent big data analytics model for efficient cardiac disease prediction with IoT devices in WSN using fuzzy rules. Wireless Personal Communications . https://doi.org/10.1007/s11277-021-08788-3

Alsaig, A., Alagar, V., Chammaa, Z., & Shiri, N. (2019). Characterization and efficient management of big data in IoT-driven smart city development. Sensors, 19 , 2430.

Tang, R., & Fong, S. (2018). Clustering big IoT data by metaheuristic optimized mini-batch and parallel partition-based DGC in Hadoop. Future Generation Computer Systems, 86 , 1395–1412.

Kotenko, I., Saenko, I., & Branitskiy, A. (2018). Framework for mobile internet of things security monitoring based on big data processing and machine learning. IEEE Access . https://doi.org/10.1109/ACCESS.2018.2881998

Wang, T., Bhuiyan, M. Z. A., Wang, G., Rahman, M. A., Wu, J., & Cao, J. (2018). Big data reduction for a smart city’s critical infrastructural health monitoring. IEEE Communications Magazine, 56 , 128–133.

Kaur, I., Lydia, E. L., Nassa, V. K., Shrestha, B., Nebhen, J., Malebary, S., et al. (2021). Generative adversarial networks with quantum optimization model for mobile edge computing in IoT big data. Wireless Personal Communications . https://doi.org/10.1007/s11277-021-08706-7

Lakshmanaprabu, S., Shankar, K., Ilayaraja, M., Nasir, A. W., Vijayakumar, V., & Chilamkurti, N. (2019). Random forest for big data classification in the internet of things using optimal features. International journal of machine learning and cybernetics, 10 , 2609–2618.

Ullah, F., Habib, M. A., Farhan, M., Khalid, S., Durrani, M. Y., & Jabbar, S. (2017). Semantic interoperability for big-data in heterogeneous IoT infrastructure for healthcare. Sustainable Cities and Society, 34 , 90–96.

Manogaran, G., Varatharajan, R., Lopez, D., Kumar, P. M., Sundarasekar, R., & Thota, C. (2018). A new architecture of Internet of Things and big data ecosystem for secured smart healthcare monitoring and alerting system. Future Generation Computer Systems, 82 , 375–387.

Hendawi, A., Gupta, J., Liu, J., Teredesai, A., Ramakrishnan, N., Shah, M., et al. (2019). Benchmarking large-scale data management for internet of things. The Journal of Supercomputing, 75 , 8207–8230.

Mo, Y. (2019). A data security storage method for IoT under hadoop cloud computing platform. International Journal of Wireless Information Networks, 26 , 152–157.

Tu, L., Liu, S., Wang, Y., Zhang, C., Li, P. (2019). An optimized cluster storage method for real-time big data in internet of things. The Journal of Supercomputing . 1–17.

Tripathi, A. K., Sharma, K., Bala, M., Kumar, A., Menon, V. G., & Bashir, A. K. (2020). A parallel military-dog-based algorithm for clustering big data in cognitive industrial internet of things. IEEE Transactions on Industrial Informatics, 17 , 2134–2142.

Alelaiwi, A. (2017). A collaborative resource management for big IoT data processing in Cloud. Cluster Computing, 20 , 1791–1799.

Meerja, K. A., Naidu, P. V., & Kalva, S. R. K. (2019). Price versus performance of big data analysis for cloud based internet of things networks. Mobile Networks and Applications, 24 , 1078–1094.

Wang, T., Liang, Y., Zhang, Y., Arif, M., Wang, J., & Jin, Q. (2020). An intelligent dynamic offloading from cloud to edge for smart IoT systems with big data. IEEE Transactions on Network Science and Engineering . https://doi.org/10.1109/TNSE.2020.2988052

Vasconcelos, D., Andrade, R., Severino, V., & Souza, J. D. (2019). Cloud, fog, or mist in IoT? That is the question. ACM Transactions on Internet Technology (TOIT), 19 , 1–20.

Jamil, B., Ijaz, H., Shojafar, M., Munir, K., & Buyya, R. (2022). Resource allocation and task scheduling in fog computing and internet of everything environments: A taxonomy, review, and future directions. ACM Computing Surveys (CSUR) . https://doi.org/10.1145/3513002

Javadzadeh, G., & Rahmani, A. M. (2020). Fog computing applications in smart cities: A systematic survey. Wireless Networks, 26 , 1433–1457.

Cao, B., Zhang, J., Liu, X., Sun, Z., Cao, W., Nowak, R. M., et al. (2021). Edge–cloud resource scheduling in space–air–ground-integrated networks for internet of vehicles. IEEE Internet of Things Journal, 9 , 5765–5772.

Linaje, M., Berrocal, J., & Galan-Benitez, A. (2019). Mist and edge storage: Fair storage distribution in sensor networks. IEEE Access, 7 , 123860–123876.

Mehdipour, F., Noori, H., & Javadi, B. (2016). Energy-efficient big data analytics in datacenters. In Advances in Computers . Vol. 100. Elsevier, pp. 59–101.

Zhou, L., Mao, H., Zhao, T., Wang, V. L., Wang, X., & Zuo, P. (2022). How B2B platform improves Buyers’ performance: Insights into platform’s substitution effect. Journal of Business Research, 143 , 72–80.

García-Magariño, I., Lacuesta, R., & Lloret, J. (2017). Agent-based simulation of smart beds with Internet-of-Things for exploring big data analytics. IEEE Access, 6 , 366–379.

Bi, Z., Jin, Y., Maropoulos, P., Zhang, W.-J., & Wang, L. (2021). Internet of things (IoT) and big data analytics (BDA) for digital manufacturing (DM). International Journal of Production Research . https://doi.org/10.1080/00207543.2021.1953181

Ahmed, I., Ahmad, M., Jeon, G., & Piccialli, F. (2021). A framework for pandemic prediction using big data analytics. Big Data Research, 25 , 100190.

Puschmann, D., Barnaghi, P., & Tafazolli, R. (2016). Adaptive clustering for dynamic IoT data streams. IEEE Internet of Things Journal, 4 , 64–74.

Bu, F. (2018). An efficient fuzzy c-means approach based on canonical polyadic decomposition for clustering big data in IoT. Future Generation Computer Systems, 88 , 675–682.

Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018). High-order possibilistic c-means algorithms based on tensor decompositions for big data in IoT. Information Fusion, 39 , 72–80.

Lavalle, A., Teruel, M. A., Maté, A., & Trujillo, J. (2020). Improving sustainability of smart cities through visualization techniques for big data from IoT devices. Sustainability, 12 , 5595.

Li, P., Chen, Z., Yang, L. T., Zhang, Q., & Deen, M. J. (2017). Deep convolutional computation model for feature learning on big data in internet of things. IEEE Transactions on Industrial Informatics, 14 , 790–798.

Patterson, E. K., Gurbuz, S., Tufekci, Z., & Gowdy, J. N. (2002). CUAVE: A new audio-visual database for multimodal human-computer interface research. In 2002 IEEE International conference on acoustics, speech, and signal processing , pp. II-2017-II-2020.

Zhang, Q., Yang, L. T., & Chen, Z. (2015). Deep computation model for unsupervised feature learning on big data. IEEE Transactions on Services Computing, 9 , 161–171.

Cauteruccio, F., Cinelli, L., Corradini, E., Terracina, G., Ursino, D., Virgili, L., et al. (2021). A framework for anomaly detection and classification in Multiple IoT scenarios. Future Generation Computer Systems, 114 , 322–335.

Liang, W., Li, W., & Feng, L. (2021). Information security monitoring and management method based on big data in the internet of things environment. IEEE Access, 9 , 39798–39812.

Vahdat, S. (2022). A review of pathophysiological mechanism, diagnosis, and treatment of thrombosis risk associated with COVID-19 infection. IJC Heart & Vasculature . https://doi.org/10.1016/j.ijcha.2022.101068

Abbasi, S., Naderi, Z., Amra, B., Atapour, A., Dadkhahi, S. A., Eslami, M. J., et al. (2021). Hemoperfusion in patients with severe COVID-19 respiratory failure, lifesaving or not? Journal of Research in Medical Sciences, 26 , 34.

Li, W., Chai, Y., Khan, F., Jan, S. R. U., Verma, S., Menon, V. G., et al. (2021). A comprehensive survey on machine learning-based big data analytics for IoT-enabled smart healthcare system. Mobile Networks and Applications, 26 , 234–252.

Biswas, R. (2022). Outlining big data analytics in health sector with special reference to Covid-19. Wireless Personal Communications, 124 , 2097–2108.

Wu, X., Zhang, Y., Wang, A., Shi, M., Wang, H., & Liu, L. (2020). MNSSp3: Medical big data privacy protection platform based on Internet of things. Neural Computing and Applications . https://doi.org/10.1007/s00521-020-04873-z

Elhoseny, M., Abdelaziz, A., Salama, A. S., Riad, A. M., Muhammad, K., & Sangaiah, A. K. (2018). A hybrid model of internet of things and cloud computing to manage big data in health services applications. Future generation computer systems, 86 , 1383–1394.

Jan, M. A., He, X., Song, H., & Babar, M. (2021). Machine learning and big data analytics for IoT-enabled smart cities. Mobile Networks and Applications, 26 , 156–158.

Liu, Z., Wang, Y., & Feng, J. (2022). Vehicle-type strategies for manufacturer’s car sharing. Kybernetes . https://doi.org/10.1108/K-11-2021-1095

Khan, M. A., Siddiqui, M. S., Rahmani, M. K. I., & Husain, S. (2021). Investigation of big data analytics for sustainable smart city development: An emerging country. IEEE Access, 10 , 16028–16036.

Sivaparthipan, C., Muthu, B. A., Manogaran, G., Maram, B., Sundarasekar, R., Krishnamoorthy, S., et al. (2020). Innovative and efficient method of robotics for helping the Parkinson’s disease patient using IoT in big data analytics. Transactions on Emerging Telecommunications Technologies, 31 , e3838.

Yang, L., Xiong, Z., Liu, G., Hu, Y., Zhang, X., & Qiu, M. (2021). An analytical model of page dissemination for efficient big data transmission of C-ITS. IEEE Transactions on Intelligent Transportation Systems . https://doi.org/10.1109/TITS.2021.3134557

Zantalis, F., Koulouras, G., Karabetsos, S., & Kandris, D. (2019). A review of machine learning and IoT in smart transportation. Future Internet, 11 , 94.

Guo, J., Liu, R., Cheng, D., Shanthini, A., & Vadivel, T. (2022). Urbanization based on IoT using big data analytics the impact of internet of things and big data in urbanization. Arabian Journal for Science and Engineering . https://doi.org/10.1007/s13369-021-06124-2

Shao, N. (2022). Research on architectural planning and landscape design of smart city based on computational intelligence. Computational Intelligence and Neuroscience. 2022.

Jia, T., Cai, C., Li, X., Luo, X., Zhang, Y., & Yu, X. (2022). Dynamical community detection and spatiotemporal analysis in multilayer spatial interaction networks using trajectory data. International Journal of Geographical Information Science . https://doi.org/10.1080/13658816.2022.2055037

Kahveci, S., Alkan, B., Mus’ab, H. A., Ahmad, B., & Harrison, R. (2022). An end-to-end big data analytics platform for IoT-enabled smart factories: A case study of battery module assembly system for electric vehicles. Journal of Manufacturing Systems, 63 , 214–223.

Nitti, M., Girau, R., & Atzori, L. (2013). Trustworthiness management in the social internet of things. IEEE Transactions on knowledge and data engineering, 26 , 1253–1266.

Shahab, S., Agarwal, P., Mufti, T., & Obaid, A. J. (2022). SIoT (social internet of things): A review. ICT Analysis and Applications . https://doi.org/10.1007/978-981-16-5655-2_28

Atzori, L., Iera, A., Morabito, G., & Nitti, M. (2012). The social internet of things (siot)–when social networks meet the internet of things: Concept, architecture and network characterization. Computer networks, 56 , 3594–3608.

Baldassarre, G., Giudice, P. L., Musarella, L., & Ursino, D. (2019). The MIoT paradigm: Main features and an “ad-hoc” crawler. Future Generation Computer Systems, 92 , 29–42.

Meghana, J., Hanumanthappa, J., & Prakash, S. S. (2021). Performance comparison of machine learning algorithms for data aggregation in social internet of things. Global Transitions Proceedings, 2 , 212–219.

Lo Giudice, P., Nocera, A., Ursino, D., & Virgili, L. (2019). Building topic-driven virtual iots in a multiple iots scenario. Sensors, 19 , 2956.

McCall, J. (1994). Quality factors, encyclopedia of software engineering. (vol. 2, p. 760). New York: Wiley

Boehm, B., & In, H. (1996). Identifying quality-requirement conflicts. IEEE software, 13 , 25–35.

Grady, R. B. (1992). Practical software metrics for project management and process improvement : Prentice-Hall, Inc.

Talia, D. (2019). A view of programming scalable data analysis: From clouds to exascale. Journal of Cloud Computing, 8 , 1–16.

Firmani, D., Mecella, M., Scannapieco, M., & Batini, C. (2016). On the meaningfulness of “big data quality.” Data Science and Engineering, 1 , 6–20.

Jabbar, S., Ullah, F., Khalid, S., Khan, M., & Han, K. (2017). Semantic interoperability in heterogeneous IoT infrastructure for healthcare. Wireless Communications and Mobile Computing, 2017

Rialti, R., Marzi, G., Caputo, A., & Mayah, K. A. (2020) Achieving strategic flexibility in the era of big data. Management Decision .

Roy, D., Srivastava, R., Jat, M., & Karaca, M. S. (2022). A complete overview of analytics techniques: descriptive, predictive, and prescriptive. Decision intelligence analytics and the implementation of strategic business management, 15–30.

Rahul, K., Banyal, R. K., Goswami, P., & Kumar, V. (2021). Machine learning algorithms for big data analytics. In Computational Methods and Data Engineering , Springer, pp. 359–367.

Nti, I. K., Quarcoo, J. A., Aning, J., & Fosu, G. K. (2022). A mini-review of machine learning in big data analytics: Applications, challenges, and prospects. Big Data Mining and Analytics, 5 , 81–97.

Rajendran, R., Sharma, P., Saran, N. K., Ray, S., Alanya-Beltran, J., & Tongkachok, K. (2022) An exploratory analysis of machine learning adaptability in big data analytics environments: A data aggregation in the age of big data and the internet of things. In 2022 2nd International Conference on Innovative Practices in Technology and Management (ICIPTM) , pp. 32–36.

Angelopoulos, A., Michailidis, E. T., Nomikos, N., Trakadas, P., Hatziefremidis, A., Voliotis, S., et al. (2019). Tackling faults in the industry 4.0 era—a survey of machine-learning solutions and key aspects. Sensors, 20 , 109.

Zhou, L., Pan, S., Wang, J., & Vasilakos, A. V. (2017). Machine learning on big data: Opportunities and challenges. Neurocomputing, 237 , 350–361.

Prastyo, D. D., Khoiri, H. A., Purnami, S. W., Fam, S.-F., & Suhermi, N. (2020). Survival support vector machines: A simulation study and its health-related application. Supervised and Unsupervised Learning for Data Science (pp. 85–100). Cham: Springer.

Chapter   Google Scholar  

Pink, C. M. (2016). Forensic ancestry assessment using cranial nonmetric traits traditionally applied to biological distance studies. In Biological Distance Analysis , Elsevier, pp. 213–230.

Lu, W. (2019). Improved K-means clustering algorithm for big data mining under Hadoop parallel framework. Journal of Grid Computing . https://doi.org/10.1007/s10723-019-09503-0

Zheng, W., Liu, X., & Yin, L. (2021). Research on image classification method based on improved multi-scale relational network. PeerJ Computer Science, 7 , e613.

Goswami, S., & Kumar, A. (2022). Survey of deep-learning techniques in big-data analytics. Wireless Personal Communications . https://doi.org/10.1007/s11277-022-09793-w

Roni, M., Karim, H., Rana, M., Pota, H., Hasan, M., & Hussain, M. (2022). Recent trends in bio-inspired meta-heuristic optimization techniques in control applications for electrical systems: A review. International Journal of Dynamics and Control . https://doi.org/10.1007/s40435-021-00892-3

Swayamsiddha, S. (2020). Bio-inspired algorithms: principles, implementation, and applications to wireless communication. In Nature-Inspired Computation and Swarm Intelligence . Elsevier, pp. 49–63.

Ni, J., Wu, L., Fan, X., & Yang, S. X. (2016). Bioinspired intelligent algorithm and its applications for mobile robot control: a survey. Computational intelligence and neuroscience, 2016 .

Game, P. S., & Vaze, D. (2020). Bio-inspired Optimization: metaheuristic algorithms for optimization. arXiv preprint arXiv:2003.11637 .

Romero, C. D. G., Barriga, J. K. D., & Molano, J. I. R. (2016) Big data meaning in the architecture of IoT for smart cities. In International Conference on Data Mining and Big Data , pp. 457–465.

Santana, E. F. Z., Chaves, A. P., Gerosa, M. A., Kon, F., & Milojicic, D. S. (2017). Software platforms for smart cities: Concepts, requirements, challenges, and a unified reference architecture. ACM Computing Surveys (Csur), 50 , 1–37.

Granat, J., Batalla, J. M., Mavromoustakis, C. X., & Mastorakis, G. (2019). Big data analytics for event detection in the IoT-multicriteria approach. IEEE Internet of Things Journal, 7 , 4418–4430.

Xiong, Z., Zhang, Y., Luong, N. C., Niyato, D., Wang, P., & Guizani, N. (2020). The best of both worlds: A general architecture for data management in blockchain-enabled Internet-of-Things. IEEE Network, 34 , 166–173.

Oktian, Y. E., Lee, S.-G., & Lee, B.-G. (2020). Blockchain-based continued integrity service for IoT big data management: A comprehensive design. Electronics, 9 , 1434.

Liu, F., Zhang, G., & Lu, J. (2020). Multisource heterogeneous unsupervised domain adaptation via fuzzy relation neural networks. IEEE Transactions on Fuzzy Systems, 29 , 3308–3322.

Dong, J., Cong, Y., Sun, G., Fang, Z., & Ding, Z. (2021). Where and how to transfer: knowledge aggregation-induced transferability perception for unsupervised domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence . https://doi.org/10.1109/TPAMI.2021.3128560

Zenggang, X., Xiang, L., Xueming, Z., Sanyuan, Z., Fang, X., Xiaochao, Z., et al. (2022). A service pricing-based two-stage incentive algorithm for socially aware networks. Journal of Signal Processing Systems . https://doi.org/10.1007/s11265-022-01768-1

Benhamaid, S., Lakhlef, H., & Bouabdallah, A. (2021) Towards energy efficient mobile data collection in cluster-based IoT networks. In 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) , pp. 340-343.

Sun, W., Lv, X., & Qiu, M. (2020). Distributed estimation for stochastic Hamiltonian systems with fading wireless channels. IEEE Transactions on Cybernetics .

Lv, Z., Qiao, L., & You, I. (2020). 6G-enabled network in box for internet of connected vehicles. IEEE transactions on intelligent transportation systems, 22 , 5275–5282.

Xifilidis, T., & Psannis, K. E. (2022). Correlation-based wireless sensor networks performance: The compressed sensing paradigm. Cluster Computing, 25 , 965–981.

Mohammadi, A., Ciuonzo, D., Khazaee, A., & Rossi, P. S. (2022). Generalized locally most powerful tests for distributed sparse signal detection. IEEE Transactions on Signal and Information Processing over Networks, 8 , 528–542.

Aziz, A., Osamy, W., Khedr, A. M., El-Sawy, A. A., & Singh, K. (2020). Grey Wolf based compressive sensing scheme for data gathering in IoT based heterogeneous WSNs. Wireless Networks, 26 , 3395–3418.

Djelouat, H., Amira, A., & Bensaali, F. (2018). Compressive sensing-based IoT applications: A review. Journal of Sensor and Actuator Networks, 7 , 45.

Wang, K., Zhang, B., Alenezi, F., & Li, S. (2022). Communication-efficient surrogate quantile regression for non-randomly distributed system. Information Sciences, 588 , 425–441.

Lee, G. H., Han, J., & Choi, J. K. (2021). MPdist-based missing data imputation for supporting big data analyses in IoT-based applications. Future Generation Computer Systems, 125 , 421–432.

Zhang, F., Zhai, J., Shen, X., Mutlu, O., & Du, X. (2021). POCLib: A high-performance framework for enabling near orthogonal processing on compression. IEEE Transactions on Parallel and Distributed Systems, 33 , 459–475.

Abualigah, L., Diabat, A., & Elaziz, M. A. (2021). Intelligent workflow scheduling for big data applications in IoT cloud computing environments. Cluster Computing, 24 , 2957–2976.

Naas, M. I., Lemarchand, L., Raipin, P., & Boukhobza, J. (2021). IoT data replication and consistency management in fog computing. Journal of Grid Computing, 19 , 1–25.

Ma, Z., Zheng, W., Chen, X., & Yin, L. (2021). Joint embedding VQA model based on dynamic word vector. PeerJ Computer Science, 7 , e353.

Rahouma, K. H., Aly, R. H., & Hamed, H. F. (2020). Challenges and solutions of using the social internet of things in healthcare and medical solutions—a survey. Toward Social Internet of Things (SIoT): Enabling Technologies, Architectures and Applications (pp. 13–30). Cham: Springer.

Corradini, E., Nicolazzo, S., Nocera, A., Ursino, D., & Virgili, L. (2022). A two-tier Blockchain framework to increase protection and autonomy of smart objects in the IoT. Computer Communications, 181 , 338–356.

Pincheira, M., Antonini, M., & Vecchio, M. (2022). Integrating the IoT and blockchain technology for the next generation of mining inspection systems. Sensors, 22 , 899.

Tchagna Kouanou, A., Tchito Tchapga, C., Sone Ekonde, M., Monthe, V., Mezatio, B. A., Manga, J., et al. (2022). Securing data in an internet of things network using blockchain technology: smart home case. SN Computer Science, 3 , 1–10.

Ursino, D., & Virgili, L. (2020). An approach to evaluate trust and reputation of things in a Multi-IoTs scenario. Computing, 102 , 2257–2298.

Download references

Author information

Arezou Naghib

Present address: Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

Authors and Affiliations

Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

Arash Sharifi

Department of Computer Engineering, Faculty of Engineering and Natural Sciences, Kadir Has University, Istanbul, Turkey

Nima Jafari Navimipour

Department of Computer Engineering, Tabriz Branch, Islamic Azad University, Tabriz, Iran

Institute of Research and Development, Duy Tan University, Da Nang, Vietnam

Mehdi Hosseinzadeh

School of Medicine and Pharmacy, Duy Tan University, Da Nang, Vietnam

Computer Science, University of Human Development, Sulaymaniyah, 0778-6, Iraq

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Nima Jafari Navimipour .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Naghib, A., Jafari Navimipour, N., Hosseinzadeh, M. et al. A comprehensive and systematic literature review on the big data management techniques in the internet of things. Wireless Netw 29 , 1085–1144 (2023). https://doi.org/10.1007/s11276-022-03177-5

Download citation

Accepted : 19 October 2022

Published : 15 November 2022

Issue Date : April 2023

DOI : https://doi.org/10.1007/s11276-022-03177-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Big data management
• Internet of things
• Knowledge discovery
• Systematic literature review (SLR)
• Find a journal
• Publish with us
• Track your research

• DOI: 10.4236/JCC.2015.35021
• Corpus ID: 43291586

Internet of Things (IoT): A Literature Review

• Somayya Madakam , R. Ramaswamy , Siddharth Tripathi
• Published 25 May 2015
• Computer Science, Engineering
• Journal of Computational Chemistry

Figures from this paper

1,307 Citations

A study on internet of things, a small review on internet of things (iot), internet of things (iot), applications and challenges: a comprehensive review, analysis on internet of things, application, challenges and related future technologies (a literature review), internet of things (iot) a gateway for smarter life, an analysis of internet of things (iot): novel architectures, modern applications, security aspects and future scope with latest case studies.

• Highly Influenced

Internet of Things (IoT): The Most Up-To-Date Challenges, Architectures, Emerging Trends and Potential Opportunities

Towards a more in-depth understanding of the iot paradigm and its challenges.

• 11 Excerpts

Design of a Simple Gas Knob: An Application of IoT

A review on internet of things’ operating systems, platforms and applications, 37 references, internet of things - applications and challenges in technology and standardization, future internet: the internet of things architecture, possible applications and key challenges, integrating rfids and smart objects into a unified internet of things architecture, internet of things (iot): a vision, architectural elements, and future directions, architecture of internet of things and its key technology integration based-on rfid, rfid security in the context of "internet of things", research on the architecture of internet of things, spamming the internet of things: a possibility and its probable solution, the internet of things, research on key technology and applications for internet of things, related papers.

Showing 1 through 3 of 0 Related Papers

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Machine learning-based intrusion detection methods in iot systems: a comprehensive review.

1. Introduction

2. materials and methods, 2.1. eligibility criteria, 2.2. data sources and search strategy, data  sources, 2.3. search strategy.

• IoT: ”IoT”, ”Internet of Things”, ”IoT system”.
• Intrusion detection: ”intrusion detection”, ”anomaly detection”, ”cybersecurity”.
• Machine learning: ”machine learning”, ”artificial intelligence”, ”ML”, ”AI”, ”deep learning”, ”supervised learning”, ”unsupervised learning”, ”neural network”, ”random forest”, ”support vector machine”, ”SVM”, ”Random Forest”, ”Decision Tree”, ”DNN”, ”ANN”, ”KNN”, ”GAN”, ”logistic regression”, ”ANN”.
• Challenges: ”security challenges”, ”IoT security issues”, ”cybersecurity challenges”, ”AI challenges”, ”threat detection challenges”, ”IoT vulnerabilities”, ”AI limitations in IoT security”.

2.4. Study Selection

• Title evaluation;
• Abstract and keyword evaluation;
• Full-text evaluation.

3. Internet of Things

3.1. definition and growth of iot, 3.2. basic architecture of iot, 4. classification of iot attacks based on vulnerabilities and layers, 4.1. categorization of vulnerabilities, 4.1.1. physical layer, 4.1.2. network layer, 4.1.3. application layer, 4.2. categorization of attacks, 4.2.1. perception layer, 4.2.2. network layer, 4.2.3. application layer, 5. traditional intrusion detection methods, 5.1. classification based on data source, 5.2. classification based on detection method, 5.3. limits of traditional approaches and comparison with machine learning-based ids, 6. machine learning for intrusion detection, 7. supervised learning, 7.1. popular methods and applications, 7.1.1. artificial neural networks (anns), 7.1.2. support vector machine (svm), 7.1.3. k-nearest neighbors (knns), 7.1.4. logistic regression (lr), 7.1.5. decision tree, 8. unsupervised learning, 8.1. popular methods and applications, 8.1.1. clustering, 8.1.2. principal component analysis (pca), 8.1.3. autoencoders, 8.1.4. density-based anomaly detection, 8.2. semi-supervised learning, 9. deep learning, 9.1. popular methods and applications, 9.1.1. deep neural networks (dnns), 9.1.2. convolutional neural networks (cnns), 9.1.3. recurrent neural networks (rnns), 9.1.4. generative adversarial networks (gans), 9.2. review of datasets, 10. discussion, 10.1. challenges and current limitations, 10.2. current trends, 11. conclusions, author contributions, conflicts of interest.

• Ashton, K. That ’Internet Of Things’ Thing. RFID J. 2009 , 2 , 97–114. [ Google Scholar ]
• Perera, C.; Liu, C.H.; Jayawardena, S.; Chen, M. A Survey on Internet of Things From Industrial Market Perspective. IEEE Access 2014 , 2 , 1660–1679. [ Google Scholar ] [ CrossRef ]
• Islam, N.; Farhin, F.; Sultana, I.; Kaiser, M.S.; Rahman, M.S.; Mahmud, M.; Hosen, A.S.M.S.; Cho, G.H. Towards Machine Learning Based Intrusion Detection in IoT Networks. Comput. Mater. Contin. 2021 , 69 , 1801–1821. [ Google Scholar ] [ CrossRef ]
• Ahmad, Z.; Khan, A.S.; Nisar, K.; Haider, I.; Hassan, R.; Haque, M.R.; Tarmizi, S.; Rodrigues, J.J.P.C. Anomaly Detection Using Deep Neural Network for IoT Architecture. Appl. Sci. 2021 , 11 , 7050. [ Google Scholar ] [ CrossRef ]
• Union Internationale des Télécommunications. Infrastructure Mondiale de l’Information, Protocole Internet et RÉSeaux de Prochaine Génération ; UIT: Tromsø, Norway, 2012. [ Google Scholar ]
• Corici, A.A.; Emmelmann, M.; Luo, J.; Shrestha, R.; Corici, M.; Magedanz, T. IoT inter-security domain trust transfer and service dispatch solution. In Proceedings of the 2016 IEEE 3rd World Forum on Internet of Things (WF-IoT), Reston, VA, USA, 12–14 December 2016; pp. 694–699. [ Google Scholar ] [ CrossRef ]
• Sha, K.; Errabelly, R.; Wei, W.; Yang, T.A.; Wang, Z. EdgeSec: Design of an Edge Layer Security Service to Enhance IoT Security. In Proceedings of the 2017 IEEE 1st International Conference on Fog and Edge Computing (ICFEC), Madrid, Spain, 14–15 May 2017; pp. 81–88. [ Google Scholar ] [ CrossRef ]
• Al-Sarawi, S.; Anbar, M.; Abdullah, R.; Al Hawari, A.B. Internet of Things Market Analysis Forecasts, 2020–2030. In Proceedings of the 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4), London, UK, 27–28 July 2020; pp. 449–453. [ Google Scholar ] [ CrossRef ]
• Görmüş, S.; Aydın, H.; Ulutaş, G. Security for the internet of things: A survey of existing mechanisms, protocols and open research issues. J. Fac. Eng. Archit. Gazi Univ. 2018 , 33 , 1247–1272. [ Google Scholar ]
• Ibrahim, M.; Abdullah, M.T.; Abdullah, A.; Perumal, T. An Epidemic Based Model for the Predictions of OOFI in an IoT Platform. Int. J. Eng. Trends Technol. 2020 , 52–56. [ Google Scholar ] [ CrossRef ]
• Rebah, H.B. Gateway IoT de pilotage et de surveillance des capteurs domestiques via le protocole MQTT. Researche Gate 2022 , 3. [ Google Scholar ]
• Atzori, L.; Iera, A.; Morabito, G. The Internet of Things: A survey. Comput. Netw. 2010 , 54 , 2787–2805. [ Google Scholar ] [ CrossRef ]
• Lombardi, M.; Pascale, F.; Santaniello, D. Internet of Things: A General Overview between Architectures, Protocols and Applications. Information 2021 , 12 , 87. [ Google Scholar ] [ CrossRef ]
• Hasan, M.A.M.; Nasser, M.; Ahmad, S.; Molla, K.I. Feature Selection for Intrusion Detection Using Random Forest. J. Inf. Secur. 2016 , 7 , 129–140. [ Google Scholar ] [ CrossRef ]
• Zhao, K.; Ge, L. A Survey on the Internet of Things Security. In Proceedings of the 2013 Ninth International Conference on Computational Intelligence and Security, Emeishan, China, 14–15 December 2013; pp. 663–667. [ Google Scholar ] [ CrossRef ]
• Alaba, F.A.; Othman, M.; Hashem, I.A.T.; Alotaibi, F. Internet of Things security: A survey. J. Netw. Comput. Appl. 2017 , 88 , 10–28. [ Google Scholar ] [ CrossRef ]
• Sicari, S.; Rizzardi, A.; Grieco, L.; Coen-Porisini, A. Security, privacy and trust in Internet of Things: The road ahead. Comput. Netw. 2015 , 76 , 146–164. [ Google Scholar ] [ CrossRef ]
• Jing, D.; Chen, H.B. SVM Based Network Intrusion Detection for the UNSW-NB15 Dataset. In Proceedings of the 2019 IEEE 13th International Conference on ASIC (ASICON), Chongqing, China, 29 October–1 November 2019; pp. 1–4. [ Google Scholar ] [ CrossRef ]
• Weber, R.H.; Studer, E. Cybersecurity in the Internet of Things: Legal aspects. Comput. Law Secur. Rev. 2016 , 32 , 715–728. [ Google Scholar ] [ CrossRef ]
• Chen, Z.; Liu, J.; Shen, Y.; Simsek, M.; Kantarci, B.; Mouftah, H.T.; Djukic, P. Machine Learning-Enabled IoT Security: Open Issues and Challenges under Advanced Persistent Threats. ACM Comput. Surv. 2023 , 55 , 1–37. [ Google Scholar ] [ CrossRef ]
• Rathore, S.; Park, J.H. Semi-supervised learning based distributed attack detection framework for IoT. Appl. Soft Comput. 2018 , 72 , 79–89. [ Google Scholar ] [ CrossRef ]
• Mishra, A.K.; Tripathy, A.K.; Puthal, D.; Yang, L.T. Analytical Model for Sybil Attack Phases in Internet of Things. IEEE Internet Things J. 2019 , 6 , 379–387. [ Google Scholar ] [ CrossRef ]
• Chen, B.; Ho, D.W.C.; Hu, G.; Yu, L. Secure Fusion Estimation for Bandwidth Constrained Cyber-Physical Systems under Replay Attacks. IEEE Trans. Cybern. 2018 , 48 , 1862–1876. [ Google Scholar ] [ CrossRef ]
• Chen, L.; Kuang, X.; Xu, A.; Suo, S.; Yang, Y. A Novel Network Intrusion Detection System Based on CNN. In Proceedings of the 2020 Eighth International Conference on Advanced Cloud and Big Data (CBD), Taiyuan, China, 5–6 December 2020; pp. 243–247. [ Google Scholar ] [ CrossRef ]
• Liu, H.; Lang, B. Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey. Appl. Sci. 2019 , 9 , 4396. [ Google Scholar ] [ CrossRef ]
• Sundararajan, K.; Garg, L.; Srinivasan, K.; Bashir, A.K.; Kaliappan, J.; Ganapathy, G.P.; Selvaraj, S.K.; Meena, T. A Contemporary Review on Drought Modeling Using Machine Learning Approaches. Comput. Model. Eng. Sci. 2021 , 128 , 447–487. [ Google Scholar ] [ CrossRef ]
• Anitha, A.A.; Arockiam, D.L. ANNIDS: Artificial Neural Network based Intrusion Detection System for Internet of Things. Int. J. Innov. Technol. Explor. Eng. 2019 , 8 , 2583–2588. [ Google Scholar ] [ CrossRef ]
• Hanif, S.; Ilyas, T.; Zeeshan, M. Intrusion Detection In IoT Using Artificial Neural Networks On UNSW-15 Dataset. In Proceedings of the 2019 IEEE 16th International Conference on Smart Cities: Improving Quality of Life Using ICT & IoT and AI (HONET-ICT), Charlotte, NC, USA, 6–9 October 2019; pp. 152–156. [ Google Scholar ] [ CrossRef ]
• Jamal, A.; Faisal, H.M.; Nasir, M. Malware Detection and Classification in IoT Network using ANN. Mehran Univ. Res. J. Eng. Technol. 2022 , 41 , 80–91. [ Google Scholar ] [ CrossRef ]
• Goeschel, K. Reducing false positives in intrusion detection systems using data-mining techniques utilizing support vector machines, decision trees, and naive Bayes for off-line analysis. In Proceedings of the SoutheastCon 2016, Norfolk, VA, USA, 30 March–3 April 2016; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Ioannou, C.; Vassiliou, V. Network Attack Classification in IoT Using Support Vector Machines. J. Sens. Actuator Netw. 2021 , 10 , 58. [ Google Scholar ] [ CrossRef ]
• Pouyanfar, S.; Sadiq, S.; Yan, Y.; Tian, H.; Tao, Y.; Reyes, M.P.; Shyu, M.L.; Chen, S.C.; Iyengar, S.S. A Survey on Deep Learning. ACM Comput. Surv. 2019 , 51 , 1–36. [ Google Scholar ] [ CrossRef ]
• Zhu, R.; Ji, X.; Yu, D.; Tan, Z.; Zhao, L.; Li, J.; Xia, X. KNN-Based Approximate Outlier Detection Algorithm over IoT Streaming Data. IEEE Access 2020 , 8 , 42749–42759. [ Google Scholar ] [ CrossRef ]
• Abdaljabar, Z.H.; Ucan, O.N.; Alheeti, K.M.A. An Intrusion Detection System for IoT Using KNN and Decision-Tree Based Classification. In Proceedings of the 2021 International Conference of Modern Trends in Information and Communication Technology Industry (MTICTI), Sana’a, Yemen, 4–6 December 2021; pp. 1–5. [ Google Scholar ] [ CrossRef ]
• Li, W.; Yi, P.; Wu, Y.; Pan, L.; Li, J. A New Intrusion Detection System Based on KNN Classification Algorithm in Wireless Sensor Network. J. Electr. Comput. Eng. 2014 , 2014 , 1–8. [ Google Scholar ] [ CrossRef ]
• Govindarajan, M.; Chandrasekaran, R. Intrusion detection using k-Nearest Neighbor. In Proceedings of the 2009 First International Conference on Advanced Computing, Chennai, India, 13–15 December 2009; pp. 13–20. [ Google Scholar ] [ CrossRef ]
• Aref, M.A.; Jayaweera, S.K.; Machuzak, S. Multi-Agent Reinforcement Learning Based Cognitive Anti-Jamming. In Proceedings of the 2017 IEEE Wireless Communications and Networking Conference (WCNC), San Francisco, CA, USA, 19–22 March 2017; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Bapat, R.; Mandya, A.; Liu, X.; Abraham, B.; Brown, D.E.; Kang, H.; Veeraraghavan, M. Identifying malicious botnet traffic using logistic regression. In Proceedings of the 2018 Systems and Information Engineering Design Symposium (SIEDS), Charlottesville, VA, USA, 27–27 April 2018; pp. 266–271. [ Google Scholar ] [ CrossRef ]
• Sambangi, S.; Gondi, L. A Machine Learning Approach for DDoS (Distributed Denial of Service) Attack Detection Using Multiple Linear Regression. Proceedings 2020 , 63 , 51. [ Google Scholar ] [ CrossRef ]
• Qaddoura, R.; Al-Zoubi, A.M.; Almomani, I.; Faris, H. A Multi-Stage Classification Approach for IoT Intrusion Detection Based on Clustering with Oversampling. Appl. Sci. 2021 , 11 , 3022. [ Google Scholar ] [ CrossRef ]
• Ingre, B.; Yadav, A.; Soni, A.K. Decision Tree Based Intrusion Detection System for NSL-KDD Dataset ; Springer: Berlin/Heidelberg, Germany, 2018; pp. 207–218. [ Google Scholar ] [ CrossRef ]
• Rai, K.; Devi, M.S.; Guleria, A. Decision Tree Based Algorithm for Intrusion Detection. Adv. Netw. Appl. 2016 , 7 , 2828–2834. [ Google Scholar ]
• Al-Jarrah, O.Y.; Al-Hammdi, Y.; Yoo, P.D.; Muhaidat, S.; Al-Qutayri, M. Semi-supervised multi-layered clustering model for intrusion detection. Digit. Commun. Netw. 2018 , 4 , 277–286. [ Google Scholar ] [ CrossRef ]
• Muniyandi, A.P.; Rajeswari, R.; Rajaram, R. Network Anomaly Detection by Cascading K-Means Clustering and C4.5 Decision Tree Algorithm. Procedia Eng. 2012 , 30 , 174–182. [ Google Scholar ] [ CrossRef ]
• Peng, K.; Leung, V.C.M.; Huang, Q. Clustering Approach Based on Mini Batch Kmeans for Intrusion Detection System over Big Data. IEEE Access 2018 , 6 , 11897–11906. [ Google Scholar ] [ CrossRef ]
• Luo, T.; Nagarajan, S.G. Distributed Anomaly Detection Using Autoencoder Neural Networks in WSN for IoT. In Proceedings of the 2018 IEEE International Conference on Communications (ICC), Kansas City, MO, USA, 20–24 May 2018; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Aboelwafa, M.M.N.; Seddik, K.G.; Eldefrawy, M.H.; Gadallah, Y.; Gidlund, M. A Machine-Learning-Based Technique for False Data Injection Attacks Detection in Industrial IoT. IEEE Internet Things J. 2020 , 7 , 8462–8471. [ Google Scholar ] [ CrossRef ]
• Garg, S.; Kaur, K.; Batra, S.; Kaddoum, G.; Kumar, N.; Boukerche, A. A multi-stage anomaly detection scheme for augmenting the security in IoT-enabled applications. Future Gener. Comput. Syst. 2020 , 104 , 105–118. [ Google Scholar ] [ CrossRef ]
• Al-Garadi, M.A.; Mohamed, A.; Al-Ali, A.K.; Du, X.; Ali, I.; Guizani, M. A Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security. IEEE Commun. Surv. Tutor. 2020 , 22 , 1646–1685. [ Google Scholar ] [ CrossRef ]
• Brun, O.; Yin, Y.; Yin, Y.; Gelenbe, E. Deep Learning with Dense Random Neural Network for Detecting Attacks against IoT-Connected Home Environments. In Security in Computer and Information Sciences: First International ISCIS Security Workshop 2018, Euro-CYBERSEC 2018, London, UK, February 26–27 2018, Revised Selected Papers 1 ; Springer International Publishing: Berlin/Heidelberg, Germany, 2018; pp. 458–463. [ Google Scholar ]
• Kim, J.; Shin, N.; Jo, S.Y.; Kim, S.H. Method of intrusion detection using deep neural network. In Proceedings of the 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju, Republic of Korea, 13–16 February 2017; pp. 313–316. [ Google Scholar ] [ CrossRef ]
• Kim, J.; Kim, J.; Kim, H.; Shim, M.; Choi, E. CNN-Based Network Intrusion Detection against Denial-of-Service Attacks. Electronics 2020 , 9 , 916. [ Google Scholar ] [ CrossRef ]
• Park, S.H.; Park, H.J.; Choi, Y.J. RNN-Based Prediction for Network Intrusion Detection. In Proceedings of the 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Fukuoka, Japan, 19–21 February 2020; pp. 572–574. [ Google Scholar ] [ CrossRef ]
• Tang, T.A.; Mhamdi, L.; McLernon, D.; Zaidi, S.A.R.; Ghogho, M. Deep Recurrent Neural Network for Intrusion Detection in SDN-Based Networks. In Proceedings of the 2018 4th IEEE Conference on Network Softwarization and Workshops (NetSoft), Montreal, QC, Canada, 25–29 June 2018; pp. 202–206. [ Google Scholar ] [ CrossRef ]
• Torres, P.; Catania, C.; Garcia, S.; Garino, C.G. An analysis of Recurrent Neural Networks for Botnet detection behavior. In Proceedings of the 2016 IEEE Biennial Congress of Argentina (ARGENCON), Buenos Aires, Argentina, 15–17 June 2016; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput. 1997 , 9 , 1735–1780. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Evaluation of gated recurrent neural networks on sequence modeling. arXiv 2014 , arXiv:1412.3555. [ Google Scholar ]
• Ferdowsi, A.; Saad, W. Generative Adversarial Networks for Distributed Intrusion Detection in the Internet of Things. In Proceedings of the 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA, 9–13 December 2019; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Liao, D.; Huang, S.; Tan, Y.; Bai, G. Network Intrusion Detection Method Based on GAN Model. In Proceedings of the 2020 International Conference on Computer Communication and Network Security (CCNS), Xi’an, China, 21–23 August 2020; pp. 153–156. [ Google Scholar ] [ CrossRef ]
• Panda, M.; Patra, M.R. Network Intrusion Detection Using Naïve Bayes. IJCSNS Int. J. Comput. Sci. Netw. Secur. 2007 , 7 , 258–263. [ Google Scholar ]
• Gumus, F.; Sakar, C.O.; Kursun, O. Network Intrusion Detection Using Naïve Bayes. In Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, Beijing, China, 17–20 August 2014. [ Google Scholar ]
• Kim, J.; Kim, J.; Thu, H.L.T.; Kim, H. Long Short Term Memory Recurrent Neural Network Classifier for Intrusion Detection. In Proceedings of the 2016 International Conference on Platform Technology and Service (PlatCon), Jeju, Republic of Korea, 15–17 February 2016; pp. 1–5. [ Google Scholar ] [ CrossRef ]
• Zarpelão, B.B.; Miani, R.S.; Kawakani, C.T.; de Alvarenga, S.C. A survey of intrusion detection in Internet of Things. J. Netw. Comput. Appl. 2017 , 84 , 25–37. [ Google Scholar ] [ CrossRef ]
• Cassales, G.W.; Senger, H.; de Faria, E.R.; Bifet, A. IDSA-IoT: An Intrusion Detection System Architecture for IoT Networks. In Proceedings of the 2019 IEEE Symposium on Computers and Communications (ISCC), Barcelona, Spain, 29 June–3 July 2019; pp. 1–7. [ Google Scholar ]
• Yahyaoui, A.; Lakhdhar, H.; Abdellatif, T.; Attia, R. Machine learning based network intrusion detection for data streaming IoT applications. In Proceedings of the 2021 21st ACIS International Winter Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD-Winter), Ho Chi Minh City, Vietnam, 28–30 January 2021; pp. 51–56. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Kikissagbe, B.R.; Adda, M. Machine Learning-Based Intrusion Detection Methods in IoT Systems: A Comprehensive Review. Electronics 2024 , 13 , 3601. https://doi.org/10.3390/electronics13183601

Kikissagbe BR, Adda M. Machine Learning-Based Intrusion Detection Methods in IoT Systems: A Comprehensive Review. Electronics . 2024; 13(18):3601. https://doi.org/10.3390/electronics13183601

Kikissagbe, Brunel Rolack, and Meddi Adda. 2024. "Machine Learning-Based Intrusion Detection Methods in IoT Systems: A Comprehensive Review" Electronics 13, no. 18: 3601. https://doi.org/10.3390/electronics13183601

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

A Literature Review on Internet of Things (IoT)

Related Papers

Treffyn Koreshoff

This paper offers an HCI perspective on the emergent agenda of the Internet of Things (IoT). The purpose is to provide insights and resources for how HCI could engage productively with the IoT agenda while it is still evolving and being realised. We examined and reviewed HCI- related literature and commercial products of the IoT, categorising a final collection of 89 research papers and 93 commercial products into two tables. Through this, we are able to provide a snapshot of the types, extent and foci of both research and commercial efforts. It has also revealed trends, opportunities, as well as gaps for how HCI could proceed when engaging more deeply with the IoT. Finally, this review provides insights for HCI, suggesting tools, methods and potential approaches that can help ensure a human-centred IoT.

Proceedings of the 25th Australian Computer-Human Interaction Conference on Augmentation, Application, Innovation, Collaboration - OzCHI '13

Tuck Wah Leong

Journal of Ambient Intelligence and Smart Environments

Somaya Ben Allouch

This paper surveys recent Internet of Things (IoT) related HCI literature, and examines it in light of a comprehensive framework by Atzori et al. (2010). Mapping HCI literature to this framework helped us understand the extent and the focus of IoT related HCI efforts, including a lack of HCI engagement with deeper human-centred perspectives of the IoT. It also revealed HCI considerations for the IoT which we added to the framework. This extended the framework to a tool for an HCI audience that can be used for ‘thinking through’ the design of IoT technologies. We close the paper by demonstrating how this tool has been found useful in an IoT research project and at the same time illustrating our approach in how to engage more deeply with human- centred concerns.

Communications of the Association for Information Systems

Viswanath Venkatesh

The Internet of Things (IoT) is emerging as an integrated set of digital innovations with the potential to unleash unprecedented opportunities as well as to create significant challenges from both technological and societal perspectives. The emergence of IoT heralds a new dimension of a digital era with impact and influence that are not yet fully clear. This signals the opening of valuable opportunities for scholarly inquiries, particularly for information systems (IS) scholars. We posit that, as the IS discipline sits at the intersection of technical, business, and social applications of IT, which are also the essential dimensions of the impact of IoT, IS scholars are well positioned to understand and contribute to advancing research on this new topic and associated phenomena. This paper outlines the distinctive attributes of IoT and their implications for existing traditions of IS research. It further highlights some illustrative research perspectives from which IoT can be studied by IS scholars. We highlight a research agenda for IS in two different ways: first, by suggesting four categories of implications on IS research: (1) introduction of the physio-digital continuum; (2) multi-level exploration of IS; (3) composite affordances; and (4) heterogeneity; and second, by introducing four thematic impact domains: (1) impact on organizations; (2) impact on technology; (3) impact on individuals; and (4) impact on society.

Journal of Ambient Intelligence and Humanized Computing

Kaja Ystgaard

The rapid evolution of Internet of Things (IoT) technologies, with ever more profound implications for humans and societies, has triggered visions and initiatives to re-align the Next-Generation IoT with what works for humans and humanity first. However, despite the increased push towards “human-centric” IoT, it is still poorly understood what “human-centric” actually means in this context, how it is interpreted and embedded into the design, by whom, and for which purposes. To address these questions, we conducted a systematic literature review (N = 84) on the theory, principles, and design requirements of human-centric IoT. A key observation is that, despite the recent increase in research on humane perspectives for IoT, “human-centredness” often still seems to be used primarily as a label and overarching paradigm, not leading to a profound change in the underlying practices. We found no shared understanding of what “human-centric” implies in this context or common agreement on whi...

International Journal of Distributed and Cloud Computing

Publishing India Group

The primary objective of this research is to identify and analyze all the issues and challenges associated with the Internet-of-Things in the context of business management. The paper develops an exploratory research model to identify and analyze all the issues and challenges associated with the Internet-of-Things in business administration. The researcher gathered all the relevant data and information with the help of a previous literature review recently published on the Internet-of-Things and business management. To compile relevant content on the subject, the researcher used secondary resources such as Internet sites, the official website of organizations that have adopted the Internet-of-Things for management, published research papers, peer-reviewed journals, magazines, periodical articles, and books. This research study will aid researchers and management in developing a unique security framework to minimize and eliminate security and privacy risks and the adoption of the Internet-of-Things using new communication tools and technologies in business management practices. According to the findings of this study, the Internet-of-Things not only assist in business processes and activities but also improve the efficiency and productivity of the available things.

… and Applications for …

Sergio Gusmeroli

Nomusa Dlodlo , Promise Sthembiso Mvelase , T. Foko , Sizakele Mathba

JEFFERSON F A L C Ã O SALES

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Jonathan Woetzel

Journal of Computer and Communications

Dr. Siddharth Tripathi

Eija Kaasinen , Heikki Ailisto , Seppo Leminen , Eila Ovaska

Journal of Software Engineering Research and Development

Rebeca Motta

IT Professional

Abdulmajeed A D I L Yazdeen

selvakumar samuel

Salvatore Ferrigno

European Journal of Interdisciplinary Studies

Raluca Constantinescu

Leonardo Da Vinci Sydney

Advances in Science, Technology and Engineering Systems Journal

Manuel Monteiro

Irene C L Ng , Susan Wakenshaw

Journal of Theoretical and Applied Electronic Commerce Research

Victor Tiberius

Dawda Wally

International Journal of Advanced Computer Science and Applications

Bhagyashree Ambore

Monica A. Senor

Industry 4.0 - Perspectives and Applications- IntechOpen

Benjamin Appiah Osei , Emmanuela Kwao-Boateng

Dr. Rajinder Tiwari

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024