Real-Time Big Data Analytical Architecture for Remote Sensing Application

  • September 15, 2015
  • Blog
No Comments

Real-Time Big Data Analytical Architecture for Remote Sensing Application

ABSTRACT:

The assets of remote senses digital world daily generate massive volume of real-time data (mainly referred to the term “Big Data”), where insight information has a potential significance if collected and aggregated effectively. In today’s era, there is a great deal added to real-time remote sensing Big Data than it seems at first, and extracting the useful information in an efficient manner leads a system toward a major computational challenges, such as to analyze, aggregate, and store, where data are remotely collected. Keeping in view the above mentioned factors, there is a need for designing a system architecture that welcomes both realtime, as well as offline data processing. Therefore, in this paper, we propose real-time Big Data analytical architecture for remote sensing satellite application. The proposed architecture comprises three main units, such as 1) remote sensing Big Data acquisition unit (RSDU); 2) data processing unit (DPU); and 3) data analysis decision unit (DADU). First, RSDU acquires data from the satellite and sends this data to the Base Station, where initial processing takes place. Second, DPU plays a vital role in architecture for efficient processing of real-time Big Data by providing filtration, load balancing, and parallel processing. Third, DADU is the upper layer unit of the proposed architecture, which is responsible for compilation, storage of the results, and generation of decision based on the results received from DPU. The proposed architecture has the capability of dividing, load balancing, and parallel processing of only useful data. Thus, it results in efficiently analyzing real-time remote sensing Big Data using earth observatory system. Furthermore, the proposed architecture has the capability of storing incoming raw data to perform offline analysis on largely stored dumps, when required. Finally, a detailed analysis of remotely sensed earth observatory Big Data for land and sea area are provided using Hadoop. In addition, various algorithms are proposed for each level of RSDU, DPU, and DADU to detect land as well as sea area to elaborate the working of an architecture.

EXISTING SYSTEM:

  • Most recently designed sensors used in the earth and planetary observatory system are generating continuous stream of data.
  • Moreover, majority of work have been done in the various fields of remote sensory satellite image data, such as change detection, gradient-based edge detection, region similarity based edge detection, and intensity gradient technique for efficient intraprediction.

DISADVANTAGES OF EXISTING SYSTEM:

  • Consequences of transformation of remotely sensed data to the scientific understanding are a critical task.
  • Normally, the data collected from remote areas are not in a format ready for analysis.
  • In remote access networks, where the data source such as sensors can produce an overwhelming amount of raw data.

PROPOSED SYSTEM:

  • In this paper, we referred the high speed continuous stream of data or high volume offline data to “Big Data,” which is leading us to a new world of challenges.
  • This paper presents a remote sensing Big Data analytical architecture, which is used to analyze real time, as well as offline data. At first, the data are remotely preprocessed, which is then readable by the machines. Afterward, this useful information is transmitted to the Earth Base Station for further data processing.
  • Earth Base Station performs two types of processing, such as processing of real-time and offline data. In case of the offline data, the data are transmitted to offline data-storage device.
  • The incorporation of offline data-storage device helps in later usage of the data, whereas the real-time data is directly transmitted to the filtration and load balancer server, where filtration algorithm is employed, which extracts the useful information from the Big Data.
  • On the other hand, the load balancer balances the processing power by equal distribution of the real-time data to the servers. The filtration and load-balancing server not only filters and balances the load, but it is also used to enhance the system efficiency.
  • The proposed architecture and the algorithms are implemented in Hadoop using MapReduce programming by applying remote sensing earth observatory data.
  • The proposed architecture is composed of three major units, such as 1) RSDU; 2) DPU; and 3) DADU. These units implement algorithms for each level of the architecture depending on the required analysis.

ADVANTAGES OF PROPOSED SYSTEM:

  • With data acquisition, in which much of the data are of no interest that can be filtered or compressed by orders of magnitude. With a view to using such filters, they do not discard useful information.
  • With data extraction, which drags out the useful information from the underlying sources and delivers it in a structured formation suitable for analysis. For instance, the data set is reduced to single-class label to facilitate analysis, even though the first thing that we used to think about Big Data as always describing the fact.
  • The incorporation of offline data-storage device helps in later usage of the data,
  • The load balancer balances the processing power by equal distribution of the real-time data to the servers.

SYSTEM ARCHITECTURE:

snip 5

SYSTEM FLOW:

snip 6

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS:

  • System :         Pentium IV 2.4 GHz.
  • Hard Disk :         40 GB.
  • Floppy Drive : 44 Mb.
  • Monitor : 15 VGA Colour.
  • Mouse :
  • Ram : 512 Mb.

SOFTWARE REQUIREMENTS:

  • Operating system : Windows 7/UBUNTU.
  • Coding Language : Java 1.7 ,Hadoop 0.8.1
  • IDE : Eclipse
  • Database : MYSQL

REFERENCE:

Muhammad Mazhar Ullah Rathore, Anand Paul, Senior Member, IEEE, Awais Ahmad, Student Member, IEEE, Bo-Wei Chen, Member, IEEE, Bormin Huang, and Wen Ji, Member, IEEE, “Real-Time Big Data Analytical Architecture for Remote Sensing Application”, IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, 2015.

Fields marked with an * are required

(optional)

(optional)

Please, let us know the best time to contact you by phone (if provided).

I would like to get new blog posts by email