System and method for extending security of Big Data Systems using Blockchain technologies

The invention is in the field of Big Data Systems built on the basis of Hadoop, in which data security is expanded by applying additional technological solutions typical of Blockchain systems (Blockchain networks), identified as Blockchain technologies. 

Description 

The system consists of 21 functional blocks that perform various tasks of processing, protection, verification and management of data 

Key blocks include: 

• Data collection, cleaning and enrichment blocks – process and prepare data for storage and analysis. 
• Real-time data management blocks – enable fast processing and response when needed. 
• Blockchain ledger creation and management blocks – here data is recorded in the form of blocks, each with its own cryptographic protection, hashes and authentication parameters. 
• Verification blocks – check the correctness and integrity of data in the blockchain ledger through parallel checks. 
• Access management and authentication blocks – mechanisms such as Kerberos and cryptographic keys are used to authenticate and control user access. 
• Automatic correction blocks – when errors are detected in the blockchain ledger, the system automatically corrects the data using replicas. 

Two advanced security levels: 

1.First advanced level 

• Data is processed with increased security and can be used in almost real time (for example, via Spark, HBase, Cassandra). 
• User authentication and authorization through integration of Kerberos and blockchain mechanisms. 
• Data is anonymized and protected from unauthorized access. 

2.Second advanced level 

• Data is stored as a blockchain ledger, where each block contains information about creation time, source, content, authentication parameters and hash of the previous block. 
• Here, security is even higher, but processing is not in real time, but with a faster response than a classic blockchain network. 
• In case of errors in the data, the system automatically corrects the content using several replicas of the blockchain ledger. 

Key features and benefits 

• Automated data processing and protection – removing duplicate records, correcting inconsistencies, anonymizing sensitive information. 
• Access control and auditing – every access, change or entry in the blockchain ledger is verified and logged. 
• Flexible integration – can be added to existing big data systems without requiring a separate blockchain infrastructure. 
• High level of security and transparency – through a combination of traditional (Kerberos) and modern (blockchain) technologies. 
• Automatic error correction – guarantees data integrity without human intervention. 

Opportunities 

This system and method enable organizations to significantly expand the security of their big data platforms by combining the advantages of blockchain (immutability, traceability, decentralization) with the possibility of fast processing and easy integration to already existing infrastructures. This provides more secure management, storage and sharing of large volumes of data, with automatic control and protection against unauthorized access or manipulation.