Big data resources
The following resources may help in understanding big data architecture and components:
Apache Hadoop project : A project that contains libraries that allows for the distributed processing of large data sets across clusters of computers using simple programming models. There are several modules, including the Hadoop Distributed File System (HDFS): A distributed file system that provides high-throughput access to application data and Hadoop MapReduce, which is a key algorithm to distribute work around a cluster.
Avro: A data serialization system
Cassandra: A scalable multi-master database with no single points of failure
HBase: A scalable, distributed database that supports structured data storage for large tables
Hive: A data warehouse infrastructure that provides data summarization and on-demand querying
Pig: A high-level, data-flow language and execution framework for parallel computation
ZooKeeper: A high-performance coordination service for distributed applications
MongoDB: A NoSQL open source document-oriented database system developed and supported by 10gen
Splunk: A data collection, visualization and indexing engine for operational intelligence that is developed by Splunk, Inc.
CouchDB: A NoSQL open source document-oriented database system developed and supported by Apache
Sqoop: Software for transferring data between relational databases and Hadoop
Oozie: A workflow scheduler system to manage Hadoop jobs
Last updated
Was this helpful?