Cassandra

 Apache Cassandra Tutorial

Apache Cassandra

* Cassandra is an open-source distributed database management system.

* Apache Cassandra is highly scalable, distributed and high-performance NoSQL database.

* Cassandra is designed to handle a huge amount of data across multiple data centers with no single point of failure.

* Cassandra handles the huge amount of data with its distributed architecture. Data is placed on different machines with more than one replication factor that provides high availability and no single point of failure.

    Let us first understand what a NoSQL database does.

NoSQLDatabase
* A NoSQL database (sometimes called as Not Only SQL) is a database that provides a mechanism to store and retrieve data other than the tabular relations such as relational databases.

* These databases are schema-free, support easy replication, have simple API, eventually consistent, and can handle huge amounts of data.

* NoSql databases use different data structures compared to relational databases. It makes some operations faster in NoSQL.

* NoSQL databases include MongoDB, HBase, and Cassandra. There are following properties of NoSQL databases.
1. Design Simplicity
2. Horizontal Scaling
3. High Availability

* Data structures used in Cassandra are more specified than data structures used in relational databases. Cassandra data structures are faster than relational database structures.

Nosql Cassandra Database Vs Relational Databases



History of Cassandra
* Apache Cassandra was originally developed at Facebook in 2008 to power Facebook’s in-box search feature.

* The original authors were Avinash Lakshman, who also is one of the authors of the Amazon Dynamo paper, and Prashant Malik.

* After being in production at Facebook for a while, Cassandra was released as an open-source project on Google Code in July of 2008.

* In March of 2009, it was accepted to the Apache Foundation as an incubator project.

* In February of 2010, it became a top-level Apache project.

Who Uses Cassandra?
     Cassandra is in wide use around the world, and usage is growing all the time. Companies like Netflix, eBay, Twitter, Reddit, and Ooyala all use Cassandra to power pieces of their architecture, and it is critical to the day-to-day operations of those organisations.

     Before we get too deep into Cassandra, it is important to understand some of the basic concepts that surround databases so you know what concessions you may have to make when choosing a system. There are three main sets of properties that define what database systems are capable of. Those are ACID, CAP, and BASE.

ACID
     ACID stands for Atomicity, Consistency, Isolation, and Durability. In order to understand ACID and how it relates to databases, we need to talk about transactions. When it comes to databases, a transaction is defined as a single logical operation. For example, if you are shopping online, every time you add an item to your shopping cart, that item and its quantity make up the database transaction. Even if you add multiple items or multiple quantities of the same item with a single click, that entire shopping cart addition is just a single transaction.
Atomicity
     Atomicity means that each transaction either works or it doesn’t. This is to say that if any single part of the transaction fails, the entire transaction fails. This should hold true for every situation related to a transaction that could cause a failure. Network failure, power outage, or even a node outage occurring at transaction time should cause a complete transaction failure in an atomic system.

Consistency
     Consistency ensures that when a transaction is complete, whether it is successful or not, the database is still in a valid state. This means that any data written to the database must also be valid. When writing data to the database, you also need to include any database application-level rules such as constraints, cascades, triggers, or stored procedures. The application of those rules should also leave the data in a valid state.

Isolation 
    Isolation is a property that ensures that all transactions that are run concurrently appear as if they were executed serially (one right after the other). Each transaction must be run in a vacuum (isolation). This is to say that if two transactions are run at the same time, they remain independent of each other during the transaction. Some examples of isolation are locks (table, row, column, etc.), dirty reads, and deadlocks. The reason these are relevant is concurrency. Multiple changes can be attempted on the same data or set of data. Knowing what version of the data is the correct one is important for keeping the entire system in a sane state.

Durability
     Changes must be saved permanently to the database, that is the amount deposited must be added to your previous balance. It might happen that the transaction survives all hurdles and is written in the log which stores the entries to be finalized in the database.

CAP
      CAP stands for Consistency, Availability, and Partition tolerance. Although the C in CAP also stands for “consistency” (similar to the C in ACID), the meaning is different. Consistency means that all nodes in a grouping see the same data at the same time. In other words, any particular query hitting any node in the system will return the same result for that specific query. Consistency also further implies that when a query updates a value in one node, the data will be updated to reflect the new value prior to the next query.

     The availability of a system speaks to the guarantee that regardless of the success or failure of a request, the requestor will receive a response. This means that system operations will be able to continue even if part of the system is down, whatever the reason. Availability is what lets the software attempt to cope with and compensate for externalities such as hardware failures, network outages, power failures, and the like.

     Partition tolerance refers to the capability of a distributed system to effectively distribute the load across multiple nodes. The load could be data or queries. This implies that even if a few nodes are down, the system will continue to function. Sharding is a commonly used management technique for distributing load across a cluster. Sharding, which is similar to horizontal partitioning, is a way of splitting data into separate parts and moving them to another server or physical location, generally for performance improvements.

BASE
     BASE stands for Basically Available, Soft state, and Eventual consistency.Having a system be basically available means that the system will respond to any request. The caveat is that the response may be a failure to get the data or that the data may be in an inconsistent or changing state.

     The idea of a soft-state system means the system is always changing. This is typically due to eventual consistency. It is common for soft-state systems to undergo changes even when there is no additional input to them.

     Eventual consistency refers to the concept that once a system stops receiving input, the data will propagate to wherever else it needs to be in the system sooner or later. The beauty of this is that the system does not check for consistency on every transaction as is expected in an ACID-compliant system.

 Cassandra Installation


Step 1
      Run the Datastax community edition setup. You can download here Cassandra Datastax community edition setup from Apache website. After running the Setup, following page will be displayed. Here in the screenshot 64 bit version is being installed. You can download 32 bit version as well according to your requirements. But I recommend 64 bit version to use.


     Here this page gives you information about the Cassandra version you are going to install. Press the 'next' button.

Step 2
     After pressing the 'next' button, then following page will be displayed.


     Here this page is about the license agreement. Mark the checkbox and press the next button.

Step 3
     After pressing the 'next' button, the following page will be displayed. This page asks about the installation location.
1. Default location is C:\Program Files. You can change installation location if you want to change. It is recommended not to change installation location.

2. After setting installation location, press the 'next' button

Step 4
     After pressing 'next' button in above step, the following page will be displayed. This page asks about whether you want to automatically start Cassandra and OpsCenter.
1. Mark the checkboxes if you to want to automatically start Cassandra and opsCenter.
2. After providing this information, press the 'next' button.


Step 5
     After pressing the next button, following page will be displayed.

     Setup has collected all the necessary information and now the setup is ready to install. Press install button.

Step 6
     After pressing 'install' button, following page will be displayed.

     Datastax community edition is being installed. After installation is completed, click on next button. When setup is installed successfully, press the 'Finish' button. 
     Now go to windows start programs, search Cassandra CQL Shell and run the Cassandra Shell. After running Cassandra shell, you will see the following command line.


     Now you connect to cassandra and create the keyspace and tables.

 Cassandra Architecture


Cassandra Architecture
* Cassandra is designed to handle big data. 
* Cassandra main feature is to store data on multiple nodes with no single point of failure. 
* The reason for this kind of Cassandra’s architecture was that the hardware failure can occur at any time. Any node can be down. In case of failure data stored in another node can be used. Hence, Cassandra is designed with its distributed architecture.
* Cassandra has peer-to-peer distributed system across its nodes, and data is distributed among all the nodes in a cluster.
* All the nodes in a cluster play the same role. Each node is independent and at the same time interconnected to other nodes.
* Each node in a cluster can accept read and write requests, regardless of where the data is actually located in the cluster.
* When a node goes down, read/write requests can be served from other nodes in the network.
* All the nodes exchange information with each other using Gossip protocol. Gossip is a protocol in Cassandra by which nodes can communicate with each other. 
Node
* A Node is an instance of Cassandra. A Cassandra cluster is made up of many nodes.
* Node is the basic component of Cassandra.
* Node is the place where data is stored. 
Data Center
* Collection of nodes are refered as a data center. Many nodes are categorized as a data center.
Cluster
* Cluster is the collection of many data centers.

Commit Log
* Commit log is used for crash recovery.
* When you perform a write operation, it’s immediately written to the commit log. The commit log is a crash-recovery mechanism that supports Cassandra’s durability goals
Mem-table
* After data written to the commit log, the value is written to a memory-resident data structure called the memtable.
* Data is written in Mem-table temporarily.
SSTable
* When the number of objects stored in the memtable reaches a threshold, the contents of the memtable are flushed to disk in a file called an SSTable. A new memtable is then created.

Cassandra Data Replication
     In a distributed system like Cassandra, data replication enables high availability and durability. As hardware problem can occur or link can be down at any time during data process, a solution is required to provide a backup when the problem has occurred. So data is replicated for assuring no single point of failure. Cassandra places replicas of data on different nodes based on these two factors.

1. Where to place next replica is determined by the Replication Strategy.
2. While the total number of replicas placed on different nodes is determined by the Replication Factor.
     One Replication factor means that there is only a single copy of data while three replication factor means that there are three copies of the data on three different nodes.

     For ensuring there is no single point of failure, replication factor must be three. There are two kinds of replication strategies in Cassandra.

1. SimpleStrategy
     SimpleStrategy is used when you have just one data center. SimpleStrategy places the first replica on the node selected by the partitioner. After that, remaining replicas are placed in clockwise direction in the Node ring. Here is the pictorial representation of the SimpleStrategy. It is commonly used when nodes are in a single data center.


2. NetworkTopologyStrategy
     NetworkTopologyStrategy is used when you have more than two data centers. In NetworkTopologyStrategy, replicas are set for each data center separately. NetworkTopologyStrategy places replicas in the clockwise direction in the ring until reaches the first node in another rack. This strategy tries to place replicas on different racks in the same data center. This is due to the reason that sometimes failure or problem can occur in the rack. Then replicas on other nodes can provide data. Here is the pictorial representation of the Network topology strategy.


Write Operation
     The coordinator sends a write request to replicas. If all the replicas are up, they will receive write request regardless of their consistency level. Consistency level determines how many nodes will respond back with the success acknowledgement. The node will respond back with the success acknowledgement if data is written successfully to the commit log and memTable.

     For example, in a single data center with replication factor equals to three, three replicas will receive write request. If consistency level is one, only one replica will respond back with the success acknowledgement, and the remaining two will remain dormant. Suppose if remaining two replicas lose data due to node downs or some other problem, Cassandra will make the row consistent by the built-in repair mechanism in Cassandra.Here it is explained, how write process occurs in Cassandra,
1. When write request comes to the node, first of all, it logs in the commit log.

2. Then Cassandra writes the data in the mem-table. Data written in the mem-table on each write request also writes in commit log separately. Mem-table is a temporarily stored data in the memory while Commit log logs the transaction records for back up purposes.

3. When mem-table is full, data is flushed to the SSTable data file.


Read Operation
     There are three types of read requests that a coordinator sends to replicas.
1. Direct request
2. Digest request
3. Read repair request

     The coordinator sends direct request to one of the replicas. After that, the coordinator sends the digest request to the number of replicas specified by the consistency level and checks whether the returned data is an updated data. After that, the coordinator sends digest request to all the remaining replicas. If any node gives out of date value, a background read repair request will update that data. This process is called read repair mechanism.

 Cassandra Architecture


Cassandra Architecture
* Cassandra is designed to handle big data. 
* Cassandra main feature is to store data on multiple nodes with no single point of failure. 
* The reason for this kind of Cassandra’s architecture was that the hardware failure can occur at any time. Any node can be down. In case of failure data stored in another node can be used. Hence, Cassandra is designed with its distributed architecture.
* Cassandra has peer-to-peer distributed system across its nodes, and data is distributed among all the nodes in a cluster.
* All the nodes in a cluster play the same role. Each node is independent and at the same time interconnected to other nodes.
* Each node in a cluster can accept read and write requests, regardless of where the data is actually located in the cluster.
* When a node goes down, read/write requests can be served from other nodes in the network.
* All the nodes exchange information with each other using Gossip protocol. Gossip is a protocol in Cassandra by which nodes can communicate with each other. 
Node
* A Node is an instance of Cassandra. A Cassandra cluster is made up of many nodes.
* Node is the basic component of Cassandra.
* Node is the place where data is stored. 
Data Center
* Collection of nodes are refered as a data center. Many nodes are categorized as a data center.
Cluster
* Cluster is the collection of many data centers.

Commit Log
* Commit log is used for crash recovery.
* When you perform a write operation, it’s immediately written to the commit log. The commit log is a crash-recovery mechanism that supports Cassandra’s durability goals
Mem-table
* After data written to the commit log, the value is written to a memory-resident data structure called the memtable.
* Data is written in Mem-table temporarily.
SSTable
* When the number of objects stored in the memtable reaches a threshold, the contents of the memtable are flushed to disk in a file called an SSTable. A new memtable is then created.

Cassandra Data Replication
     In a distributed system like Cassandra, data replication enables high availability and durability. As hardware problem can occur or link can be down at any time during data process, a solution is required to provide a backup when the problem has occurred. So data is replicated for assuring no single point of failure. Cassandra places replicas of data on different nodes based on these two factors.

1. Where to place next replica is determined by the Replication Strategy.
2. While the total number of replicas placed on different nodes is determined by the Replication Factor.
     One Replication factor means that there is only a single copy of data while three replication factor means that there are three copies of the data on three different nodes.

     For ensuring there is no single point of failure, replication factor must be three. There are two kinds of replication strategies in Cassandra.

1. SimpleStrategy
     SimpleStrategy is used when you have just one data center. SimpleStrategy places the first replica on the node selected by the partitioner. After that, remaining replicas are placed in clockwise direction in the Node ring. Here is the pictorial representation of the SimpleStrategy. It is commonly used when nodes are in a single data center.


2. NetworkTopologyStrategy
     NetworkTopologyStrategy is used when you have more than two data centers. In NetworkTopologyStrategy, replicas are set for each data center separately. NetworkTopologyStrategy places replicas in the clockwise direction in the ring until reaches the first node in another rack. This strategy tries to place replicas on different racks in the same data center. This is due to the reason that sometimes failure or problem can occur in the rack. Then replicas on other nodes can provide data. Here is the pictorial representation of the Network topology strategy.


Write Operation
     The coordinator sends a write request to replicas. If all the replicas are up, they will receive write request regardless of their consistency level. Consistency level determines how many nodes will respond back with the success acknowledgement. The node will respond back with the success acknowledgement if data is written successfully to the commit log and memTable.

     For example, in a single data center with replication factor equals to three, three replicas will receive write request. If consistency level is one, only one replica will respond back with the success acknowledgement, and the remaining two will remain dormant. Suppose if remaining two replicas lose data due to node downs or some other problem, Cassandra will make the row consistent by the built-in repair mechanism in Cassandra.Here it is explained, how write process occurs in Cassandra,
1. When write request comes to the node, first of all, it logs in the commit log.

2. Then Cassandra writes the data in the mem-table. Data written in the mem-table on each write request also writes in commit log separately. Mem-table is a temporarily stored data in the memory while Commit log logs the transaction records for back up purposes.

3. When mem-table is full, data is flushed to the SSTable data file.


Read Operation
     There are three types of read requests that a coordinator sends to replicas.
1. Direct request
2. Digest request
3. Read repair request

     The coordinator sends direct request to one of the replicas. After that, the coordinator sends the digest request to the number of replicas specified by the consistency level and checks whether the returned data is an updated data. After that, the coordinator sends digest request to all the remaining replicas. If any node gives out of date value, a background read repair request will update that data. This process is called read repair mechanism.


 Cassandra Data Model


     When creating a data model for your keyspace, the most important thing to do is to forget everything you know about relational data modeling. Relational data models are designed for efficient storage, relational lookups, and associations between concerns. The Cassandra data model is designed for raw performance and storage of vast amounts of data.


     Unlike relational databases, the data model for Cassandra is based on the query patterns required. This means that you have to know the read/write patterns before you create. your data model.

Cassandra Data Model
     To understand how to model in Cassandra, you must first understand how the Cassandra data model works. When creating a table using CQL, you are not only telling Cassandra what the name and type of data are, you are also telling it how to store and distribute your data. This is done via the PRIMARY KEY operator. The PRIMARY KEY tells the Cassandra storage system to distribute the data based on the value of this key; this is known as a partition key. When there are multiple fields in the PRIMARY KEY, as is the case with compound keys, the first field is the partition key (how the data is distributed) and the subsequent fields are known as the clustering keys (how the data is stored on disk).

     Clustering keys allow you to pregroup your data by the values in the keys. Using compound keys in Cassandra is commonly referred to as “wide rows.” “Wide rows” refers to the rows that Cassandra is storing on disk, rather than the rows that are represented to you when you make a query.

     Following figure shows how the data might be stored in a five-node cluster using PRIMARY KEY.
CREATE TABLE animals (
name TEXT PRIMARY KEY,
species TEXT,
subspecies TEXT,
genus TEXT,
family TEXT
);

SELECT * FROM animals;

name |family   | genus | species          | subspecies
------------------------------------------------------------
dog  | Canidae | Canis | C. lupus         | C. l. familiaris
cat  | Felidae | Felis | F. catus         | null
duck | Anatidae| Anas  | A. platyrhynchos | null
wolf | Canidae | Canis | C. lupus         | null
     Following figure shows how the data might be stored in a five-node cluster using COMPOUND KEY.

CREATE TABLE animals (
name TEXT,
species TEXT,
subspecies TEXT,
genus TEXT,
family TEXT,
PRIMARY KEY(family, genus)
);

SELECT * FROM animals;

name | family   | genus | species          | subspecies
-------------------------------------------------------------
dog  | Canidae  | Canis | C. lupus         | C. l. familiaris
wolf | Canidae  | Canis | C. lupus         | null
cat  | Felidae  | Felis | F. catus         | null
duck | Anatidae | Anas  | A. platyrhynchos | null
     When we use a COMPOUND KEY, the data for wolf and for dog is stored on the same server. This is because we changed the partition to “family” and clustered on “genus.” Literally, this means that the data for each family will be stored on the same replica sets and presorted, or clustered, by the genus. This will allow for very fast lookups when the family and genus for an animal are known.

0 comments:

Post a Comment