Cassandra: Structured Storage System over a P2P Network

Cassandra: Structured Storage System over a P2P Network

Cassandra Structured Storage System over a P2P Network Avinash Lakshman, Prashant Malik Why Cassandra? Lots of data Copies of messages, reverse indices of messages, per user data. Many incoming requests resulting in a lot of random reads and random writes. No existing production ready solutions in the market meet these requirements. Design Goals High availability Eventual consistency trade-off strong consistency in favor of high availability

Incremental scalability Optimistic Replication Knobs to tune tradeoffs between consistency, durability and latency Low total cost of ownership Minimal administration Data Model Name : tid2 Value : Value : Value : Value : TimeStamp : t1

TimeStamp : t2 TimeStamp : t3 TimeStamp : t4 ColumnFamily1 Name : MailList KEY Column Families are declared upfront SuperColumns are added and modified Columns are dynamically

added and modified dynamically Name : tid1 Columns are added and Type : Simple Sort : Name modified Name : tid3 dynamically Name : tid4 ColumnFamily2 Name : WordList Name : aloha

Type : Super Sort : Time Name : dude C1 C2 C3 C4 C2 C6 V1

V2 V3 V4 V2 V6 T1 T2 T3 T4 T2

T6 Write Operations A client issues a write request to a random node in the Cassandra cluster. The Partitioner determines the nodes responsible for the data. Locally, write operations are logged and then applied to an in-memory version. Commit log is stored on a dedicated disk local to the machine. Write Properties No locks in the critical path

Sequential disk access Behaves like a write back Cache Append support without read ahead Atomicity guarantee for a key per replica Always Writable accept writes during failure scenarios Read Client Query Result Cassandra Cluster Closest replica Read repair if digests differ

Result Replica A Digest Response Replica B Digest Query Digest Response Replica C Cluster Membership and Failure Detection

Gossip protocol is used for cluster membership. Super lightweight with mathematically provable properties. State disseminated in O(logN) rounds where N is the number of nodes in the cluster. Every T seconds each member increments its heartbeat counter and selects one other member to send its list to. A member merges the list with its own list . Accrual Failure Detector Valuable for system management, replication, load balancing etc. Defined as a failure detector that outputs a value, PHI, associated with each process. Also known as Adaptive Failure detectors - designed to adapt to

changing network conditions. The value output, PHI, represents a suspicion level. Applications set an appropriate threshold, trigger suspicions and perform appropriate actions. In Cassandra the average time taken to detect a failure is 10-15 seconds with the PHI threshold set at 5. Properties of the Failure Detector If a process p is faulty, the suspicion level (t) as t . If a process p is faulty, there is a time after which (t) is monotonic increasing. A process p is correct (t) has an ub over an infinite execution. If process p is correct, then for any time T, (t) = 0 for t >= T.

Performance Benchmark Loading of data - limited by network bandwidth. Read performance for Inbox Search in production: Search Interactions Term Search Min 7.69 ms 7.78 ms Median 15.69 ms 18.27 ms Average

26.13 ms 44.41 ms Lessons Learnt Add fancy features only when absolutely required. Many types of failures are possible. Big systems need proper systems-level monitoring. Value simple designs Questions?

Recently Viewed Presentations

  • Gene-specific Methylation Analyses

    Gene-specific Methylation Analyses

    methylation. in mammals occurs at . CpG. sites, where it serves many functions: Self vs. non-self identification (such as in viral/bacterial infection) Target for further regulatory events; examples: Maintenance methyltransferases (functions after replication on hemi-methylated DNA) Proteins involved in chromatin...
  • Digital Games and Sociology Research Alex Burns (

    Digital Games and Sociology Research Alex Burns ([email protected])

    Digital Games and Sociology Research Alex Burns ([email protected]) Smart Internet Technology CRC 13 September 2005 Industry & Government Partners Agenda Computer Game History Global and Australian Industry Context Auteurs and Independents Digital Game-Based Learning Game Studies Computer Game History 1...
  • Volume of Prisms 1) Defining Volume 2) Volume

    Volume of Prisms 1) Defining Volume 2) Volume

    Volume of Prisms 1) Defining Volume 2) Volume = lwh 3) Volume = Bh Created by: David W. Cummins 3units 3units 3units 1unit 1unit 1unit 1 cubic unit Volume is the space that a figure occupies. It is measured in...
  • Welcome to Radiology

    Welcome to Radiology

    The change of direction of the radiation due to the angle of the bevel on the anode. Ideal bevel angle is <15° from vertical. This makes the x-ray . beam very narrow. Narrow beam = high . resolution image. Purpose...
  • Self-Study Modules on Tuberculosis, 1-5 Centers for Disease

    Self-Study Modules on Tuberculosis, 1-5 Centers for Disease

    Review slide content Mention that as the number of cases and deaths declined, many people began to hope that TB could be eliminated from the U.S., like smallpox and polio History of TB - Module 1, p. 5
  • GOLD SPONSORS - Americollect

    GOLD SPONSORS - Americollect

    Provide the Final Notice with 30 days to pay before initiating ECAs. How will your organization notify your collection agency if you approved an application and ECAs have to be reversed and a notice was sent to the responsible individual...


    Operations. Purchasing Raw Materials. Appropriate methods of production. Warehousing and distribution. Managing stock control. Quality techniques to ensure quality. ... Most manufacturing companies use a mix of labour-intensive and capital-intensive production.
  • Literal Imagery - Ms. Riley&#x27;s Class

    Literal Imagery - Ms. Riley's Class

    What is closed form poetry? Closed form poetry, (it's also referred to as fixed form), consists of poems that follow strict patterns of lines, meter, rhymes and stanzas.When writing a closed form poem, you must follow specific rules to fit...