No-SQL Unit 1 & 2: Extra Questions

1. Explain why NoSQL databases became important for modern web-scale applicationsCopy Anchor Link

Traditional Relational Database Management Systems (RDBMS) were designed for structured data on single servers. The explosion of the internet and web-scale applications (e.g., social networks, e-commerce, IoT) exposed critical bottlenecks, making NoSQL essential due to the following factors:

• Handling Massive Data Volumes (The 3Vs): Web-scale systems generate terabytes to petabytes of data daily (Volume), at high ingestion rates (Velocity), in diverse formats like JSON, multimedia, and plain text (Variety). RDBMS cannot ingest or query this unstructured data efficiently at scale.
• Horizontal Scalability (Scale-Out): RDBMS scales vertically (buying larger, expensive servers). NoSQL natively supports horizontal scaling (sharding), allowing systems to distribute data across clusters of commodity hardware cheaply and transparently.
• Schema Flexibility (Agile Development): Traditional databases require predefined schemas; altering a table with billions of rows incurs significant downtime. NoSQL engines are schema-agnostic, letting developers add fields dynamically without schema migration delays.
• High Availability & Fault Tolerance: Web applications require 99.999% uptime. NoSQL systems deploy masterless or replica-set architectures that automatically reroute traffic and replicate data across geographic nodes during network or machine failures.
• Low Latency & High Performance: By eliminating complex multi-table JOIN operations and storing related data together (denormalization), NoSQL achieves sub-millisecond reads and writes required for real-time applications.

2. Write short notes on document databases and key-value stores with suitable examplesCopy Anchor Link

Key-Value Stores

• Concept: The simplest NoSQL data model where every data element is stored as an opaque value indexed by an explicit unique key. The database does not inspect or understand the internal structure of the value (stored as a BLOB, string, or binary).
• Operations: Highly optimized for basic GET(key), SET(key, value), and DELETE(key) lookups.
• Strengths: Maximum throughput, minimal read/write latency ($O(1)$ lookup time), and straightforward partitioning.
• Example Systems: Redis, Amazon DynamoDB, Memcached.
• Use Case: Session state management, caching, user shopping carts.

Document Databases

• Concept: An evolution of key-value stores where each record is a self-describing, structured document (typically JSON, BSON, or XML). Unlike key-value stores, the database understands and indexes the internal fields of each document.
• Strengths: Natural support for complex, nested structures (arrays, embedded objects), eliminating the need for relational joins. Supports rich query languages, secondary indexing, and partial updates.
• Example Systems: MongoDB, Couchbase, Apache CouchDB.
• Use Case: Content management systems, e-commerce product catalogs, user profiles.

3. What is the impedance mismatch problem? Explain with a simple e-commerce order exampleCopy Anchor Link

Impedance Mismatch refers to the conceptual and technical friction that arises when mapping object-oriented programming models (classes, objects, inheritance, collections) to relational database models (two-dimensional tables, foreign keys, constraints).

• Object Model: Hierarchical, navigational, and rich (objects reference lists of other objects).
• Relational Model: Relational and tabular (data is decomposed into normalized rows and columns linked by primary/foreign keys).

E-commerce Order Example:

In an application, an Order is represented as a single cohesive object:

JSON

{
  "orderId": 101,
  "customer": "Alice",
  "shippingAddress": { "city": "Bengaluru", "zip": "560001" },
  "items": [
    { "product": "Laptop", "price": 65000, "qty": 1 },
    { "product": "Mouse", "price": 1200, "qty": 2 }
  ]
}

• Relational Mapping Problem: To persist this single object into an RDBMS, the developer must normalize it across multiple tables:
  1. Orders (orderId, customerId, date)
  2. Addresses (addressId, city, zip, orderId)
  3. Order_Items (itemId, orderId, product, price, qty)
• Drawbacks:
  • Writing requires multiple INSERT operations wrapped in transactions.
  • Reading requires multiple expensive SQL JOIN queries to reconstruct the object graph in memory.
  • Requires complex Object-Relational Mapping (ORM) tools, which add performance overhead and maintenance complexity.
• NoSQL Solution: Document databases store the entire order in a single document matching the native object structure directly, resolving the mismatch.

4. Define the three components of the CAP theorem and explain why all three cannot be fully guaranteed during a network partitionCopy Anchor Link

The CAP Theorem (formulated by Eric Brewer) states that a distributed data store can simultaneously provide at most two out of three guarantees:

1. Consistency (C): Every read receives the most recent write or an error. All nodes see the exact same data at the same time.
2. Availability (A): Every non-failing node returns a non-error response for every request, though it is not guaranteed to contain the most recent write.
3. Partition Tolerance (P): The system continues to operate despite an arbitrary number of messages being dropped or delayed by the network between nodes.

Why All Three Cannot Be Guaranteed During a Partition:

In physical distributed networks, partitions (packet drops, severed lines, high latency) are inevitable ($P$ is mandatory). Suppose a network splits nodes into Group A and Group B:

[ Client 1 ] ---> [ Node A ]  <=== NETWORK CUT ===>  [ Node B ] <--- [ Client 2 ]

A write comes into Node A:

• To guarantee Consistency (C): Node A must replicate the write to Node B before acknowledging success. Because of the network partition, Node A cannot reach Node B. The system must either block or return an error to requests on Node B. Choosing consistency means sacrificing Availability (CP).
• To guarantee Availability (A): Both Node A and Node B must continue accepting reads and writes. Node A accepts the new write, but Node B serves stale data because updates cannot traverse the partition. Choosing availability means sacrificing Consistency (AP).

Therefore, distributed systems must trade off Consistency vs. Availability in the presence of a network partition.

5. Explain BASE properties and compare them with ACID propertiesCopy Anchor Link

Modern distributed NoSQL databases trade the strict transactional consistency of ACID for scalability and availability through BASE properties:

• Basically Available (BA): The system guarantees availability by serving requests even if parts of the cluster are degraded, partitioned, or down.
• Soft State (S): The state of the system may change over time without user interaction due to background data replication across nodes.
• Eventual Consistency (E): If no new updates are made, all replicas will eventually synchronize and return identical data.

6. Define a document database. How is it different from a relational database?Copy Anchor Link

A Document Database is a non-relational database management system that stores, retrieves, and manages semi-structured data in hierarchical documents (typically JSON, BSON, or XML). Each document contains self-describing field-value pairs and can vary in structure from other documents within the same collection.

7. Explain database, collection and document in MongoDB with an exampleCopy Anchor Link

MongoDB organizes data using a three-tier hierarchical structure:

• Database: The physical container for collections. A single MongoDB server instance can host multiple databases, each having its own set of files on disk.
• Collection: An grouped set of documents, equivalent to an RDBMS table. Collections do not enforce a rigid schema, meaning documents inside a single collection can have varying fields.
• Document: A single record within a collection, equivalent to an RDBMS row. Documents are stored internally as BSON (Binary JSON) and consist of key-value pairs, nested sub-documents, and arrays.

Hierarchical Example:

Database: "universityDB"
   └── Collection: "students"
         └── Document:
             {
               "_id": ObjectId("65f1a2b3c4d5e6f7a8b9c0d1"),
               "rollNo": 101,
               "name": "Arun Kumar",
               "department": "CSE",
               "cgpa": 8.75,
               "skills": ["Python", "Docker"]
             }

8. Write mongosh commands to create a database, insert one student document and read it backCopy Anchor Link

1. **Switch / Create Database:**Python

   (MongoDB creates the database implicitly upon the first write operation)

   Copy Text

   use collegeDB;

2. **Insert One Student Document:**Python

   Copy Text

   db.students.insertOne({
     rollNo: 202601,
     name: "Rahul Sharma",
     department: "Computer Science",
     semester: 6,
     email: "rahul.s@example.com"
   });

3. **Read the Document Back:**Python

   Copy Text

   // Query with projection or formatted output
   db.students.find({ rollNo: 202601 }).pretty();

9. Explain updateOne(), updateMany(), deleteOne() and deleteMany() with examplesCopy Anchor Link

.

• updateOne(filter, update, options): Updates the first document that matches the filter criteria. Python

  Copy Text

  // Increases the fee of the first matching student by 1000
  db.students.updateOne(
    { rollNo: 101 },
    { $set: { status: "Active" }, $inc: { feePaid: 1000 } }
  );

• updateMany(filter, update, options): Updates all documents matching the specified filter. Python

  Copy Text

  // Updates department name for all students in "IT"
  db.students.updateMany(
    { department: "IT" },
    { $set: { department: "Information Technology" } }
  );

• deleteOne(filter, options): Deletes the first document matching the filter criteria. Python

  Copy Text

  // Removes one student with rollNo 105
  db.students.deleteOne({ rollNo: 105 });

• deleteMany(filter, options): Removes all documents matching the criteria. Python

  Copy Text

  // Removes all inactive students
  db.students.deleteMany({ status: "Inactive" });

10. Explain embedding and referencing in MongoDB data modellingCopy Anchor Link

MongoDB provides two fundamental approaches to model relationships between data entities:

1. Embedded Documents (Denormalized Model)

• Concept: Related data is nested directly within a single document as sub-documents or arrays.
• When to use: For "contains" or 1-to-1 and bounded 1-to-Few relationships where entities are viewed and updated together.
• Advantages: High read performance (atomic single-document reads without joins).
• Disadvantage: Documents can grow towards MongoDB's 16MB document limit; data duplication.

JSON

{
  "_id": 1,
  "name": "Kavitha",
  "address": { "street": "MG Road", "city": "Bengaluru", "zip": "560001" }
}

2. References (Normalized Model)

• Concept: Data is separated into distinct collections, and relationships are maintained using references (such as _id values), similar to foreign keys.
• When to use: For 1-to-Many (unbounded) and Many-to-Many relationships, or when data is updated frequently across multiple views.
• Advantages: Avoids data redundancy; documents stay compact.
• Disadvantage: Requires multiple queries or $lookup aggregation stages (server-side joins), impacting query latency.

JSON

// Collection: users
{ "_id": 101, "name": "Kavitha" }

// Collection: orders
{ "_id": 5001, "userId": 101, "amount": 1500 }

11. Discuss the need for NoSQL databases. Explain the limitations of relational databases in large-scale distributed applications and the advantages offered by NoSQL systemsCopy Anchor Link

Limitations of RDBMS in Large-Scale Distributed Environments:

• Scalability Bottlenecks: RDBMS relies on vertical scale-up; horizontal partitioning (manual sharding) across distributed nodes breaks transactional integrity, foreign key validation, and referential constraints.
• High Latency from Joins: As datasets grow into billions of rows, multi-table JOIN operations cause severe I/O degradation and CPU spikes.
• Rigid Schemas: Modern applications continuously integrate polymorphic and unstructured data. Relational schema alterations require costly ALTER TABLE locks.
• Impedance Mismatch: Significant engineering overhead is required to serialize object graphs into tabular schemas.

Advantages Offered by NoSQL Systems:

• Distributed Native Architecture: Engineered from inception to distribute data across commodity clusters using automatic sharding and replication.
• Dynamic & Flexible Data Models: Ingests structured, semi-structured, and unstructured data without schema migrations.
• Optimized High-Throughput I/O: Denormalized data structures allow reads and writes to be satisfied with localized disk reads instead of scattered disk seeks.
• Cost Efficiency: Scales linearly on low-cost cloud virtual machines rather than expensive multi-socket bare-metal mainframes.

12. Explain the CAP theorem in detail. Use examples to show the difference between CP and AP systems. Why is partition tolerance important in distributed databases?Copy Anchor Link

The CAP theorem proves that a distributed database running across a network cannot simultaneously provide Consistency (C), Availability (A), and Partition Tolerance (P). Because physical networks inherently experience hardware faults, latency, and cable breaks, Partition Tolerance (P) is non-negotiable; distributed databases must choose between C and A during a network failure.

Difference Between CP and AP Systems:

Why Partition Tolerance is Important: In any real distributed deployment (spanning data centers or racks), switches fail and packets drop. If a system cannot tolerate partitions, a single severed link will crash the entire database. Partition tolerance ensures the database cluster survives communication failures without catastrophic failure.

13. Describe the four major types of NoSQL databases. For each type, explain its data model, strengths, limitations and suitable real-world use casesCopy Anchor Link

1. Key-Value Stores

• Data Model: Hash table mapping unique keys to arbitrary binary/string values.
• Strengths: Blazing speed ($O(1)$ lookup), low latency, horizontal scalability.
• Limitations: No secondary indexing; queries are restricted to the primary key.
• Use Case: Session management, caching layers (e.g., Redis, Memcached).

2. Document Databases

• Data Model: Semi-structured documents (JSON, BSON) with key-value pairs and arrays.
• Strengths: Schema flexibility, natural alignment with object models, deep field queries.
• Limitations: Lacks cross-collection declarative constraints; can consume excess memory due to duplicated keys.
• Use Case: E-commerce catalogs, content management platforms (e.g., MongoDB, Couchbase).

3. Column-Family (Wide-Column) Stores

• Data Model: Rows containing dynamic columns indexed by keyspace, column family, and row-key (two-dimensional key-value pairs).
• Strengths: Extreme write throughput, efficient data compression, scalable over thousands of machines.
• Limitations: Complex schema design (queries must be planned around primary clustering keys).
• Use Case: Time-series tracking, IoT sensor data, financial tickers (e.g., Apache Cassandra, HBase).

4. Graph Databases

• Data Model: Nodes (entities), Edges (relationships), and Properties (metadata) based on graph theory.
• Strengths: Fast traversal of deep and complex relationships without expensive join operations.
• Limitations: Poor horizontal scaling; not designed for heavy aggregate analytics over entire tables.
• Use Case: Fraud detection networks, social recommendation engines, identity and access graphs (e.g., Neo4j, Amazon Neptune).

14. Explain strategic database selection. What factors should be considered before choosing a database for a software project? Support your answer with examplesCopy Anchor Link

Selecting a database requires evaluating functional and operational requirements rather than defaulting to familiar technologies:

• Data Structure & Schema Predictability: Stable, highly structured data requiring strict relational integrity demands an RDBMS (e.g., PostgreSQL for accounting). Semi-structured or rapidly evolving structures fit document databases (e.g., MongoDB for product catalogs).
• Read vs. Write Ratios: Read-heavy workloads benefit from key-value caching (Redis). Massive append-heavy write streams (IoT logs) require wide-column architectures (Cassandra).
• Scalability Pattern: If the workload fits within a single server with read-replicas, an RDBMS is suitable. If linear scale across multi-region clusters is needed, horizontally sharded NoSQL is required.
• Consistency vs. Availability (CAP Needs): Financial transactions prioritize strong consistency (ACID / CP). Social feeds or analytics dashboards prioritize constant availability (AP / BASE).
• Query Complexity: Highly connected data with arbitrary link traversals calls for a Graph DB (Neo4j). Complex analytical reporting across billions of rows calls for OLAP/Columnar storage (ClickHouse, Snowflake).
• Operational Overhead & Ecosystem: Availability of skilled engineering talent, backup tooling, and managed cloud offerings (e.g., AWS RDS vs. MongoDB Atlas).

15. A start-up is building an online learning platform storing student profiles, course videos, login sessions, course recommendations, and activity logs. Which database type would you choose for student profiles and why?Copy Anchor Link

Selected Database Type: Document Database (e.g., MongoDB).

Justification for Student Profiles:

• Polymorphic and Evolving Attributes: Learners have diverse data requirements: some have enrolled degrees, others short certifications, differing notification preferences, and variable skill tags. Document databases accommodate dynamic fields without schema migrations.

• Natural Data Encapsulation: A student profile naturally forms an aggregate root. Data like addresses, academic history, enrolled courses, and enrolled dates can be embedded in a single BSON document: JSON

  Copy Text

  {
    "_id": "usr_987",
    "name": "Sita Raman",
    "enrolledCourses": ["CS101", "AI202"],
    "preferences": { "darkTheme": true, "subtitles": true },
    "achievements": [ { "badge": "Fast Learner", "year": 2026 } ]
  }

• Performance & Low Latency: Fetching an entire learner dashboard requires a single read operation by _id, bypassing multiple SQL JOINs across user, profile, preference, and role tables.

• Horizontal Scalability: Accommodates rapid learner growth through transparent cluster sharding on the user identifier.

16. Explain indexing strategies in MongoDB. Include single-field, compound, multikey, text and unique indexes with commandsCopy Anchor Link

Indexes enhance query performance by storing a small portion of the collection's data set in an ordered B-tree structure, avoiding expensive full-collection scans ($O(N)$ vs $O(\log N)$).

• Single-Field Index: Indexes a single field in ascending (1) or descending (1) order. Python

  Copy Text

  db.students.createIndex({ rollNo: 1 });

• Compound Index: Indexes multiple fields to support queries filtering on several keys. (Order of fields matters based on the Equality, Sort, Range rule). Python

  Copy Text

  db.students.createIndex({ department: 1, cgpa: -1 });

• Multikey Index: Automatically created when indexing a field that contains an array value; indexes each element of the array. Python

  Copy Text

  // skills is an array: ["Java", "Docker", "AWS"]
  db.students.createIndex({ skills: 1 });

• Text Index: Supports text search queries inside string content, providing tokenization, stemming, and stop-word elimination. Python

  Copy Text

  db.courses.createIndex({ description: "text" });

• Unique Index: Rejects duplicate values for the indexed key, enforcing entity integrity. Python

  Copy Text

  db.students.createIndex({ email: 1 }, { unique: true });

17. Describe the MongoDB aggregation framework. Explain $match,$project, $group,$sort and$limit with examplesCopy Anchor Link

The Aggregation Framework is a pipeline-based data processing model where documents pass through a sequence of multi-stage transformations to compute aggregated results.

• $match: Filters documents to pass only those matching the specified condition (acts like SQL WHERE).
• $project: Reshapes documents by selecting, omitting, or computing new fields (acts like SQL SELECT).
• $group: Groups input documents by an identifier expression and computes metrics (e.g., sum, avg) (acts like SQL GROUP BY).
• $sort: Reorders documents in the pipeline stream (acts like SQL ORDER BY).
• $limit: Restricts the number of documents passed to the next stage (acts like SQL LIMIT).

Complete Pipeline Example:

Python

db.students.aggregate([
  // 1. Filter: CSE students only
  { $match: { department: "CSE" } },

  // 2. Group: Calculate average marks and total count per semester
  { $group: {
      _id: "$semester",
      avgScore: { $avg: "$marks" },
      totalCount: { $sum: 1 }
  }},

  // 3. Project: Clean up field display
  { $project: {
      semester: "$_id",
      avgScore: { $round: ["$avgScore", 2] },
      totalCount: 1,
      _id: 0
  }},

  // 4. Sort: Highest average score first
  { $sort: { avgScore: -1 } },

  // 5. Limit: Top 3 semesters
  { $limit: 3 }
]);

18. A college wants to store student profiles, addresses, skills and semester marks. Design a MongoDB document structure and justify which fields should be embeddedCopy Anchor Link

Document Structure:

JSON

{
  "_id": ObjectId("65f42a1b9c8d1e2f3a4b5c6d"),
  "rollNo": "22CS104",
  "name": "Pooja Sundaram",
  "email": "pooja.s@college.edu",
  "phone": "+91-9876543210",
  "address": {
    "street": "24 Cross, 5th Main",
    "city": "Krishnagiri",
    "state": "Tamil Nadu",
    "pincode": "635001"
  },
  "skills": ["Python", "Python", "MongoDB", "Node.js"],
  "semesterMarks": [
    { "semester": 1, "gpa": 8.4, "credits": 22 },
    { "semester": 2, "gpa": 8.8, "credits": 24 },
    { "semester": 3, "gpa": 9.1, "credits": 20 }
  ]
}

Justification for Embedded Fields:

• Address (Embedded Sub-document): Address has a direct 1-to-1 relationship with the student. It is always queried alongside personal profile details and rarely accessed independently.
• Skills (Embedded Array of Strings): Skills represent a bounded set of lightweight strings. Embedding enables simple multi-key indexing and prevents redundant joins with an external skills table.
• Semester Marks (Embedded Array of Documents): Marks have a bounded 1-to-Few relationship (a degree has a fixed number of semesters, usually 6 to 8). Embedding eliminates joins, keeps the entire academic transcript intact, and fits comfortably within the 16MB document size limit while enabling single-read lookups.

19. Write a complete mongosh session to create a database, insert five student documents, update one record, delete one record and display all remaining recordsCopy Anchor Link

Python

// 1. Switch or create database
use universityDB;

// 2. Insert five student documents
db.students.insertMany([
  { rollNo: 101, name: "Aarav", dept: "CSE", cgpa: 8.5, status: "Active" },
  { rollNo: 102, name: "Bhavna", dept: "ECE", cgpa: 9.1, status: "Active" },
  { rollNo: 103, name: "Chetan", dept: "MECH", cgpa: 7.2, status: "Active" },
  { rollNo: 104, name: "Divya", dept: "CSE", cgpa: 8.9, status: "Active" },
  { rollNo: 105, name: "Eshwar", dept: "CIVIL", cgpa: 6.8, status: "Inactive" }
]);

// 3. Update one record (Increase CGPA for student 101)
db.students.updateOne(
  { rollNo: 101 },
  { $set: { cgpa: 8.8, status: "Honor Roll" } }
);

// 4. Delete one record (Remove student 105)
db.students.deleteOne({ rollNo: 105 });

// 5. Display all remaining records in clean format
db.students.find().pretty();

20. Collection Design and Strategy in MongoDBCopy Anchor Link

(This addresses the standalone "collection" design topic on the sheet)

• Collection Concept: A schema-free grouping of related BSON documents inside a MongoDB database.
• Collection Modeling Principles:
  • Keep documents that are accessed together in the same collection (Co-location).
  • Avoid collections with unbounded array growth; split large sets into separate collections using references.
  • Use Capped Collections for high-throughput fixed-size circular logging (e.g., db.createCollection("logs", { capped: true, size: 5242880, max: 5000 })).
  • Always enforce schema validation rules at the collection level via JSON Schema validation when data structure integrity is required.