GCP Classroom notes 15/Sep/2024

Options to move data into Google storage

• Use cases:
  • Datacenter migration
  • Backup and Archival
  • Machine learning
• Options
  • GCS Transfer Tools:
    • For small transfers upto a few TBs
    • tools: gsutil, ui, json apis
  • Transfer Service
    • Large scale online Transfer
    • On-premises Agents with GUI
    • Refer Here for agent pools
  • Transfer Appliance
    • Large scale Transfer from low bandwidth networks
    • Offline transfer where gcp will send an appliance
    • Refer Here for appliance
  • Big Query Transfer Service
    • Data from SaaS Applications or third party apps

Cloud Storage FUSE

• Filesystem in Userspace is a framework for exposing the file system to the Linux kernel
• Cloud Storage Fuse is an Opensource adapter that allows users to mount Cloud Storage buckets as filesystems on Linux and macOS platforms Refer Here

Databases

• Types of Databases:
  • Relational Databases:
    • orgnize structured data fields into columns and rows
    • we have SQL to query the data
    • Used when
      • Data doesn’t change very often
      • Accuracy is crucial
      • Transactional usecases
      • general purpose
    • Priniples:
      • ACID (Atomicity, Consistency, Isolation and Durability)
    • Relational database can be scaled vertically, if multiple servers are needed we perform replications
    • Services:
      • Cloud SQL
      • Cloud Spanner
  • Non Relational Databases
    • Organize unstructured data into any of the below forms/types
    • Types:
      • key value: large amounts of data with simple lookup queries
      • Document: General purpose
      • Graphs
      • In memory: caching
      • Wide column: large datasets
    • Principles:
      • BASE:
        • BA => Basically Available
        • S => Soft State
        • E => Eventually consistent
      • These databases are designed to scale horizontally
    • Used when:
      • Frequent changes in data
      • scale and availbility is more crucial
      • Analytics use cases
    • Services:
      • Memorystore
      • Firestore
      • Cloud Bigtable

Databases on Cloud

• Managed
• Scalable
• Easy to access
• Disaster Recovery
• Secure

Cloud SQL

• Cloud SQL offers mysql, postgres and SQL server
• For connecting the databases use azure data studio Refer Here
• For how to create databases refer class room video