AWS Summaries

  • Database Summary 
    • RDS(OLTP)
      • SQL
      • My SQL
      • Post Gre SQL
      • Oracle
      • Aurora
      • Maria DB
    • Dynamo DB(No SQL)
    • Redshift OLAP
    • Elastic cache
      • Memcached
      • Redis
    • RDS runs on virtual machines
    • We cannot log into this operating systems however
      • Patching of our RDS Operating system and DB is Amazons responsibility.
      • RDS is not server-less.
      • Aurora surverless is seven less which is an exception.
      • Dynamo DB is server-less.
    • There are two types of backups for RDS
      • Automated backups
      • Database snapshots
    • Read replicas
      • Can be multi availability zones.
      • Use to increase performance
      • Must have backups turned on.
      • Can be in different regions.
      • Can be MySQL,PostgreSQL,Maria DB, Oracle, Aurora, SQL Server is not supported.
      • Can be promoted to master, this will break read replica.
    • Multi availability zone
      • Used for DRG only and not performance.
      • You can force a fail over from one availability zone to another by re-booting the RDS instance.
    • Encryption at Rest Is supported for My SQL, Oracle, SQL server, Postgre SQL, Maria DB and Aurora. Encryption is done using the AWS key management service (KMS) service. Once your RDS instance is encrypted, the data stored at rest in the underlying storage is Encrypted, as are it’s Automated backups, read replicas, and snapshots.
    • Dynamo DB
      • Stored on SSD storage
      • Spread across three geographically distinct data centres.
      • Eventual consistent reads(default).
      • Strongly Consistent reads
    • Red shift
      • Used for business intelligence
      • Available in one availability zone
      • Backups enabled by default with one day retention period.
      • Maximum retention period is 35 days.
      • Red shift always attempts to maintain at least three copies of your data (The original and replica on the computer nodes and back up in Amazon S3).
      • Redshift Can also asynchronously Replicate your snapshots to S3 in another region for disaster recovery.
    • Aurora
      • Two copies of your data are contained in each availability zone,With minimum of three availability zone.Six copies of your data
      • You can share Aurora snapshots with other AWS accounts.
      • Three types of replicas available. Aurora replica, MySQL replica and Postgre SQL replica. Automated fail over is only available with Aurora replicas.
      • Aurora has automated backups turned on by default. You can also take snapshots with Aurora. You can share these snapshots with other AWS accounts.
      • Use Aurora server-less if you want a simple, cost-effective Option For infrequent intermittent or unpredictable workloads.
    • Elastic cache
      • Use elastic cache to increase database and web Application performance.
      • Redis is multi availability zone.
      • You can do backups and restores of Redis.
      • If you need to scale horizontally, use memcached.
    • If you want to run a data base on EC2 instance then we should use EBS storage option
      • EBS is recommended block level storage for EC2 instances if you were running a database on an EC2 instance else for non-EC2 RDS is the best choice.
    • AWS no SQL product offering is known as dynamo DB.
    • With new RDS DB instances, automated back ups are enabled by default.
    • MySQL installations default to port number 3306.
      • The default endpoint port for my SQL installations is 3306.
    • When creating a single availability Zone Amazon RDS instance, we can select the availability zone into which we deploy it.
      • When you create a DB instance you can choose an availability zone or have AWS choose one for you.An Availability zone is represented  by an AWS region code Followed by a letter identifier (for example us–east–1a).
    • Amazon dynamo DB is a non-relational database
      • It delivers reliable performance at any scale. It is fully managed, multi region, multi Master database that provides consistent single digit millisecond latency, and offers built in security, back up and restore, and In-memory cacheing.
    • To increase the number of IOPS Follow my SQL database hosted on root volume of EC2 add 2 additional EBS SSD volumes and create a RAID 0 volume to host the database.
      • RAID 0 provides performance improvements compared with the single volume as data can be read and written to multiple disks simultaneously. Two disks each with a bandwidth of 4000 IOPS Will provide a combined bandwidth of 8000 IOPS.
    • In RDS, changes to the back up window are implemented during the next scheduled maintenance window or immediately.
      • When applying changes to the backup window we can choose either to have the changes done during the next scheduled maintenance window or immediately the schedule it self is modified Right away.
    • Following dynamo DB features are chargeable when using in a single region
      • Storage of data, read and write capacity.
      • There will always be a charge for provisioning read and write capacity and the storage of data within dynamo DB.
      • There is no charge for the transfer of data into dynamo DB,Providing you to Stay Within a single region.
      • If we cross regions, we will be charged at both the ends of the transfer.
      • There is no charge for local secondary indexes.
    • If you want your application to check RDS for an error, have it look for an error node in the Response from Amazon RDS API.
      • Typically you want your application to check whether a request generated an error before you spend any time processing results. The easiest way to find out if an error occurred is to look for an error node in the response from the Amazon RDS API.
    • We cannot SSH into and control The operating system where our Amazon RDS mySQL instance is running.
      • Amazon RDS provides a managed database offering, so we can’t SSH and have control over the underlying operating system configuration where our Amazon RDS MySQL instance is running. 
      • We can only have such control when we deploy and manage our databases on EC2 instances.
    • No transfer charge is associated with action of replicating data between availability zones for your Amazon RDS mySQL in a multi-availability zone deployment.
      • Data transferred between availability zones for replication of multi availability zone deployments is free.
    • Input/output operations of a single – availability zone RDS instance during a database snapshot or back up me be briefly suspended while the Back up process initialises  (typically under a few seconds),And you may experience a brief period of elevated latency.
    • If you use online transaction processing in your production environments you should choose provisioned IOPS over standard storage when creating an RDS instance.
      • Provisioned IOPS becomes important when you are running production environments requiring rapid responses, such as those which run e-commerce websites. Without high-performance responses from an RDS instance page loads of the website could suffer resulting in loss of business.
      • If your workloads Are not latency sensitive or you Are running a test environment the additional cost of provision and IOPS will not be cost beneficial to your project.
    • Amazon dynamo DB does not supports Mongo DB workloads.
      • This is not a feature supported by Dynamo DB.
      • Amazon document DB with Mongo DB compatibility Is a fast, scalable, highly available, and fully managed document database service that supports Mongo DB workloads.
    • Amazon elastic cache uses Redis and Memcached.
    • If we are using Amazon RDS provision IOPS storage with the Microsoft SQL server database engine, the maximum RDS volume we can have by default is 16 TB.
      • We can create Amazon RDS for SQL server database instances with up to 16 TB storage.
      • The 16 TB storage limit is available when using the provisioned IOPS and general purpose (SSD) storage types.
    • DB security groups are used with DB instances that are not in VPC and on the EC2 classic Platform.When we create a DB security group, we may not need to specify a destination port number.
      • We don’t need to specify a destination port number when you create a DB security group rules.
      • The port number defined for the dB instance is used as the destination port number for all rules Defined for the DB security group.
    • Amazon Athena supports Apache Parquet, Apache ORC and JSON data formats.
      • Amazon Athena Is an interactive query Service that makes it easy to analyse data in Amazon S3, using standard SQL commands.
      • It will work with a number of data formats including Apache Parquet, Apache ORC,JSON Amongst others, but XML is not a format that is supported.
    • Red shift is most suitable for OLAP (online analytical processing) and also used for business intelligence tools/data warehousing.
    • RDS with provisioned IOPS delivers fast, predictable and consistent I/O performance that is great for OLTP (online transactional processing) database workloads.
    • The maximum value of back up retention period is 35 days for RDS.
    • Redshift AWS Service is ideal for business intelligence tools/data warehousing.
    • RDS reserved instance are available for multi-AZ deployments.
      • Reserved DB instance benefits apply for both multi-AZ and single-AZ configurations.
    • Amazon aurora automatically maintains six copies of your data across three availability zones.
    • RDS AWS database service is most suitable for OLTP (Online transactional processing workloads)
      • Amazon RDS with provision IOPS delivers fast, predictable, and consistent I/O performance that is great for OLTP (online transactional processing) Database workloads.
    • Following set of Amazon RDS database Engines is currently available
      • Amazon Aurora
      • My SQL
      • Maria DV
      • Oracle
      • SQL server
      • Post GRE SQL
  • EC2 Summary 
    • EC2 is like a virtual machine that is hosted in AWS Instead of your data center.
      • Select the capacity that you need right now.
      • Grow and shrink when you need.
      • Pay for what you use.
      • Wait minutes, not months.
    • EC2 pricing options
      • On Demand
        • Pay by the hour, or the second , depending on the type of instance, you run, great for flexibility.
      • Reserved
        • Reserve capacity for one or three years up to 72% Discount on the hourly charge. Great if you have known fixed requirements.
      • Spot
        • Purchase unused capacity at a discount of up to 90%, prices, fluctuate with supply and demand. Great for applications with flexible start and end times.
      • Dedicated
        • A physical EC2 server dedicated for your use. Great if you have servo bound licenses to reuse or compliance requirements.
    • Instance Types
      • Determines the hardware of computer
      • Each instance type offers different, Compute, memory, and storage capabilities. These types are grouped in instance, families.
      • Select an instance type based on requirement of your application.
      • Highly available and scalable storage volumes you can attach to an EC2 instance.
    • Types EBS volume(SSD Volumes)
      • gp2
        • General purpose SSD.
        • Suitable for boot disks, and General Applications up to 16,000 IOPS per volume, up to 99.9% durability.
      • gp3
        • Latest generation, general purpose SSD
        • Suitable for boot disks, and General Applications baseline for 3000 IOPS for all volumes. Up to 16,000 IOPS per volume. 
        • Up to 99.9% durability.
      • io1
        • Provides are you IOPS SSD.
        • Suitable for OLTP and latency sensitive applications 50 IOPS/GB.
        • Up to 60,000 IOPS per volume.
        • High performance, and most expensive up to 99.9% durability.
      • io2
        • Latest generation provisioned, IOPS SSD.
        • Suitable for OLTP and latency sensitive applications, 500 IOPS/GB.
        • Up to 64,000 IOPS per volume.
        • 99.999% durability.
      • io2 block express
        • Provisioned IOPS SSD.
        • For the largest, most critical, high-performance applications
        • SAP HANA, Oracle, Microsoft SQL Server and IBM DB2.
        • Up to 64 TB, 256,000 IOP per volume.
        • 99.999% durability.
    • Elastic Block Store -HDD volumes
      • st1
        • Throughput optimized HDD
        • Suitable for Bigdata, data warehouses, ETL.
        • Max throughput is 500 MB/s per volume.
        • Cannot be a boot volume
      • SC1
        • Cold HDD
        • Max throughput of 250 MB/S per volume.
        • Less frequently accessed data.
        • Cannot be a boot volume.
        • Lowest cost.
        • Up to 99.9% durability.
    • EBS snapshots
      • A point in time copy of EBS volume. Great for backing up EBS Volumes, you can use a snapshot to create. A new EBS volume.
      • Encrypted snapshots
        • If you create a new EBS volume from an encrypted snapshot, then you will get an encrypted volume.
      • Unencrypted snapshots
        • If you create a new EBS volume from an unencrypted snapshot, then you will get an unencrypted volume.
    • Elastic load balancer
      • Application Load Balancer
        • Intelligent load, balancing for HTTP, HTTPS.
        • Routes requests to a specific web server based on the request type.
      • Network Load Balancer
        • Provides high-performance load, balancing for TCP traffic.
      • Classic Load Balancer
        • The Legacy option that supports both HTTP/HTTPS and TCP.
      • Gateway load balancer
        • Provides load-balancing for third party, virtual applications.
    • X – forwarded for
      • If you need the IPV4 , address for your end-user look for the X forwarded for HTTP header.
    • 504 error 
      • Gateway timeout.
      • The application is not responding within the time out period.
      • Troubleshoot your application or database server.
    • Secrets Manager
      • Secure way of securing secrets so that they can be accessed by applications and AWS resources.
      • It’s a great place to store database credentials as well as API keys.
      • It supports Automatic rotation of your database, passwords and API keys.
    • Parameter store
      • Use cases like configuration variables that are not secrets, parameters that you define, or for license keys.
      • Does not support password rotation.
    • Memory DB for Redis
      • It is an in memory database with ultra fast performance. Microsecond, read and single digit millisecond write. Massively Scalable > 100 TB.
      • Use cases include high-performance, large scale micro service applications. Online game with millions of users, sharing digital assets.
      • Memory DB can store your whole data set in memory – no database required as compared to Elastic cache which is an in-memory cache For databases. Example RDS. Elastic cache is an add on to databases.
    • RDS proxy
      • RDS proxy pools and shares database connections to assist with application scalability and database efficiency.
      • RDS proxy is the connection in a way that our application is pointed towards RDS proxy and RDS database receives information from the application through RDS proxy.
      • RDS Proxy is server-less and scales automatically to your workload through pooling and sharing Database connections.
      • Preserves Database application connection during failover.
      • Detects Failover and routes Requests to standby quickly.
      • Deployable over multi-AZ for protection from, infrastructure failure.
      • Enables up to 66% faster failover time.
    • EC2 Image Builder
      • Automates the process of creating and maintaining AMI and Container images.
      • Process to create an EC2 image is as follows.
        • Select a base OS image.
          • Provide base OS image like Amazon, Linux AMI.
        • Customise by adding software test.
          • Define the software we want to install like (.Net, Node.js,, Python, latest security, updates, latest kernel, Security settings).
          • Run Tests On the new image for example, does it boot correctly.
        • Distribute in your chosen region. 
          • Distribute the image to the regions of your choice. By default, it is the region you are operating in.
          • To use an AMI in different region,create a copy.
          • Specify Destination region encryption status.
            • We can apply encryption during the copying process.
            • We cannot remove encryption during the copying process.
  • Route 53
    • It is Amazons DNS service
    • Maps a domain name to
      • EC2 instance
      • Elastic load balancer
      • S3 bucket
    • Terminology
        • Hosted zone
        • A container for DNS records for your domain.
      • Alias
        • Allows you to route traffic addressed to the zone apex, or the top off the DNS namespace, and send it to a resource within AWS example an elastic load balancer.
      • A record
        • Allows you to route traffic to a resource, such as a web server, using IPV4 address.
  • CLI
    • Least privilege
      • Always give your users the minimum amount of access required to do their job.
    • Use groups
      • Create IAM Groups and assign your users to groups.
      • Group permissions are assigned using IAM Policy documents.
      • You are users will automatically inherit the permissions of the group.
    • Secret access key
      • You will see this key only once.
      • If you lose it, then you can delete the access, key ID and secret access key and regenerate them.
      • You will need to run AWS configure again.
      • Don’t share your key pairs 
      • Each developer should have their own access, key ID and secret access key just like passwords, they should not be shared.
    • Supports Linux, windows, macOS
      • You can install CLI on your Mac Linux or Windows PC. You can also use it on EC2 instances.
  • Roles
    • Roles can be used to give EC2 instances access to AWS resources like S3.
      • We can create an IAM role with S3 access.
      • Create an EC2 instance and attach the whole, we just created.
      • We can access S3 from our EC2 instance.
    • Roles are preferred from a security perspective.
    • Avoid hardcoding your credentials.
    • Roles allow you to provide access without having to manage access key IDs and secret access keys.
  • Policies
    • Policies control a role permissions.
    • Updates
      • You can update a policy attached to a role and it will take immediate effect.
    • Attaching and detaching
      • You can attach and detach roles to running EC2 instances without having to stop or terminate these instances.
  • RDS
    • RDS database types
      • SQL server, Oracle , mySQL, Postgres SQL, Maria DB AND Amazon Aurora.
    • RDS for OLTP workloads
      • Great for processing lots of small transactions like customer orders, banking transactions, payments, and booking systems.
    • Not suitable for OLAP
      • Use red shift for OLAP and data warehousing tasks like analyzing large amounts of data, reporting, and sales forecasting.
    • RDS encryption
      • Enable at creation
        • Includes all underlying storage, automated backups, snapshots, logs, and read replicas.
      • KMS integration
        • Uses the AWS Key Management Service (KMS) service for AES – 256 bit encryption.
      • Existing RDS Instances
        • Take a snapshot, encrypt the snapshot, then create a new RDS Instance from the encrypted snapshot.
    • Automate Backup DB Snapshot
      • Automated, enabled by default, you define the Backup window.
      • Point in time snapshot plus transaction logs.
      • Retention period of up to 35 days.
      • Can be used to recover your database to any point in time within the retention period.
      • User initiated, ad hoc.
      • Point in time snapshot only.
      • No retention period, Stored indefinitely.
      • Used to back up your DB instance to a known state and restore to that specific state at anytime example before making a change to database.
      Multi AZ Read Replica
      • An exact copy of Your production database in another availability zone.
      • Used for disaster recovery.
      • In the event of a failure, RDS will automatically fail over to the standby instance.
      • A read-only copy of your primary database in the same availability zone, cross availability zone or cross region.
      • Used to increase or scale read performance.
      • Great for read heavy, work loads and takes the load of your primary database for read-only workloads. example business intelligence reporting jobs.
  • Elastic Cache
    • Elastic cache is in in memory cache designed to improve read performance for read heavy Databases.
    • Memcached Redis
      • In memory key value, data store.
      • object cacheing is your primary goal.
      • you want to keep things as simple as possible.
      • You don’t need persistence or multi availability zone.
      • You don’t need to support advanced datatypes or sorting.
      • In memory, key value, data store.
      • You are performing data, sorting and ranking, such as gaming leaderboards.
      • You have advanced Datatypes, such as lists and hashes.
      • You need data persistence.
      • You need multi availability zone.

  • Parameter Store
    • Store confidential information
      • Password, database, connections, strings, and license codes.
    • Plain text, or encrypted
      • You can store Values as plain text, or encrypt them.
    • Reference
      • You can reference your parameters using the parameter name example in a Bootstrap script.
    • Integrated with AWS services
      • You can use parameters store with EC2 , cloud, formation, lambda, code build, code pipeline and code deploy.
  • Lambda Ephemeral vs Permanent storage
    • Native with lambda
      • /tmp (Ephemeral Storage)
        • Used for storing temporary data.
        • Store up to 512MB – 10 GB
        • Dynamic read/write
        • Shared within execution environment
      • Lambda layer
        • Used for Libraries and SDK’s.
        • Size limit is 50 MB zipped and 250 MB unzipped.
        • Updates require a new layer
        • Shared across execution environments to store and retrieve data.
    • External storage options
      • S3
        • Used for persistent data.
        • Size limit is elastic.
        • Stole and retrieve for dynamic update.
        • S3 can be shared across execution environments.
      • EFS
        • Used for persistent data
        • Elastic size limit
        • Dynamic read/write by opening file and append to it.
        • Shared across execution environments.
  • Lambda Environment Variables
    • Adjust your function behaviour, without changing your code.
      • Key : Env , Value : Development 
      • Key : DB , Value : mydevdb
    • Configureable Parameters
      • Allow you to adjust various settings that apply to your function.
      • Located in the configuration tab
    • Configure capacity
      • Memory , Ephemeral storage, concurrency
    • Connectivity
    • Connect to other AWS SERVICES like CLOUD WATCH AND X-RAY FOR MONITORING VPC’S and EFS file system.
  • Handling lambda event, life cycle and errors
    • When the function returns an error in lambda It can be handled as follows
      • Retries
        • If a function returns an error, Lambda automatically performs two retries.
      • Dead Letter Queues
        • Save field invocations for further processing SQS or SNS are Supported.Use SNS to fan out to multiple destinations.
      • Lambda destination
        • Configure one destination for sending Successful invocation records and other unsuccessful invocations.
        • Example SQS, SNS, Lambda and Event Bridge.

Comments

Popular posts from this blog

Effect : Deny vs No Action

Infrastructure Setup using Cloud Formation Templates