Posts

Ephemeral vs Persistent Data Storage Patterns

Storage patterns available with lambda Ephemeral Storage Patterns Persistent Storage Patterns Lambda is stateless Functions are stateless, meaning that you can’t permanently store any data in the function(Example, session data, customer data, etc). Lambda is Ephemeral Not used for applications that need to run for longer than 15 minutes(Example, a database application or a web server That needs to stay up and running). Persisting data To persist data, The function must interact with a data store(Example save it to S3, EFS, Or Dynamo DB).

Different between role and permissions

Permissions are permanent and role is temporary. Admin creates “updateDataBucket” role Which allows access to read/write in data bucket in S3. Admin grants members of the group permissions to assume the “updateDataBucket” role.

Effect : Deny vs No Action

No action allows us to provide granular level control over resources operations. For example, if we use effect, deny For all actions in IAM An, then we use effect allow over some actions in IAM then deniable supersede allow. The way to overcome above issue is to use NoAction on all IAM operations And action on Allowed IAM operations. Example { "Version" : "2012-10-17" , "Statement" : [ { "Effect" : "Allow" , "NotAction" : [ "iam:*" , "organizations:*" , "account:*" ], "Resource" : "*" }, { "Effect" : "Allow" , "Action" : [ "iam:CreateServiceLinkedRole" , "iam:DeleteServiceLinkedRole" , "iam:ListRoles" , "organizations:DescribeOrganization" , "account:ListRegions" , ...

Infrastructure Setup using Cloud Formation Templates

Services Required to create CICD Code Build Build code and deploy image in jfrog Code Pipeline Has (Continuous Integration Continuous Deployment)CICD flow Cloud Formation To deploy and create infrastructure we use yml files. S3 Store our data packages and yml files which are used by CFT. ECS To deploy our application on fargate servers We can also create our manual EC2 instances from pipeline. Secrets Manager Manages all the secrets. ELB/ALB Elastic/Application Load Balancer To map context paths of different services GitHub Stores our code Jfrog Used to store all the docker images Dockerfile Used to create a customised docker image Scanning for Vulnerabilities free code Sonarcube Checkmarx Twistlock Cloudwatch Used to store logs We create log group for each of instances Lambda Functions Used for serverless technologies. Creating a new Environment Create the YML images of the different resources required in the environment. The CFT templates are stored in S3 or can be uploaded from syst...

High Level Diagrams(HLD's)

Image
High Level Diagram for Building a Serverless application using Step Functions, API Gateway, Lambda, and S3 in AWS HLD for Building and Troubleshooting a Serverless Web

AWS Summaries

Database Summary  RDS(OLTP) SQL My SQL Post Gre SQL Oracle Aurora Maria DB Dynamo DB(No SQL) Redshift OLAP Elastic cache Memcached Redis RDS runs on virtual machines We cannot log into this operating systems however Patching of our RDS Operating system and DB is Amazons responsibility. RDS is not server-less. Aurora surverless is seven less which is an exception. Dynamo DB is server-less. There are two types of backups for RDS Automated backups Database snapshots Read replicas Can be multi availability zones. Use to increase performance Must have backups turned on. Can be in different regions. Can be MySQL,PostgreSQL,Maria DB, Oracle, Aurora, SQL Server is not supported. Can be promoted to master, this will break read replica. Multi availability zone Used for DRG only and not performance. You can force a fail over from one availability zone to another by re-booting the RDS instance. Encryption at Rest Is supported for My SQL, Oracle, SQL server, Postgre SQL, Maria DB and Aurora. En...

Exam Tips

AWS CLI Least Privilege Always give your users, the minimum amount of access required to do the job. Use Groups Create IAM policy groups and assign your users to the groups. Group permissions are assigned using IAM policy documents. Secret access key You only see this ones. If you lose it you can delete the access key and secret access key and regenerate them. You need to run AWS configure again. Don’t share key pairs Each developer should have their own access Key ID and secret access Key. Just like passwords, they should not be shared. Supports Linux, Windows, macOS You can install CLI on your Mac, Linux, or Windows PC. You can also use it on EC2 instances. AWS CLI Pagination If you see errors like “ timed out”, or errors related to too many results being returned. Adjust the pagination of CLI results to avoid errors generated by too many results. aws S3 api list-objects - - bucket my-bucket  - - page-size 100. The CLI still retrieves the full list, but performs a large number of...

Global Infrastructure

What is VPC? It is a virtual Network our data center dedicated to you are AWS account. One can launch EC2 Instances into their VPC. VPC is the abbreviation of Virtual Private Cloud. There can be multiple availability zones in a VPC. A private cloud is a cloud architecture owned by a company where a company can create its own data center, storage, network devices, which is accessible only to the authorized premises of that company. A virtual private cloud is a virtual network that closely resembles a traditional networking that you operate in your own data center, with the benefits of using the scalable infrastructure. It is logically isolated from other virtual net work in the AWS cloud. Maximum of five VPC can be created in a region and 200 subnets can be created in one VPC. We can create 200 routing tables in a VPC corresponding to VPC. We can allocate five elastic IP in an account. We can extend this limit after requesting Amazon. Once we create a VPC DHCP, NACL and security group w...

Queues

  Q what is the difference between SNS & SQS & Event Bridge? Abbreviation SNS stands for simple notification service. SQS stands for simple Queue service. Usage SNS uses a Publisher Subscriber system, we own a topic and we publish to that topic and subscribers get notified of events that are delivered to that topic. One to many fan out. High throughput. Many subscribers. SQS is a queuing service for message processing. SQS can be a subscriber to SNS. When somebody publishes a message to SNS our SQS will get a message that may be processed at a later time. Allows application owner to publish messages to a queue and be decouple application from one another. One of the oldest service from AWS. SQS has a synchronous communication. Temporary message holding pool. Ordered message processing. Event Bridge is an improved version of SNS. Event Bridge provides third-party integration, such as Shopify, Pager duty, data docs, etc. One too many. AWS, SQS, third party application integra...

Serverless

Q Is Fargate container or serverless? Ans: It is both. Q What is the advantage of using ECS and Fargate? Ans:  The benefits in ECS are as follows A managed service for containerized applications that does not require much user input to run docker applications. Micro services consisting of multiple applications run in isolated containers. Auto scales tasks based on application Load. Integrates with other AWS services including IAM, Cloud watch, elastic load-balancing, cloud formation templates, EBS volumes, Batch, ECR, and cloud tail logs. A Virtual private cloud(VPC) with no resources shared with other users. Provides support for running a code pipeline with ECS as the deployment platform. Supports the latest docker version. Fargate provides the following additional benefits: With a Fargate launch type, a user does not create or manage any EC2 instances, as none are exposed on the cluster. Tasks are directly exposed to user via an ENI. The underlying infrastructure is provisioned b...

General Questions

What is a X forwarded for header? Identify the originating IP address of a client through a load balancer. If you need the IPV four address of your end, user, look for the X forwarded for header. What is an OSI model A conceptual framework which describes the functions of a network. Beginning with the Application layer which directly serves the end, user, down to physical layer. What the end user sees HTTP, web browsers. Different layers of the model are as follows. Presentation layer Data is in a usable format, encryption, SSH. Session Layer Maintains connections and sessions. Transport layer Transmits data using TCP and UDP. Network layer Logically routs packets based on IP address. Data Link layer Physically transmits data based on MAC addresses. Physical layer Transmits bits and bytes over physical devices. What is the difference between IOPS and throughput? IOPS Measures the number of read and write operations per second. Important metrics for Quick transactions, low latency apps,...

Scenarios based Questions

Your company has built-in internal scrum tool for Running  all your scrum ceremonies. Usages predictably high between 9-10AM Monday-Friday and also 1 PM-2 PM Thursday and Friday. Which feature of autoscaling will easily repair your system To handle the Load? Target tracking could to work but we need to invest time in determining correct metric to track example (CPU, memory, load balancer requests). Also Manuel Scaling requires that someone changes configuration to scale up and scale down every day.  Finally over provisioning in order to cope with peak demand defeats the purpose of Elastic Scaling of Our compute. For situations where your traffic is very predictable, the easiest way to scale with demand is to create scheduled Scaling Actions. A product manager vox into your office and advises that this simple single node MySQL RDS Instance that has been used for a pilot needs to be upgraded for production. She also advises that they may need to alter the size of The instance on...