Posts

Infrastructure Setup using Cloud Formation Templates

Services Required to create CICD Code Build Build code and deploy image in jfrog Code Pipeline Has (Continuous Integration Continuous Deployment)CICD flow Cloud Formation To deploy and create infrastructure we use yml files. S3 Store our data packages and yml files which are used by CFT. ECS To deploy our application on fargate servers We can also create our manual EC2 instances from pipeline. Secrets Manager Manages all the secrets. ELB/ALB Elastic/Application Load Balancer To map context paths of different services GitHub Stores our code Jfrog Used to store all the docker images Dockerfile Used to create a customised docker image Scanning for Vulnerabilities free code Sonarcube Checkmarx Twistlock Cloudwatch Used to store logs We create log group for each of instances Lambda Functions Used for serverless technologies. Creating a new Environment Create the YML images of the different resources required in the environment. The CFT templates are stored in S3 or can be uploaded from syst

High Level Diagrams(HLD's)

Image
High Level Diagram for Building a Serverless application using Step Functions, API Gateway, Lambda, and S3 in AWS HLD for Building and Troubleshooting a Serverless Web

AWS Summaries

Database Summary  RDS(OLTP) SQL My SQL Post Gre SQL Oracle Aurora Maria DB Dynamo DB(No SQL) Redshift OLAP Elastic cache Memcached Redis RDS runs on virtual machines We cannot log into this operating systems however Patching of our RDS Operating system and DB is Amazons responsibility. RDS is not server-less. Aurora surverless is seven less which is an exception. Dynamo DB is server-less. There are two types of backups for RDS Automated backups Database snapshots Read replicas Can be multi availability zones. Use to increase performance Must have backups turned on. Can be in different regions. Can be MySQL,PostgreSQL,Maria DB, Oracle, Aurora, SQL Server is not supported. Can be promoted to master, this will break read replica. Multi availability zone Used for DRG only and not performance. You can force a fail over from one availability zone to another by re-booting the RDS instance. Encryption at Rest Is supported for My SQL, Oracle, SQL server, Postgre SQL, Maria DB and Aurora. Encryp

Exam Tips

AWS CLI Least Privilege Always give your users, the minimum amount of access required to do the job. Use Groups Create IAM policy groups and assign your users to the groups. Group permissions are assigned using IAM policy documents. Secret access key You only see this ones. If you lose it you can delete the access key and secret access key and regenerate them. You need to run AWS configure again. Don’t share key pairs Each developer should have their own access Key ID and secret access Key. Just like passwords, they should not be shared. Supports Linux, Windows, macOS You can install CLI on your Mac, Linux, or Windows PC. You can also use it on EC2 instances. AWS CLI Pagination If you see errors like “ timed out”, or errors related to too many results being returned. Adjust the pagination of CLI results to avoid errors generated by too many results. aws S3 api list-objects - - bucket my-bucket  - - page-size 100. The CLI still retrieves the full list, but performs a large number of API

Global Infrastructure

What is VPC? It is a virtual Network our data center dedicated to you are AWS account. One can launch EC2 Instances into their VPC. VPC is the abbreviation of Virtual Private Cloud. There can be multiple availability zones in a VPC. A private cloud is a cloud architecture owned by a company where a company can create its own data center, storage, network devices, which is accessible only to the authorized premises of that company. A virtual private cloud is a virtual network that closely resembles a traditional networking that you operate in your own data center, with the benefits of using the scalable infrastructure. It is logically isolated from other virtual net work in the AWS cloud. Maximum of five VPC can be created in a region and 200 subnets can be created in one VPC. We can create 200 routing tables in a VPC corresponding to VPC. We can allocate five elastic IP in an account. We can extend this limit after requesting Amazon. Once we create a VPC DHCP, NACL and security group w

Queues

  Q what is the difference between SNS & SQS & Event Bridge? Abbreviation SNS stands for simple notification service. SQS stands for simple Queue service. Usage SNS uses a Publisher Subscriber system, we own a topic and we publish to that topic and subscribers get notified of events that are delivered to that topic. One to many fan out. High throughput. Many subscribers. SQS is a queuing service for message processing. SQS can be a subscriber to SNS. When somebody publishes a message to SNS our SQS will get a message that may be processed at a later time. Allows application owner to publish messages to a queue and be decouple application from one another. One of the oldest service from AWS. SQS has a synchronous communication. Temporary message holding pool. Ordered message processing. Event Bridge is an improved version of SNS. Event Bridge provides third-party integration, such as Shopify, Pager duty, data docs, etc. One too many. AWS, SQS, third party application integration

Serverless

Q Is Fargate container or serverless? Ans: It is both. Q What is the advantage of using ECS and Fargate? Ans:  The benefits in ECS are as follows A managed service for containerized applications that does not require much user input to run docker applications. Micro services consisting of multiple applications run in isolated containers. Auto scales tasks based on application Load. Integrates with other AWS services including IAM, Cloud watch, elastic load-balancing, cloud formation templates, EBS volumes, Batch, ECR, and cloud tail logs. A Virtual private cloud(VPC) with no resources shared with other users. Provides support for running a code pipeline with ECS as the deployment platform. Supports the latest docker version. Fargate provides the following additional benefits: With a Fargate launch type, a user does not create or manage any EC2 instances, as none are exposed on the cluster. Tasks are directly exposed to user via an ENI. The underlying infrastructure is provisioned by Fa