DNS could be the culprit in slower downloads

 Hi,

       Thought of sharing an interesting observation, well at least it's intriguing for me. We were having a slower download issue in our application, the file/object is stored in s3 bucket, the storage class for this specific object is standard. The download of the object don't happen directly from s3, instead it's been authenticated by an internal service via mutual auth(SSL certificates). The internal service that does this authentication and responsible for getting the file from s3 and transferring to the user desktop app is hosted in EC2 instance of type t3.medium. The network bandwidth of t3.medium is 5GBps burstable. Since this file size is below 200 MB this should be more than enough to download the file under 2 minutes, but we were getting the download time between 10 - 15 minutes. Then we realized there is DNS involved in this workflow, the user desktop app uses corporate DNS to resolve the internal service hostname. So we tried to reduce this lookup time or to avoid DNS cache, we included the internal service which is fronted by the Network load balancer public ip addresses to the hosts file entries of the server where user desktop app is installed. Now when we did the download it was downloaded under 2 mins. So we should always check if DNS is involved in any workflow mostly it will be, and factor it in to our debugging process.

How To Create RAID 5 Software Array in Linux

 In this post let's create a RAID 5 array in a linux server

As you can see below, the server has 5 devices attached to it , one is root device we will use the other devices to create the RAID 5 array. 

 

 

We are creating raid5 array with 3 devices and one spare devices 

 

Confirm if the raid5 array is complete

 

 You can also run lsblk command to see all the four devices are raid members,

 

 We need to create the mdadm.conf file to store the raid configuration of the RAID5 array we created, as linux don't automatically remember this configuration, this configuration file will help in rebuilding or reactivating the raid array.

 

We will create ext4 filesystem on the raid device /dev/md0 and mount it to start using for File I/O operations

 

 

In above image you can see that the raid device /dev/md0 is of size 20G , but we used 3 devices of 10 GB each to create the array , the reason why it's 20G instead of 30G is that one disk space is used for storing the parity information. 

Now that the raid setup is complete, let's fail a device and see how the spare device replaces it to understand the redundancy it provides, in the below image we are failing the device /dev/xvdc and once the device fails the spare device /dev/xvde replaces it and starts rebuilding. 

 

Please note Raid 5 is not recommended for Amazon EBS, refer https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/raid-config.html