Tuesday, 5 March 2024

DNS could be the culprit in slower downloads

 Hi,

       Thought of sharing an interesting observation, well at least it's intriguing for me. We were having a slower download issue in our application, the file/object is stored in s3 bucket, the storage class for this specific object is standard. The download of the object don't happen directly from s3, instead it's been authenticated by an internal service via mutual auth(SSL certificates). The internal service that does this authentication and responsible for getting the file from s3 and transferring to the user desktop app is hosted in EC2 instance of type t3.medium. The network bandwidth of t3.medium is 5GBps burstable. Since this file size is below 200 MB this should be more than enough to download the file under 2 minutes, but we were getting the download time between 10 - 15 minutes. Then we realized there is DNS involved in this workflow, the user desktop app uses corporate DNS to resolve the internal service hostname. So we tried to reduce this lookup time or to avoid DNS cache, we included the internal service which is fronted by the Network load balancer public ip addresses to the hosts file entries of the server where user desktop app is installed. Now when we did the download it was downloaded under 2 mins. So we should always check if DNS is involved in any workflow mostly it will be, and factor it in to our debugging process.