Description: Helpful infrastructure software tips and tools
At camelcamelcamel we have been tracking price points of Amazon products for the past 15 years. All this data was saved into one big MySQL table that is now over 1TB in size. We decided to move it to DynamoDB to save on costs, get better performance, and reduce maintenance complexity. Composite keys where the sort key is the timestamp fit perfectly for our use case. All our queries consistently finish in less than 20ms. One of the slowest queries we had on MySQL was querying a specific product by date. In
The move from MySQL to DynamoDB took a couple of months and we have learned a few important lessons along the way. See below for a summary of our lessons.
At camelcamelcamel we use CDK to deploy our infrastructure. We have a bunch of auto-scaling groups (ASG) behind Application Load Balancer (ALB). We recently started noticing an issue where a deployment that results in ASG instances being refreshed would cause a lot of 502 errors for a few minutes during the deployment. This surprised us as we already use health checks, rolling updates and signals as recommended. To get the bottom of it, we built a timeline of the events for one specific group: