Many CIOs have an opinion, that having an application on cloud with auto scaling, will have the best performance. This is absolutely incorrect. The app and infra teams are not telling this on the CIO's face directly. When we go to consulting to solve the performance issues, the CIOs keep telling that everything is in place and still the users complain about slowness.

Auto scaling is usually done to dynamically create more cloud machines (instances), usually web/app servers, behind a load balancer. The problem here is that the auto scale is configured based on some threshold levels of CPU or memory or connections etc. The auto scale is not set based on a number, if the app's response time is above some threshold.

Imagine this situation. Initially there is only 1 web server that caters to users. Its auto scale is configured to spawn another instance, if cpu usage exceeds 80%. But even when the cpu is at 70%, users may start feeling the slowness. If the number of users stay at this point for 1 hour, then for that 1 hour, all users will feel the slowness. A new instance will get created only when the cpu of this server crosses 80%. Till then your users have to live with slowness.

To understand, at what user level, another instance to be created, we must do load testing before configuring the same. When you ramp up users, during the load test, say from 1 user to 1000 users, you will be in a better position when the response time slows down. Based on this you can set the sessions/connections as part of auto scaling configuration.

Even if auto scale is set, if there are any dependencies on a single resource such as monolithic database (instead of distributed database), with multiple auto scaled machines, your app will still be slow. Because, multiple instances will try to consume the data from a single machine that will be bottleneck.

Thus, you must do rigorous performance testing to see if auto scaling really works for your app, with your architecture. Just moving the app to cloud and setting auto scale, will not solve your performance problems.