Performance work usually doesn't get done. Why?
|
|
It seems like performance work just never gets its "proper" due. Why is it that we always feel like we're behind, deep in the backlog on making our websites faster?
It's certainly not developer motivation. Every developer I've ever met is extremely eager to work on performance issues. I get it: it's extremely fun. There's a reason I've made it my entire career. But good engineers are also internally motivated to make something, but also make it good. Consciously or not, most developers see the systems they work on as extensions of themselves, which means that they want that system to be clean, organized, and performant.
So if the developers want to work on this stuff, why isn't it getting done?
Move fast and don't fix things
Company culture is usually a good place to start. Most startups and tech companies have cultures highly biased to shipping, that is, launching and getting things out the door with big blog posts. That's what gets the product managers promoted. There's a reason for this: launches correlate to growth, which is what every business must do, and VC-funded ones must do at a rapid pace.
The cobra effect comes into play, of course. If PMs get promoted for shipping but not for fixing old products and improving them, then no one improves old products.
As we all know from shipping new greenfield projects, it's relatively easy to make a new product performant. When there's very little data in the system, the feature set is light, it's easy to make the site feel nice and zippy. But 5 years down the road, when the feature list has gotten three times longer and there's thousands more commits made, it gets much much harder. So, we ship fast products, and they slowly deteriorate into slow ones.
So the first fix is simple but not easy: make sure maintenance and continuous improvement is getting the right due in your organization. In subscription businesses, retention is as good as growth, and retention can certainly be damaged by a poor product experience.
It's also sometimes possible to tie performance into growth. For direct-to-consumer businesses, there's lots of research and case studies on the effect of site performance on conversion rates. For subscription businesses, site performance can help land bigger customers with more data.
Quantify and present the tech debt bill
In business, debt isn't a bad thing. It's a lever. If you've got a business with a high rate of return on equity, it usually makes sense to borrow a lot so you can invest even more into the business. If you've got a machine that makes 10% on your money every year, and someone is offering you money for 2% a year, then you take that money.
I actually love the tech debt analogy because debt is both useful and dangerous.
- Bad interest rates. There's low-interest loans, and then there's credit card debt and loan sharks. Not all tech debt is created equal. A performance issue on an endpoint that's hit very rarely, or only for certain types of users, is not nearly as serious as the main dashboard of the site being slow. Borrow where interest rates are low, not where they're high. To prevent this, accumulate tech debt in places where the impact is low and in less-used corners of the app.
- Too much debt. At some point, no matter the interest rate, you accumulate so much debt that the payments just on servicing that debt become a huge amount of your total revenue. If you've ever worked somewhere that's constantly firefighting the database going down, customers complaining about pages not loading, etc, you know what this is like! Keep track of how much time you spend firefighting, doing workarounds, and doing repetitive work. Don't let that get above ~10% of your workload.
- Not knowing how much debt you have. This is by far the most common problem I see. Most organizations treat tech debt like a 16 year old treats a credit card: like a magic money machine that will never come due. How much debt do I have? No idea, don't care, that's a problem for future me. Non-technical businesspeople aren't qualified to quantify tech debt: that's your job.
When it comes to tech debt and performance, presenting performance data to PMs and management is usually the missing step. Developers look at dashboards, but fail to make the case that a performance problem exists.
To do this effectively requires top-down performance instrumentation across the stack:
- A real-user-monitoring tool, like Raygun or Datadog RUM, that reports browser performance.
- An application performance monitor, like New Relic or Scout, that reports backend performance.
- Instrumentation on your other backend services, like the database.
If you can't walk to your PM and say "site load times are X, which is two times higher than Y", then is it any wonder nothing is getting done?
Until next week,
-Nate
|
|
|
|