How do you balance "must do" work (compliance, maintenance, etc.) with objective/goal oriented work?

Before jumping into the "must do" work, it's best to understand the reason for doing the work. 

Sometimes the "must do" work is warranted. For example, compliance could have high customer impact which naturally means that it'll get prioritized anyways. Sometimes the "must do" work is not impactful. For example, there might be a set of maintenance tasks that is necessary but it's highly manual and repetitive. While it needs to be done, it's more exciting for the team to reduce/eliminate this work through automation than continue doing it over and over again.

At the end of the day, there will always be some level of "must do" work. The key is to reduce the unimpactful/repetitive tasks. Once those things are eliminated, the team is often happy to take the more impactful (even if mildly annoying) "must do" work.   

The best way that I've found to manage this is to fund these out of separate envelopes.

Typically I will work with engineering to agree on the base % allocation of resources for the first set of items. Let's say it's 20%. That leaves about 70-75% of capacity for feature work (you need to leave 5-10% on a team for slack/unplanned work like incidents). Great, now you know how many FTE-equivalents you have in a given period to do features. As engineering estimates (they are estimating, right?!) effort for features, you know which of your features are going to end up above the cutline and which ones are not.

This split can be different per team (more mature products might have less KTLO/maintenance/optimization work) and it can also vary per planning period (maybe there is a lot of tech debt that absolutely needs to get addressed next cycle so the % has to be higher). It's really just a rule of thumb and a basis upon which to have a discussion about resource allocation with engineering.

Having a default allocation gives engineering both autonomy and accountability to use their time wisely. If they waste their 20% time on science projects instead of necessary maintenance activities, it will create negative outcomes eventually (the system goes down) and that waste will therefore become apparent. Even the possibility of this and the knowledge that such outcomes will land on their heads (no more "product didn't 'give us time' to take care of the system" complaints) is often enough to dissuade them from embarking on unnecessary adventures.

The balance/percentage should be agreed upon by the EM and the PM and shared with the broader org for visibility. For example, in a team of 5 engineers, one engineer's time could be set aside for 'must do' work and the rest will continue to focus on goal strategic efforts. Similar approach can be applied if the organization is big - eg. one pod assigned for maintenance work and 2 pods working on strategic efforts. 

In the end, key is alignment with the broader organization and stakeholders so right expectations are set!

The main risk is constantly deprioritizing the Compliance and maintenance work and building a tech debt at such a level that it will take forever to pay it back. 

There are two kinds of situations:

1- you do not have a vast tech debt, and you can manage compliance and maintenance work by allocating a fraction (10 to 30%, depending on the amount of work)of your capacity to this work. The remaining capacity work on development-related work.

Alternatively, whenever a developer touches a part of the code, you impose that he improves (or maintains) a part related to what he just changed.

2- You have let the tech debt grow out of control: you will have no more choice. You will need to allocate your capacity to pay the tech debt before you go "tech bankrupt." 

When the tech debt is back under control, you can apply the rules stated in 1st paragraph.