How many users do you have?  I'm setting up a new DNN instance and am looking at somewhere between 150 and 900,000 DNN users depending on the way I set it up.  If performance is going to be an issue obviously I'll be leaning towards the lesser...

I have less than 200 users. Probably about 20 really active users and 150-170 partially active.  The site is not really built around users, more toward anonymous access.  I upgraded from 5.6.8 to 7.4.2 and that is when I noticed the long delay bringing up the user account dialog to approve new users. My 5.6.8 installation was on a shared hosting provider that used shared SQL instances.  One thing I did was go through remove all unauthorized and deleted users which helped some. The delay is definitely longer when I first access a user after a restart (more than just a page recompile).  And in my local lab the CPU hits only 50 to 60%.  I know of DNN installations that handle many more users and seem to do okay, so not sure if this is an issue with having upgraded from 5.6.8, or with 7.4.2, or with Azure SQL. 

I used the User-Accounts page in Admin to delete unauthorized users and remove deleted users.  Where and how would I then delete the user folders?  And these were probably all users who were never authorized, so would they have a folder?

I have truncated the logs have the scheduler log cleared every night.  I also clear the event log each day.  I went through a whole long list of things to do to improve performance - compressed files, CDN media, removed unused modules etc.