@mark-hollis, last night I finally found the root cause and I'll be working on fixing it today. Basically, AWS has been throttling my server when it gets too busy. That's why it's intermittently slow. I'm going to upgrade to a different type of server today.
EDIT: It's done now - AWS won't throttle the server anymore. Thanks again for reporting the problem. If you see slowness again, please let me know, but I think I got it this time.