Roadmap

Stack Feature Requests

App Feature Requests Stack Feature Requests UI/UX Feature Requests Other Feature Requests

Voters 1

  • Avatar
1

Stop Monit from Restarting MySQL

Avatar
 Jordan

Currently, when Monit sees MySQL using more than 2790MB of memory it will restart the process. This can occur multiple times over and over. Resulting in site errors.

monit should notify and not restart MySQL, especially if MySQL has been set to use more than 2GB of memory.

The following entries should be changed to notify only or removed completely. MySQL wouldn't take up more memory unless configuration changes occur. Right now the default deployment allows for 3.2GB of memory to be used.

Killing mysql seems illogical. Killing MySQL in the event the system runs out of memory is a better idea. However you'd need the appropriate logic in place.

    if mem > 2790 MB for 10 cycles then exec "/usr/local/bin/gpmonitor MYSQL MEM_HIGH warning" AND repeat every 10 cycles
    if mem > 2790 MB for 20 cycles then exec "/usr/local/bin/gpmonitor MYSQL MEM_RESTART error"

  • 16-11-2021 -
A

Activity Newest / Oldest

Avatar

Jeff Cleverley

Completely user configurable

gridpane.com/kb/configure-monit-with-gp-cli/

Just set it how you like, if you don't want it to restart MySQL, then set the thresholds so it doesn't

Also the default thresholds are set based on server resources, prior to user configuration.

Avatar

Jordan

Jeff Cleverley Thanks for taking the time to reply Jeff!

These are the issues I'm seeing. Some people don't see the UI alerts from Monit, golden retriever syndrome 100%. Some don't set up Slack notifications. The MySQL restarts then go noticed, and hours and hours of time are spent troubleshooting the issue or just going insane (the GridPane customer in this case).

You can bring a horse to water, but you can't force it to drink. Users need to check the UI and set up Slack Notifications. It's pretty much a requirement out of the gate if you want to use GridPane.

I would regard restarting a service such as MySQL as high severity. It's considered service affecting, such as a server running out of memory. The difference is one is intentional, the other isn't. If the goal is protection, then very much an alarm needs to be sounded.

A notification that goes above and beyond UI + Slack. Perhaps email alerts? I know that's a huge beast to undertake. But we're already adding Sendgrid credentials to GridPane. Are you not then able to use these to send specific monit alerts via email?

Perhaps Server Specific Warnings on the server list? Notifications will always have a chance of being missed. So maybe it's a futile effort to make this whole situation better.