Grafana stop log metrics

Debug found that carbon process not running

Check /var/log/supervisor/carbon-cache.log

it says "carbon-cache (instance a) is already running with pid 36"

it work by check pid in this file/opt/graphite/storage/carbon-cache-a.pid or /var/run/carbon-cache-a.pid
which content is 36. Check cat /proc/36/cmdline , found it is another process!

check the modify time of carbon-cache-a.pid, which is very old. For some reason, it is not updated or deleted properly, which prevent new process to start, and lead to deadlock.

delete carbon-cache-a.pid, restart grafana, it work again.