The issue was pretty mundane, the server script is very simple - so I assumed it wouldn't need to have a lot of monitoring overhead to deal with traffic spikes.
Even though it's a very simple script the volume of traffic used up all the memory on the machine leading to server death, then my restart script turned out to be wrong :(
Check out cluster for node.js. It will automatically create worker processes and restart them as needed. The use upstart, monit, etc to keep the master node process running.
Even though it's a very simple script the volume of traffic used up all the memory on the machine leading to server death, then my restart script turned out to be wrong :(