NTP is purely a service without a demand. That means, if it goes down, nothing happen.
Unlike DNS which store information about name to IP translation, all NTP does is to keep "true time". If a server that your NTP server reference to goes down, it will just try another server or it will just guestimate (with past references stored within its past cache) and will keep time until your server can reference to parent NTP server.
When YOUR NTP server goes down, all your clients will do is keep their last current reference time intact and will update the clock when your NTP server comes back ontime.
IF you run time sensitive applications (which is very rare), then you are to have fail safe way of keeping your NTP server up such as fail-over.