This is absolutely a problem that could be related to your ISP. If you're sure everything else is okay, give them a call. Since you're using ADSL, ask them for a line test. Also make sure you have all the filters installed on your phone lines to cut down on interference.
Let us know how it turns out!