Re: NOAA Space Weather Prediction Center issued a Severe (G4) Geomagnetic Storm Watch

[Mel Beckman <mel () beckman org>]

They are identical units running the same firmware versions. The TM 1000A doesn’t give you an option for any kind of 
backup syncing to NTP on the network. They are pure GPS servers; theoretically they either serve the correct time or no 
time at all. So technically to give such as our time deviation would be a bug of some kind.

Unfortunately, I don’t have the actual packets received, so I can’t decode them to see if this was just bad 
interpretation on the part of PRTG. Needless to say we’re watching these closely, plus all of our other GPS NTP 
servers. I am dumping PCAP from the corresponding firewalls to a database, since the volume of NTP traffic is low.

 -mel

On May 10, 2024, at 3:09 PM, Raymond Burkholder <ray () oneunified net> wrote:

 We had some Meinberg's which did something similar but different some time ago.  NTP was out of sync with GPS.  We 
had a CheckMk instance which detected drift between sources in our network.  Turns out there was one or more configs in 
the Meinberg that enable failover from one source to another, and to ensure GPS and NTP are working together rather 
than independently.

Maybe your TimeMachines have similar config variabilities.

On 2024-05-10 14:29, Mel Beckman wrote:


We just had two TM1000 TimeMachine brand GPS NTP servers lose clock sync at the same time, in two different cities (LA 
and Santa Barbara). The  outage lasted about five minutes, during which the NTP servers were responding, but with time 
that was 1900 seconds out of sync. The devices showed satellite lock on 8 birds (not all the same ones). I've never 
seen this behavior before with years of NTP clock experience.

It could be that these inexpensive NTP servers aren't very selective about bogus inputs, as I would have expected them 
to lose synch in the event of a GPS signal failure. Instead they produced garbage. Our PRTG NTP monitor logged the 
problem this way:

      
Sensor SNTP (SNTP) ***
Device      10.2.10.90-TimeMachine NTP server (10.2.10.90)  
New Status at 5/10/2024 12:49:52 PM (Pacific Standard Time):
Down
Last Message:
The target server did not return a valid time. To resolve this issue, use a packet analyzing tool and do a trace of the 
NTP packets to check if all fields are correctly populated. (code: PE085)

________________________________
From: NANOG <nanog-bounces+mel=beckman.org () nanog org><mailto:nanog-bounces+mel=beckman.org () nanog org> on behalf 
of John Curran <jcurran () arin net><mailto:jcurran () arin net>
Sent: Friday, May 10, 2024 10:54 AM
To: NANOG <nanog () nanog org><mailto:nanog () nanog org>
Subject: NOAA Space Weather Prediction Center issued a Severe (G4) Geomagnetic Storm Watch


SWPC Issues Its First G4 Watch Since 2005 | NOAA / NWS Space Weather Prediction 
Center<https://www.swpc.noaa.gov/news/swpc-issues-its-first-g4-watch-2005>
swpc.noaa.gov<https://www.swpc.noaa.gov/news/swpc-issues-its-first-g4-watch-2005>
[favicon.ico]<https://www.swpc.noaa.gov/news/swpc-issues-its-first-g4-watch-2005>

"Multiple CMEs erupted associated with flare activity from Region 3664 on 07-09 May. These CMEs are expected to merge 
with potential arrival expected by early May 11 on the UTC day.”

(Low but distinct possibility of effects to radio and transmission systems)

FYI,
/John

John Curran
President and CEO
American Registry for Internet Numbers