MADIS DR RUN-TIME STATUS for Thu Mar 30 00:32:02 GMT 2023 on vm-bldr-madis-ops-recovery1
     INFO Node name: vm-bldr-madis-ops-recovery1
     PASS METAR processing tbl exists:  128320 Mar 28 17:30 /opt/ldm/madis/dr/decode/data/fxa/LDAD/data/NonFedAWOSStation.tbl
     PASS Station update flag exists:  0 Feb  1 16:52 /opt/ldm/madis/dr/decode/awips/fxa/GO-UPDATE-STATIONS
     PASS Metar station table exists:  1121683 Feb  1 16:51 /opt/ldm/madis/dr/sfc/iqc1/fslparms/lsfc.tbl
     FAIL Missing Accept/reject|autorej files exists: AO: 1 IQC: 1 NON-SFC: 
     FAIL Missing persistent processes exists: pqact (1+): 1 ldmd (1+): 7 DECODE (4): 8
     PASS Dcmetr check: CNT-LDAD.raw: 1214
     PASS Load resource check: 60% of the time M15 < 9 Samples: 32 %<60:  100% #10=+ 0 #9 0 #8 0 #7 0
     PASS Open file resource check: < 5000 Max files: -1 #<1000: 32 #<10000 32
     PASS Process count check: < 1000 Max Processes: 403
     FAIL No entries in file: times.out.23089
     PASS IQC Throughput test: <30 delayed #Delayed: 0 Competed timely:  0    
     PASS No '0' length Station table files in data: #: 0
     PASS No '0' length Station table files in master: #: 0
     PASS No netcdf-4.1 netCDF lib connection for routerStoreNetCDF:
     PASS Shefdecoder check 20230328_2200: #CPYRMA: 18 #NJWXNET: 636 #NERRS: 0
  COMPARE UNIQUE mesonet provider cnt for 20230328_2200: 29
     PASS DR_SendFile2LDM is updating MADIS netcdf common data directories 
  COMPARE DECODE STN TBLS           137  Newest:   214482 Mar 29 23:55 /opt/ldm/madis/dr/decode/data/fxa/LDAD/data/RAWSStation.txt  Oldest:     2699 Mar 29 15:00 /opt/ldm/madis/dr/decode/data/fxa/LDAD/data/AFAStation.txt
  COMPARE PRE-DECODE STN TBLS       137  Newest:   214482 Mar 29 23:55 /opt/ldm/madis/dr/decode/data/fxa/LDAD/data/master/RAWSStation.txt  Oldest:     2699 Mar 29 15:00 /opt/ldm/madis/dr/decode/data/fxa/LDAD/data/master/AFAStation.txt
  COMPARE Compare rough file sizes for same hour on known good system. Hour: 20230328_22
-rw-r-----. 1    316448 Mar 28 23:50 /opt/ldm/madis/dr/decode/data/fxa/LDAD/crn/netCDF/20230328_2200
-rw-r-----. 1    141360 Mar 29 00:55 /opt/ldm/madis/dr/decode/data/fxa/LDAD/hcn/netCDF/20230328_2200
-rw-r-----. 1   5767924 Mar 29 22:46 /opt/ldm/madis/dr/decode/data/fxa/LDAD/hydro/netCDF/20230328_2200
-rwxr-x---. 1 168011200 Mar 29 22:40 /opt/ldm/madis/dr/decode/data/fxa/LDAD/mesonet/netCDF/20230328_2200
-rw-r-----. 1    131508 Mar 29 23:55 /opt/ldm/madis/dr/decode/data/fxa/LDAD/nepp/netCDF/20230328_2200
-rw-r-----. 1  11816988 Mar 29 04:04 /opt/ldm/madis/dr/decode/data/fxa/LDAD/rwis/netCDF/20230328_2200
-rw-r-----. 1    526284 Mar 29 21:46 /opt/ldm/madis/dr/decode/data/fxa/LDAD/snow/netCDF/20230328_2200
-rw-r-----. 1   5338272 Mar 28 23:06 /opt/ldm/madis/dr/decode/data/fxa/LDAD/urbanet/netCDF/20230328_2200
-rw-r--r--. 1   760820 Mar 29 02:13 /opt/ldm/madis/dr/sfc/iqc1/qcms_data/buoy/20230328_2200
-rw-r--r--. 1   264736 Mar 29 23:16 /opt/ldm/madis/dr/sfc/iqc1/qcms_data/crn/20230328_2200
-rw-r--r--. 1    69692 Mar 29 22:15 /opt/ldm/madis/dr/sfc/iqc1/qcms_data/hcn/20230328_2200
-rw-r--r--. 1  4698924 Mar 29 22:15 /opt/ldm/madis/dr/sfc/iqc1/qcms_data/hydro/20230328_2200
-rw-r--r--. 1 62638360 Mar 29 23:16 /opt/ldm/madis/dr/sfc/iqc1/qcms_data/mesonet/20230328_2200
-rw-r--r--. 1  5057828 Mar 29 02:13 /opt/ldm/madis/dr/sfc/iqc1/qcms_data/metar/20230328_2200
-rw-r--r--. 1    72928 Mar 29 22:15 /opt/ldm/madis/dr/sfc/iqc1/qcms_data/nepp/20230328_2200
-rw-r--r--. 1  2400472 Mar 29 23:16 /opt/ldm/madis/dr/sfc/iqc1/qcms_data/urbanet/20230328_2200
-rw-r--r--. 1   151924 Mar 28 23:17 /opt/ldm/madis/dr/sfc/iqc2/qcms_data/buoy/20230328_2200
-rw-r--r--. 1   150028 Mar 29 23:19 /opt/ldm/madis/dr/sfc/iqc2/qcms_data/crn/20230328_2200
-rw-r--r--. 1    69692 Mar 29 22:18 /opt/ldm/madis/dr/sfc/iqc2/qcms_data/hcn/20230328_2200
-rw-r--r--. 1  1404780 Mar 29 22:18 /opt/ldm/madis/dr/sfc/iqc2/qcms_data/hydro/20230328_2200
-rw-r--r--. 1 47992780 Mar 29 23:19 /opt/ldm/madis/dr/sfc/iqc2/qcms_data/mesonet/20230328_2200
-rw-r--r--. 1  1520612 Mar 29 00:17 /opt/ldm/madis/dr/sfc/iqc2/qcms_data/metar/20230328_2200
-rw-r--r--. 1    72928 Mar 29 22:18 /opt/ldm/madis/dr/sfc/iqc2/qcms_data/nepp/20230328_2200
-rw-r--r--. 1  2493572 Mar 29 23:19 /opt/ldm/madis/dr/sfc/iqc2/qcms_data/urbanet/20230328_2200
-rw-r--r--. 1   162148 Mar 28 23:23 /opt/ldm/madis/dr/sfc/iqc3/qcms_data/buoy/20230328_2200
-rw-r--r--. 1   123892 Mar 29 22:24 /opt/ldm/madis/dr/sfc/iqc3/qcms_data/crn/20230328_2200
-rw-r--r--. 1    69692 Mar 29 22:24 /opt/ldm/madis/dr/sfc/iqc3/qcms_data/hcn/20230328_2200
-rw-r--r--. 1  2154872 Mar 29 22:24 /opt/ldm/madis/dr/sfc/iqc3/qcms_data/hydro/20230328_2200
-rw-r--r--. 1 69814584 Mar 29 23:25 /opt/ldm/madis/dr/sfc/iqc3/qcms_data/mesonet/20230328_2200
-rw-r--r--. 1  2680548 Mar 29 02:22 /opt/ldm/madis/dr/sfc/iqc3/qcms_data/metar/20230328_2200
-rw-r--r--. 1    72928 Mar 29 22:24 /opt/ldm/madis/dr/sfc/iqc3/qcms_data/nepp/20230328_2200
-rw-r--r--. 1  1919012 Mar 29 23:26 /opt/ldm/madis/dr/sfc/iqc3/qcms_data/urbanet/20230328_2200
-rw-r--r--. 1   291652 Mar 29 02:25 /opt/ldm/madis/dr/sfc/iqc4/qcms_data/buoy/20230328_2200
-rw-r--r--. 1   112276 Mar 29 22:28 /opt/ldm/madis/dr/sfc/iqc4/qcms_data/crn/20230328_2200
-rw-r--r--. 1   202396 Mar 29 23:29 /opt/ldm/madis/dr/sfc/iqc4/qcms_data/hcn/20230328_2200
-rw-r--r--. 1  2877272 Mar 29 22:28 /opt/ldm/madis/dr/sfc/iqc4/qcms_data/hydro/20230328_2200
-rw-r--r--. 1 79789888 Mar 29 23:29 /opt/ldm/madis/dr/sfc/iqc4/qcms_data/mesonet/20230328_2200
-rw-r--r--. 1  2312548 Mar 29 00:25 /opt/ldm/madis/dr/sfc/iqc4/qcms_data/metar/20230328_2200
-rw-r--r--. 1    72928 Mar 29 22:28 /opt/ldm/madis/dr/sfc/iqc4/qcms_data/nepp/20230328_2200
-rw-r--r--. 1  8850972 Mar 29 23:29 /opt/ldm/madis/dr/sfc/iqc4/qcms_data/urbanet/20230328_2200
-rw-r--r--. 1   253028 Mar 29 02:32 /opt/ldm/madis/dr/sfc/iqc5/qcms_data/buoy/20230328_2200
-rw-r--r--. 1   123892 Mar 29 23:36 /opt/ldm/madis/dr/sfc/iqc5/qcms_data/crn/20230328_2200
-rw-r--r--. 1    69692 Mar 29 22:34 /opt/ldm/madis/dr/sfc/iqc5/qcms_data/hcn/20230328_2200
-rw-r--r--. 1  3132520 Mar 29 22:34 /opt/ldm/madis/dr/sfc/iqc5/qcms_data/hydro/20230328_2200
-rw-r--r--. 1 87210756 Mar 29 23:36 /opt/ldm/madis/dr/sfc/iqc5/qcms_data/mesonet/20230328_2200
-rw-r--r--. 1  1894500 Mar 29 02:32 /opt/ldm/madis/dr/sfc/iqc5/qcms_data/metar/20230328_2200
-rw-r--r--. 1   180376 Mar 29 23:36 /opt/ldm/madis/dr/sfc/iqc5/qcms_data/nepp/20230328_2200
-rw-r--r--. 1  4307692 Mar 29 23:36 /opt/ldm/madis/dr/sfc/iqc5/qcms_data/urbanet/20230328_2200
-rw-r--r--. 1    436072 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/crn/20230328_2200
-rw-r--r--. 1    202396 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/hcn/20230328_2200
-rw-r--r--. 1   8343432 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/hydro/20230328_2200
-rw-r--r--. 1 298891732 Mar 29 23:06 /opt/ldm/madis/dr/sfc/ao/qcms_data/mesonet/20230328_2200
-rw-r--r--. 1    180376 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/nepp/20230328_2200
-rw-r--r--. 1  16131392 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/urbanet/20230328_2200
  COMPARE Compare rough file sizes for same hour on known good system. Hour: 20230328_2200
-rw-r--r--. 1   1758228 Mar 29 23:13 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/buoy/20230328_2200
-rw-r--r--. 1    436072 Mar 29 23:13 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/crn/20230328_2200
-rw-r--r--. 1    202396 Mar 29 23:13 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/hcn/20230328_2200
-rw-r--r--. 1  18882160 Mar 29 23:13 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/hfmetar/20230328_2200
-rw-r--r--. 1  12620040 Mar 29 23:13 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/hydro/20230328_2200
-rw-r--r--. 1 491737324 Mar 29 23:11 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/mesonet/20230328_2200
-rw-r--r--. 1  11519908 Mar 29 23:13 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/metar/20230328_2200
-rw-r--r--. 1    180376 Mar 29 23:13 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/nepp/20230328_2200
-rw-r--r--. 1  16131392 Mar 29 23:13 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/urbanet/20230328_2200
-rw-r--r--. 1    436072 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/crn/20230328_2200
-rw-r--r--. 1    202396 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/hcn/20230328_2200
-rw-r--r--. 1   8343432 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/hydro/20230328_2200
-rw-r--r--. 1 298891732 Mar 29 23:06 /opt/ldm/madis/dr/sfc/ao/qcms_data/mesonet/20230328_2200
-rw-r--r--. 1    180376 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/nepp/20230328_2200
-rw-r--r--. 1  16131392 Mar 29 23:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/urbanet/20230328_2200
  COMPARE Compare rough file sizes for same hour on known good system. Hour: 20230328_2100
-rw-r--r--. 1   1875236 Mar 29 22:14 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/buoy/20230328_2100
-rw-r--r--. 1    436072 Mar 29 22:14 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/crn/20230328_2100
-rw-r--r--. 1    202396 Mar 29 22:14 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/hcn/20230328_2100
-rw-r--r--. 1  18841704 Mar 29 22:14 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/hfmetar/20230328_2100
-rw-r--r--. 1  12807864 Mar 29 22:14 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/hydro/20230328_2100
-rw-r--r--. 1 494503344 Mar 29 22:12 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/mesonet/20230328_2100
-rw-r--r--. 1  11596452 Mar 29 22:14 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/metar/20230328_2100
-rw-r--r--. 1    180376 Mar 29 22:14 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/nepp/20230328_2100
-rw-r--r--. 1  16157992 Mar 29 22:14 /opt/ldm/madis/dr/sfc/ao/dr_qcms_data/urbanet/20230328_2100
-rw-r--r--. 1    436072 Mar 29 22:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/crn/20230328_2100
-rw-r--r--. 1    202396 Mar 29 22:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/hcn/20230328_2100
-rw-r--r--. 1   8679348 Mar 29 22:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/hydro/20230328_2100
-rw-r--r--. 1 301210340 Mar 29 22:06 /opt/ldm/madis/dr/sfc/ao/qcms_data/mesonet/20230328_2100
-rw-r--r--. 1    180376 Mar 29 22:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/nepp/20230328_2100
-rw-r--r--. 1  16157992 Mar 29 22:07 /opt/ldm/madis/dr/sfc/ao/qcms_data/urbanet/20230328_2100