2aea926850613e4fcef435265d32564573fdb613
lrnassar
  Wed Aug 28 11:23:21 2024 -0700
Adding the runTrimLogs.sh script made by ChrisL to the tree, this has been running for a few years now. It processes the raw error logs into a better format that can be ingested by the various stats scripts such as the monthly usage reports. Primarily it makes sure that any error log lines belonging to a single cart does not span multiple lines. The second script is a small addition to check if the error logs are no longer updating. This has come up various times over the years, most recently botht he asia and euro logs had not updated since May, roughly 4.5 months old. No RM.

diff --git src/utils/qa/checkErrorLogsAreUpToDate.py src/utils/qa/checkErrorLogsAreUpToDate.py
new file mode 100644
index 0000000..2ae4b73
--- /dev/null
+++ src/utils/qa/checkErrorLogsAreUpToDate.py
@@ -0,0 +1,26 @@
+#Checks that the Asia, Euro and hgw1/2 logs in the trimmed logs directory
+#are no more than a month old. This ensures our crons from primarily Asia/Euro
+#are still running
+
+import datetime, subprocess
+
+def bash(cmd):
+    """Input bash cmd and return stdout"""
+    rawOutput = subprocess.run(cmd,check=True, shell=True, stdout=subprocess.PIPE, universal_newlines=True)
+    return(rawOutput.stdout.split('\n')[0:-1])
+
+def checkDateTimeOnFile(machine,lastMonth):
+    latestLog = bash("ls /hive/users/chmalee/logs/trimmedLogs/result/"+machine+" | tail -1")
+    date = latestLog[0].split(".")[0]
+    datetime_object = datetime.datetime.strptime(date, '%Y%m%d')
+    if datetime_object < lastMonth:
+        print("The following machine logs are more than a month old. Latest log: "+machine+": "+str(date))
+
+def checkDateTimeOnAllMachines():
+    machinesToCheck = ['hgw1','hgw2','asiaNode','euroNode']
+    today = datetime.datetime.today()
+    lastMonth = today - datetime.timedelta(days=30)
+    for machine in machinesToCheck:
+        checkDateTimeOnFile(machine,lastMonth)
+        
+checkDateTimeOnAllMachines()