497f2788d49fda531c0836618688f954486eb87c
mspeir
  Fri Sep 4 09:44:13 2026 -0700
trackLists: do not let the public page claim the download cross-check is clean, refs #37781

The static-page rewrite collapsed mkPage.py's three-way branch on the
hgdownload cross-check into two, so the public run fell into the all-clear
wording whether or not collect.py had found a restricted file that is still
reachable. collect.py's current output has three such files, so the page was
asserting the opposite of what the check found. Restored as three cases: the
public page says only that the check runs and that anything found is reported
privately, the --internal page keeps the table of paths, and the all-clear
wording is used only when the list really is empty. The comment explaining why
the public branch must not be merged back into the all-clear one is back too.

collect.py counted any HTTP code that was not literally 404 as reachable, which
made a curl timeout or a failed connection (empty output, or 000) look like an
exposed file. That mails false alarms and, now that the branch above depends on
it, would drop the all-clear line from the public page on a network blip. Only
2xx and 3xx count as served, 4xx as blocked, and anything else is reported
separately as not checked, in collected.json and on stderr.

Also: the remaining unquoted interpolations into shell=True commands go through
the existing q() helper, rows_by_track() no longer builds the "why" set that the
new table does not use, and US spelling throughout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

diff --git src/hg/utils/otto/trackLists/trackLists.sh src/hg/utils/otto/trackLists/trackLists.sh
index c42d4719777..47aeede56e3 100755
--- src/hg/utils/otto/trackLists/trackLists.sh
+++ src/hg/utils/otto/trackLists/trackLists.sh
@@ -1,44 +1,44 @@
 #!/bin/bash
 #
 # trackLists.sh - build the mirror-facing page listing tracks we cannot
 # redistribute, tracks that update themselves, and contributed tracks. RM #37781
 #
 # Cron (otto): once a week is plenty; none of these lists move daily.
 #     32 6 * * 4 /hive/data/outside/otto/trackLists/trackLists.sh
 #
 # Writes the public page to htdocs on hgwdev. Getting it to the RR needs a
 # /root/<name>AutoPush entry in /etc/crontab, which only cluster-admin can add
 # (same pattern as the daily tips and the session thumbnails, which are likewise
 # generated into htdocs and are not tracked in git).
 #
-# Output behaviour: quiet on success, except that a restricted file found to be
+# Output behavior: quiet on success, except that a restricted file found to be
 # reachable on hgdownload always prints, so cron mails it.
 
 set -o errexit -o pipefail
 umask 002
 
 DIR=${RTDIR:-/hive/data/outside/otto/trackLists}
 HTDOCS=${HTDOCS:-/usr/local/apache/htdocs}   # override for testing
 PAGE=trackLists.html
 
 cd "$DIR"
 
 # --refresh-contrib is implicit: the crawl re-runs itself when the cache ages out
 ./collect.py --cache "$DIR/cache" -o "$DIR/collected.json"
 
 # public page: no list of reachable restricted files
 ./mkPage.py -i "$DIR/collected.json" -o "$DIR/$PAGE"
 
 # internal copy, keeps the hgdownload cross-check, stays on hgwdev
 ./mkPage.py -i "$DIR/collected.json" -o "$DIR/internal.html" --internal
 
 # only replace the live page if it actually changed
 if ! cmp -s "$DIR/$PAGE" "$HTDOCS/$PAGE"; then
     cp "$DIR/$PAGE" "$HTDOCS/$PAGE"
     # The execute bit is what makes apache run the SSI includes on a .html file
     # (XBitHack). Without it the page is served verbatim and the reader gets the
     # bare content with no menu bar and no stylesheets, which is the state
     # allTips.html is in. Do not drop this chmod.
     chmod 775 "$HTDOCS/$PAGE"
     echo "trackLists: updated $HTDOCS/$PAGE"
 fi