This has been detected to happen on the Friday, 3rd of April 2024 weekly roll (and may have happened a few times along history).

Every day, at 08:50 (AU/Perth) (or 00:50 UTC), the script in charge of publishing any new packages is run in packaging.moodle.org . Publishing implies updating the download.moodle.org information, the updates API, creating the windows packages...).

1. For tagged releases (majors and minors) the "publishing" only happens, by default, on Mondays and only if the last commit in the branches is less than 96h ago (4 days). So we can package on Friday/Saturday/Sunday and, on Monday, the publishng will happen.
2. For non-tagged releases (weeklies, continuous, on-sync, beta, rcs, ...), the "publishing" only happens if the last commit in the branches has been pushed less than 24h ago (1 day).

And, the later (non-tagged releases) is when we can face some problems depending of the times of the commits. Example (all with UTC dates):

• Integrator X pre-releases (prerelease.sh) some weeklies @ 00:30. So, the commits will have that time.
• Following the process, before releasing, it's required to wait for CI post-tests to end.
• At 00:40 (we check for changes bi-hourly in the 40 minutes) there aren't new commit in the upstream repos, so packaging has nothing to do.
• At 00:50 nothing is published, because 10 minutes earlier nothing has been packaged.
• At 01:00 (any time after 00:50) the integrator releases (release.sh) to upstream repos.
• At 02:40 (the bi-hourly check) detects the new commits upstream and build the weekly packages (note that they aren't still "published", just "packaged").
• The next day, at 00:50, it's again time for "publishing" but the tool finds that the commits of the weeklies are already > 24 hours old (see the 1st point, the commits were generated at 00:30 the day before).
• Hence, those weeklies aren't ever "published".

And that's the problem, so far. So TL;DR;

There are a few minutes in the day, where it's "dangerous" to pre-release and release because, depending of the times, that can lead to unpublished weekly packages.

Basically, what we have to do is to avoid the "packaging" (00:40) and the "publishing" (00:50) scripts in the packaging server to be executed between the "prerelease" and the "release" scripts invoked by the integrator (mdlrelease).

Or said with other words, we have to avoid to run "prerelease" and "release" when the "packaging" and "publishing" scripts are scheduled to run in the packaging server (00:40 and 00:50) in the middle.

So this issue is about to make the mdlrelease tool aware of those times (00:40 and 00:50) and:

• When prerelease is executed if we are within the last hour before 00:40, stop the process and tell the integrator to run the script again after 00:50.
• When release is executed, verify that between the commit dates and the execution time, the 00:40 and 00:50 has not happened. Note this is the final guard and we shouldn't get here ever, only if the integrator has become "distracted" and a lot of time passes between "prerelease" and "release" (> 1 hour), we'll get into this. Still, better stop, rewrite and prerelease again.

And that's the plan, I'll go with it soon, so comments are welcome.

Ciao