Why is there "no new release found" during do-release-upgrade when using an intermediary TLS termination proxy?

The question best matching my situation is described here:

do-release-upgrade failing on Ubuntu 16.04 LTS - says: No new release found

... and questions linked therein; I'd have replied to one of those as an "alternative answer", but my SE rep is insufficient.

In rare cases, the issue stems from a TLS termination proxy being used, a fact that may be obscured from a particular VPS sysadmin, or out of reach to configure.

The issue manifests as follows:

% do-release-upgrade -c
Checking for a new Ubuntu release
Failed to connect to Check your Internet connection or proxy settings
There is no development version of an LTS available.
To upgrade to the latest non-LTS develoment release
set Prompt=normal in /etc/update-manager/release-upgrades.

However, retrieving the file manually works, e.g. on Ubuntu 18.04.5 LTS:

% curl -s | grep -A10 focal
Dist: focal
Name: Focal Fossa
Version: 20.04.2 LTS
Date: Thu, 23 April 2020 20:04:00 UTC
Supported: 1
Description: This is the 20.04.2 LTS release
Release-File:
ReleaseNotes:
ReleaseNotesHtml:
UpgradeTool:
UpgradeToolSignature: 

do-release-upgrade is written in Python using urllib, its urlopen is tripping up for some reason, and I can't determine why.

1 Answer

I've started with enabling debug mode:

% export DEBUG_UPDATE_MANAGER=yesplz
% do-release-upgrade -c

... and finding the following in the output:

...
result of meta-release download: '<urlopen error Tunnel connection failed: 403 SSL CONNECT proxying not configured>'
...

The first work-around I've tried is changing the URL in file /etc/update-manager/meta-release to have http:// instead of https://. This "works", but not something to take lightly.

I've also tried installing squid (a caching proxy) on the same machine and setting environment variables http[s]_proxy (lowercase and uppercase), guessing that an extra layer of software may cargo-cult releases into becoming reachable; but that didn't work (i.e. same error).

The last workaround - which works! - is downloading the file manually:

% curl -o /home/`whoami`/.cache/update-manager-core/meta-release-lts
% sudo curl -o /var/lib/update-manager/meta-release-lts

... and then forcing this file to be used, by editing /usr/lib/python3/dist-packages/UpdateManager/Core/MetaRelease.py and inserting at line 339, before an urlopen() attempt is made:

print('UGLY: urlopen my XSS! >:(')
raise(HTTPError(code=304, url='', msg='', hdrs='', fp=open('/tmp/myXSS', 'w')))

So, to answer my own question, this is likely because of:

• the particular TLS terminating proxy configuration (that I don't have control over); or
• Python's urllib's urlopen not picking up the proxy settings, or the settings being incorrect (in the "squid workaround" case).