[2023-10-14] :: `brew` formula installing from GitHubPrivateRepositoryReleaseDownloadStrategy (removed form Homebrew core since v2)

git_private_repo_strategy.rb

# Write this file to somewhere named like `lib/private_strategy.rb`, then
# add it like: `require_relative "lib/private_strategy"` to your formula.
#
# This is based on the following, with first minor fixes by minamijoyo from Github.
# https://gist.github.com/minamijoyo/3d8aa79085369efb79964ba45e24bb0e
# https://github.com/Homebrew/brew/blob/193af1442f6b9a19fa71325160d0ee2889a1b6c9/Library/Homebrew/compat/download_strategy.rb#L48-L157
#
# Last modified by welkinSL on [2024-05-14 Tue] (May)

# BSD 2-Clause License
#
# Copyright (c) 2009-present, Homebrew contributors
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice, this
#   list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above copyright notice,
#   this list of conditions and the following disclaimer in the documentation
#   and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

# GitHubPrivateRepositoryDownloadStrategy downloads contents from GitHub
# Private Repository. To use it, add
# `:using => GitHubPrivateRepositoryDownloadStrategy` to the URL section of
# your formula. This download strategy uses GitHub access tokens (in the
# environment variables `HOMEBREW_GITHUB_API_TOKEN`) to sign the request.  This
# strategy is suitable for corporate use just like S3DownloadStrategy, because
# it lets you use a private GitHub repository for internal distribution.  It
# works with public one, but in that case simply use CurlDownloadStrategy.

class GitHubPrivateRepositoryDownloadStrategy < CurlDownloadStrategy
  require "utils/formatter"
  require "utils/github"

  # fix issue: https://github.com/Homebrew/brew/issues/15169
  # bypass a HEAD request that does NOT contains token, which will fail
  def resolve_url_basename_time_file_size(url, timeout: nil)
    url = download_url
    super
  end

  # [2023-10-14] brew relies on this output to rename the downloaded file
  # See: https://github.com/Homebrew/brew/blob/fbe50bf280bff033b968d439d5441d338afec98f/Library/Homebrew/download_strategy.rb#L305
  # Not setting this will break formulas during install stage, symtoms: Errno::ENOENT: No such file or directory - path/to/file
  def resolved_basename
    @filename
  end
  
  def initialize(url, name, version, **meta)
    super
    parse_url_pattern
    set_github_token
  end

  def parse_url_pattern
    unless match = url.match(%r{https://github.com/([^/]+)/([^/]+)/(\S+)})
      raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Repository."
    end

    _, @owner, @repo, @filepath = *match
  end

  def download_url
    "https://#{@github_token}@github.com/#{@owner}/#{@repo}/#{@filepath}"
  end

  private

  def _fetch(url:, resolved_url:)
    curl_download download_url, to: temporary_path
  end

  def set_github_token
    @github_token = ENV["HOMEBREW_GITHUB_API_TOKEN"]
    unless @github_token
      raise CurlDownloadStrategyError, "Environmental variable HOMEBREW_GITHUB_API_TOKEN is required."
    end

    validate_github_repository_access!
  end

  def validate_github_repository_access!
    # Test access to the repository
    GitHub.repository(@owner, @repo)
    # should have splitted from GitHub::HTTPNotFoundError as of [2021-02-08 Mon]
    # BUT only started failing for me recently, maybe cache, or no formula update?
  rescue GitHub::API::HTTPNotFoundError
    # We only handle HTTPNotFoundError here,
    # becase AuthenticationFailedError is handled within util/github.
    message = <<~EOS
      HOMEBREW_GITHUB_API_TOKEN can not access the repository: #{@owner}/#{@repo}
      This token may not have permission to access the repository or the url of formula may be incorrect.
    EOS
    raise CurlDownloadStrategyError, message
  end
end

# GitHubPrivateRepositoryReleaseDownloadStrategy downloads tarballs from GitHub
# Release assets. To use it, add
# `:using => GitHubPrivateRepositoryReleaseDownloadStrategy` to the URL section of
# your formula. This download strategy uses GitHub access tokens (in the
# environment variables HOMEBREW_GITHUB_API_TOKEN) to sign the request.
class GitHubPrivateRepositoryReleaseDownloadStrategy < GitHubPrivateRepositoryDownloadStrategy
  def initialize(url, name, version, **meta)
    super
  end

  def parse_url_pattern
    url_pattern = %r{https://github.com/([^/]+)/([^/]+)/releases/download/([^/]+)/(\S+)}
    unless @url =~ url_pattern
      raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Release."
    end

    _, @owner, @repo, @tag, @filename = *@url.match(url_pattern)
  end

  def download_url
    "https://#{@github_token}@api.github.com/repos/#{@owner}/#{@repo}/releases/assets/#{asset_id}"
  end

  private

  def _fetch(url:, resolved_url:, timeout:) 
    # HTTP request header `Accept: application/octet-stream` is required.
    # Without this, the GitHub API will respond with metadata, not binary.
    curl_download download_url, "--header", "Accept: application/octet-stream", to: temporary_path
  end

  def asset_id
    @asset_id ||= resolve_asset_id
  end

  def resolve_asset_id
    release_metadata = fetch_release_metadata
    assets = release_metadata["assets"].select { |a| a["name"] == @filename }
    raise CurlDownloadStrategyError, "Asset file not found." if assets.empty?

    assets.first["id"]
  end

  def fetch_release_metadata
    release_url = "https://api.github.com/repos/#{@owner}/#{@repo}/releases/tags/#{@tag}"
    GitHub::API.open_rest(release_url, data: nil, data_binary_path: nil, request_method: nil, scopes: [].freeze, parse_json: true)
  end
end

install.sh

brew tap myaccount/mytap
HOMEBREW_GITHUB_API_TOKEN=xxx brew install myformula

myformula.rb

require_relative "lib/git_private_repo_strategy"

class MyFormula < Formula
  desc "A script called projectname."
  homepage "https://github.com/myaccount/myformula"
  url "https://github.com/myaccount/myformula/releases/download/v1.1/myformula.py", using: GitHubPrivateRepositoryReleaseDownloadStrategy
  sha256 "0123456789abcdefghijklmnopqrstuvwxyz0123456789abcdefghijklmnopqrs"
  head "https://github.com/myaccount/myformula.git"

  # installing single script will fail if resolve_url_basename_time_file_size (by modifying url) but NOT resolved_basename
  def install
    bin.install "myformula.py" => "myformula"
  end

  def caveats
    "The command `myformula` must be available.  See full requirements on https://github.com/myaccount/myformula."
  end

  test do
    system "false"
    assert_equal "Usage: myformula [option]

    Expected output of `myformula -h`.
".strip, shell_output("#{bin}/myformula -h").strip
  end
end

Hi lads, hope this will be helpful for you if you are still using this strategy:

TL;DR you will need to update your strategy to override

both resolve_url_basename_time_file_size and resolved_basename** NEW
to address all use cases (see next section):

class GitHubPrivateRepositoryDownloadStrategy < CurlDownloadStrategy
  require "utils/formatter"
  require "utils/github"

  # fix issue: https://github.com/Homebrew/brew/issues/15169
  # bypass a HEAD request that does NOT contains token, which will fail
  def resolve_url_basename_time_file_size(url, timeout: nil)
    url = download_url
    super
  end

  # [2023-10-14] brew relies on this output to rename the downloaded file
  # See: https://github.com/Homebrew/brew/blob/fbe50bf280bff033b968d439d5441d338afec98f/Library/Homebrew/download_strategy.rb#L305
  # Not setting this will break formulas during install stage, symtoms: Errno::ENOENT: No such file or directory - path/to/file
  def resolved_basename
    @filename
  end
# ... skipped

I think at this point everyone should have overridden their resolve_url_basename_time_file_size method to deal
with the forced HEAD request as mentioned at Homebrew/brew#15169.

However, please read this the definition of cached_location:
https://github.com/Homebrew/brew/blob/fbe50bf280bff033b968d439d5441d338afec98f/Library/Homebrew/download_strategy.rb#L305

In short, AFAIK the practice in Homebrew is all download strategy should download stuff to temporary_path. The dependency is as follows:
temporary_path -> resolved_basename -> resolved_url_and_basename -> resolve_url_basename_time_file_size (for CurlDownloadStrategy only)

From this hierarchy we can see that if we overridden the former it will break download path too.
Symtoms are:

(during call to InstallRenamed.install_p)

Errno::ENOENT: No such file or directory - path/to/file

Of course an alternative is to just override resolve_url_basename_time_file_size with:

# from https://github.com/Homebrew/brew/issues/15169#issuecomment-1500653530
def resolve_url_basename_time_file_size(url, timeout: nil)
  [download_url, "", Time.now, 0, false]
end

However this will be less robust to changes to Homebrew's code.

---

Affected use case

From what I know this issue seems to only affect formulas that just installs a file via the bin.install directive.
For archives, my guess is extraction will be carried out transparently first, hence the file names inside the archive will be preserved.
I have NOT drill into how this the exact process is tho, so please share if you did.

However, for consistency sake I will suggest everyone to make this change.
Changing should also guard against behaviour changes on Homebrew's side, say if they decide to suffix all extracted files for whatever reasons.

In the rescue section for validate_github_repository_access!, GitHub::HTTPNotFoundError should have split to the new GitHub::API::HTTPNotFoundError as of [2021-02-08 Mon].

However it had only started failing for me recently, first noticed on [2024-05-02 Thu].
This should be due cache, and that I rarely changes my private formulas.

Anyways, changing fixed the errors thrown by brew for me so I recommend everyone else to do it.