Skip to content
  • Scott Wittenburg's avatar
    pipelines: support testing PRs from forks (#19248) · ef0a555c
    Scott Wittenburg authored
    This change makes improvements to the `spack ci rebuild` command
    which supports running gitlab pipelines on PRs from forks.  Much
    of this has to do with making sure we can run without the secrets
    previously required for running gitlab pipelines (e.g signing key,
    aws credentials, etc).  Specific improvements in this PR:
    
    Check if spack has precisely one signing key, and use that information
    as an additional constraint on whether or not we should attempt to sign
    the binary package we create.
    
    Also, if spack does not have at least one public key, add the install
    option "--no-check-signature"
    
    If we are running a pipeline without any profile or environment
    variables allowing us to push to S3, the pipeline could still
    successfully create a buildcache in the artifacts and move on.  So
    just print a message and move on if pushing either the buildcache
    entry or cdash id file to the remote mirror fails.
    
    When we attempt to generate a pacakge or gpg key index on an S3
    mirror, and there is nothing to index, just print a warning and
    exit gracefully rather than throw an exception.
    
    Support the use of PR-specific mirrors for temporary binary pkg
    storage.  This will allow quality-of-life improvement for developers,
    providing a place to store binaries over the lifetime of a PR, so
    that they must only wait for packages to rebuild from source when
    they push a new commit that causes it to be necessary.
    
    Replace two-pass install with a single pass and the new option:
     --require-full-hash-match.  Doing this also removes the need to
    save a copy of the spack.yaml to be copied over the one spack
    rewrites in between the two spack install passes.
    
    Work around a mirror configuration issue caused by using
    spack.util.executable to do the package installation.
    
    * Update pipeline trigger jobs for PRs from forks
    
    Moving to PRs from forks relies on external synchronization script
    pushing special branch names.  Also secrets will only live on the
    spack mirror project, and must be propagated to the E4S project via
    variables on the trigger jobs.
    
    When this change is merged, pipelines will not run until we update
    the "Custom CI configuration path" in the Gitlab CI Settings, as the
    name of the file has changed to better reflect its purpose.
    
    * Arg to MirrorCollection is used exclusively, so add main remote mirror to it
    
    * Compute full hash less frequently
    
    * Add tests covering index generation error handling code
    ef0a555c