1. 17 Mar, 2021 2 commits
    • Massimiliano Culpo's avatar
      Speed-up CI by reorganizing tests (#22247) · b304b4bd
      Massimiliano Culpo authored
      * unit tests: mark slow tests as "maybeslow"
      
      This commit also removes the "network" marker and
      marks every "network" test as "maybeslow". Tests
      marked as db are maintained, but they're not slow
      anymore.
      
      * GA: require style tests to pass before running unit-tests
      
      * GA: make MacOS unit tests fail fast
      
      * GA: move all unit tests into the same workflow, run style tests as a prerequisite
      
      All the unit tests have been moved into the same workflow so that a single
      run of the dorny/paths-filter action can be used to ask for coverage based
      on the files that have been changed in a PR. The basic idea is that for PRs
      that introduce only changes to packages coverage is not necessary, this
      resulting in a faster execution of the tests.
      
      Also, for package only PRs slow unit tests are skipped.
      
      Finally, MacOS and linux unit tests are now conditional on style tests passing
      meaning that e.g. we won't waste a MacOS worker if we know that the PR has
      flake8 issues.
      
      * Addressed review comments
      
      * Skipping slow tests on MacOS for package only recipes
      
      * QA: make tests on changes correct before merging
      b304b4bd
    • a-saitoh-fj's avatar
      py-aiobotocore: New package (#22258) · d36de79b
      a-saitoh-fj authored
      * py-aiobotocore: New package
      
      * py-aiobotocore: Added python dependencies, and Removed unnecessary whitespace
      d36de79b
  2. 16 Mar, 2021 16 commits
    • Todd Gamblin's avatar
      bugfix: allow imposed constraints to be overridden in special cases · 3e570ce6
      Todd Gamblin authored
      In most cases, we want condition_holds(ID) to imply any imposed
      constraints associated with the ID. However, the dependency relationship
      in Spack is special because it's "extra" conditional -- a dependency
      *condition* may hold, but we have decided that externals will not have
      dependencies, so we need a way to avoid having imposed constraints appear
      for nodes that don't exist.
      
      This introduces a new rule that says that constraints are imposed
      *unless* we define `do_not_impose(ID)`. This allows rules like
      dependencies, which rely on more than just spec conditions, to cancel
      imposed constraints.
      
      We add one special case for this: dependencies of externals.
      3e570ce6
    • Todd Gamblin's avatar
      bugfix: do not generate dep conditions when no dependency · e7cba04b
      Todd Gamblin authored
      We only consider test dependencies some of the time. Some packages are
      *only* test dependencies. Spack's algorithm was previously generating
      dependency conditions that could hold, *even* if there was no potential
      dependency type.
      
      - [x] change asp.py so that this can't happen -- we now only generate
            dependency types for possible dependencies.
      e7cba04b
    • Todd Gamblin's avatar
      concretizer: unify logic for spec conditionals · ada6ecc7
      Todd Gamblin authored
      This builds on #20638 by unifying all the places in the concretizer where
      things are conditional on specs. Previously, we duplicated a common spec
      conditional pattern for dependencies, virtual providers, conflicts, and
      externals. That was introduced in #20423 and refined in #20507, and
      roughly looked as follows.
      
      Given some directives in a package like:
      
      ```python
      depends_on("foo@1.0+bar", when="@2.0+variant")
      provides("mpi@2:", when="@1.9:")
      ```
      
      We handled the `@2.0+variant` and `@1.9:` parts by generating generated
      `dependency_condition()`, `required_dependency_condition()`, and
      `imposed_dependency_condition()` facts to trigger rules like this:
      
      ```prolog
      dependency_conditions_hold(ID, Parent, Dependency) :-
        attr(Name, Arg1)             : required_dependency_condition(ID, Name, Arg1);
        attr(Name, Arg1, Arg2)       : required_dependency_condition(ID, Name, Arg1, Arg2);
        attr(Name, Arg1, Arg2, Arg3) : required_dependency_condition(ID, Name, Arg1, Arg2, Arg3);
        dependency_condition(ID, Parent, Dependency);
        node(Parent).
      ```
      
      And we handled `foo@1.0+bar` and `mpi@2:` parts ("imposed constraints")
      like this:
      
      ```prolog
      attr(Name, Arg1, Arg2) :-
        dependency_conditions_hold(ID, Package, Dependency),
        imposed_dependency_condition(ID, Name, Arg1, Arg2).
      
      attr(Name, Arg1, Arg2, Arg3) :-
        dependency_conditions_hold(ID, Package, Dependency),
        imposed_dependency_condition(ID, Name, Arg1, Arg2, Arg3).
      ```
      
      These rules were repeated with different input predicates for
      requirements (e.g., `required_dependency_condition`) and imposed
      constraints (e.g., `imposed_dependency_condition`) throughout
      `concretize.lp`. In #20638 it got to be a bit confusing, because we used
      the same `dependency_condition_holds` predicate to impose constraints on
      conditional dependencies and virtual providers. So, even though the
      pattern was repeated, some of the conditional rules were conjoined in a
      weird way.
      
      Instead of repeating this pattern everywhere, we now have *one* set of
      consolidated rules for conditions:
      
      ```prolog
      condition_holds(ID) :-
        condition(ID);
        attr(Name, A1)         : condition_requirement(ID, Name, A1);
        attr(Name, A1, A2)     : condition_requirement(ID, Name, A1, A2);
        attr(Name, A1, A2, A3) : condition_requirement(ID, Name, A1, A2, A3).
      
      attr(Name, A1)         :- condition_holds(ID), imposed_constraint(ID, Name, A1).
      attr(Name, A1, A2)     :- condition_holds(ID), imposed_constraint(ID, Name, A1, A2).
      attr(Name, A1, A2, A3) :- condition_holds(ID), imposed_constraint(ID, Name, A1, A2, A3).
      ```
      
      this allows us to use `condition(ID)` and `condition_holds(ID)` to
      encapsulate the conditional logic on specs in all the scenarios where we
      need it. Instead of defining predicates for the requirements and imposed
      constraints, we generate the condition inputs with generic facts, and
      define predicates to associate the condition ID with a particular
      scenario. So, now, the generated facts for a condition look like this:
      
      ```prolog
      condition(121).
      condition_requirement(121,"node","cairo").
      condition_requirement(121,"variant_value","cairo","fc","True").
      imposed_constraint(121,"version_satisfies","fontconfig","2.10.91:").
      dependency_condition(121,"cairo","fontconfig").
      dependency_type(121,"build").
      dependency_type(121,"link").
      ```
      
      The requirements and imposed constraints are generic, and we associate
      them with their meaning via the id. Here, `dependency_condition(121,
      "cairo", "fontconfig")` tells us that condition 121 has to do with the
      dependency of `cairo` on `fontconfig`, and the conditional dependency
      rules just become:
      
      ```prolog
      dependency_holds(Package, Dependency, Type) :-
        dependency_condition(ID, Package, Dependency),
        dependency_type(ID, Type),
        condition_holds(ID).
      ```
      
      Dependencies, virtuals, conflicts, and externals all now use similar
      patterns, and the logic for generating condition facts is common to all
      of them on the python side, as well. The more specific routines like
      `package_dependencies_rules` just call `self.condition(...)` to get an id
      and generate requirements and imposed constraints, then they generate
      their extra facts with the returned id, like this:
      
      ```python
          def package_dependencies_rules(self, pkg, tests):
              """Translate 'depends_on' directives into ASP logic."""
              for _, conditions in sorted(pkg.dependencies.items()):
                  for cond, dep in sorted(conditions.items()):
                      condition_id = self.condition(cond, dep.spec, pkg.name)  # create a condition and get its id
                      self.gen.fact(fn.dependency_condition(  # associate specifics about the dependency w/the id
                          condition_id, pkg.name, dep.spec.name
                      ))
              # etc.
      ```
      
      - [x] unify generation and logic for conditions
      - [x] use unified logic for dependencies
      - [x] use unified logic for virtuals
      - [x] use unified logic for conflicts
      - [x] use unified logic for externals
      
      LocalWords:  concretizer mpi attr Arg concretize lp cairo fc fontconfig
      LocalWords:  virtuals def pkg cond dep fn refactor github py
      ada6ecc7
    • Tomoyasu Nojiri's avatar
      amdlibflame: Fix build error (#22316) · ce7bde24
      Tomoyasu Nojiri authored
      ce7bde24
    • Tomoyasu Nojiri's avatar
      libflame: Fix build error (#22315) · 475c877f
      Tomoyasu Nojiri authored
      475c877f
    • Harmen Stoppels's avatar
      8ffeb96c
    • Adam J. Stewart's avatar
      d7f1ff68
    • Harmen Stoppels's avatar
      Expand relative dev paths in environment files (#22045) · 19534111
      Harmen Stoppels authored
      * Rewrite relative dev_spec paths internally to absolute paths in case of relocation of the environment file
      
      * Test relative paths for dev_path in environments
      
      * Add a --keep-relative flag to spack env create
      
      This ensures that relative paths of develop paths are not expanded to
      absolute paths when initializing the environment in a different location
      from the spack.yaml init file.
      19534111
    • Harmen Stoppels's avatar
      Propagate --test= for environments (#22040) · 4f1d9d60
      Harmen Stoppels authored
      * Propagate --test= for environments
      
      * Improve help comment for spack concretize --test flag
      
      * Add tests for --test with environments
      4f1d9d60
    • Chuck Atkins's avatar
    • Harmen Stoppels's avatar
    • Adam J. Stewart's avatar
      d166ca91
    • Adam J. Stewart's avatar
      70c505a0
    • Adam J. Stewart's avatar
      78d1b28b
    • Adam J. Stewart's avatar
      3c493ae6
    • Tomoyasu Nojiri's avatar
      9c7669ed
  3. 15 Mar, 2021 3 commits
    • Christoph Junghans's avatar
      votca-*: add v2021 (#22282) · 0c8df39f
      Christoph Junghans authored
      0c8df39f
    • a-saitoh-fj's avatar
      py-aioitertools: New package (#22256) · 3e4a24c8
      a-saitoh-fj authored
      * py-aioitertools: New package
      
      * py-aioitertools: Fixed python dependencies, and Fixed style of code
      
      * switch to the source build
      3e4a24c8
    • Vanessasaurus's avatar
      do not validate variants of concrete specs in solver setup (#22272) · 6ff717f3
      Vanessasaurus authored
      
      
      Currently, regardless of a spec being concrete or not, we validate its variants in `spec_clauses` (part of `SpackSolverSetup`).  
      
      This PR skips the check if the spec is concrete.
      
      The reason we want to do this is so that the solver setup class (really, `spec_clauses`) can be used for cases when we just want the logic statements / facts (is that what they are called?) and we don't need to re-validate an already concrete spec.  We can't change existing concrete specs, and we have to be able to handle them *even if they violate constraints in the current spack*.  This happens in practice if we are doing the validation for a spec produced by a different spack install.
      Signed-off-by: default avatarvsoch <vsoch@users.noreply.github.com>
      6ff717f3
  4. 14 Mar, 2021 1 commit
  5. 13 Mar, 2021 14 commits
  6. 12 Mar, 2021 4 commits