in-tree annotations of third-party code (moz.yaml)

15th Jun 2018

I’ve recently landed changes on mozilla-central to provide initial support for in-tree annotations of third-party code. (Bug 1454868, D1208, r5df5e745ce6e).

Why

  • Provide consistency and discoverability to third-party code, its origin (repository, version, SHA, etc), and Mozilla-local modifications
  • Simplify the process for auditing vendorerd versions and licenses
  • Establish a structure which allows automation to drive vendoring

How

  • Using the example moz.yaml from the top of moz_yaml.pl create a moz.yaml in the top level of third-party code
  • Verify the manifest with mach vendor manifest --verify path/to/moz.yaml

Next

  • We will be creating moz.yaml files in-tree (help here is appreciated!)
  • Add tests to ensure moz.yaml files remain valid
  • At some point we’ll add automation-driven vendoring to simplify and standardise the process of updating vendored code

moz.yaml Template

From python/mozbuild/mozbuild/moz_yaml.py#l50

---
# Third-Party Library Template
# All fields are mandatory unless otherwise noted

# Version of this schema
schema: 1

bugzilla:
  # Bugzilla product and component for this directory and subdirectories
  product: product name
  component: component name

# Document the source of externally hosted code
origin:

  # Short name of the package/library
  name: name of the package

  description: short (one line) description

  # Full URL for the package's homepage/etc
  # Usually different from repository url
  url: package's homepage url

  # Human-readable identifier for this version/release
  # Generally "version NNN", "tag SSS", "bookmark SSS"
  release: identifier

  # The package's license, where possible using the mnemonic from
  # https://spdx.org/licenses/
  # Multiple licenses can be specified (as a YAML list)
  # A "LICENSE" file must exist containing the full license text
  license: MPL-2.0

# Configuration for the automated vendoring system.
# Files are always vendored into a directory structure that matches the source
# repository, into the same directory as the moz.yaml file
# optional
vendoring:

  # Repository URL to vendor from
  # eg. https://github.com/kinetiknz/nestegg.git
  # Any repository host can be specified here, however initially we'll only
  # support automated vendoring from selected sources initiall.
  url: source url (generally repository clone url)

  # Revision to pull in
  # Must be a long or short commit SHA (long preferred)
  revision: sha

  # List of patch files to apply after vendoring. Applied in the order
  # specified, and alphabetically if globbing is used. Patches must apply
  # cleanly before changes are pushed
  # All patch files are implicitly added to the keep file list.
  # optional
  patches:
    - file
    - path/to/file
    - path/*.patch

  # List of files that are not deleted while vendoring
  # Implicitly contains "moz.yaml", any files referenced as patches
  # optional
  keep:
    - file
    - path/to/file
    - another/path
    - *.mozilla

  # Files/paths that will not be vendored from source repository
  # Implicitly contains ".git", and ".gitignore"
  # optional
  exclude:
    - file
    - path/to/file
    - another/path
    - docs
    - src/*.test

  # Files/paths that will always be vendored, even if they would
  # otherwise be excluded by "exclude".
  # optional
  include:
    - file
    - path/to/file
    - another/path
    - docs/LICENSE.*

  # If neither "exclude" or "include" are set, all files will be vendored
  # Files/paths in "include" will always be vendored, even if excluded
  # eg. excluding "docs/" then including "docs/LICENSE" will vendor just the
  #     LICENSE file from the docs directory

  # All three file/path parameters ("keep", "exclude", and "include") support
  # filenames, directory names, and globs/wildcards.

  # In-tree scripts to be executed after vendoring but before pushing.
  # optional
  run_after:
    - script
    - another script