Commit Graph

12 Commits (20a0cda9e99abf86d37cb42cd98b1be9b727fdf8)

Author SHA1 Message Date
Jiewei Qian 68c038603f metadata: add line number reporting
Adds support to report line numbers when validation fails.

Change-Id: Iba94c5b3582d7e51f15d266d188909d3a82b75cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5740963
Reviewed-by: Jordan Brown <rop@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
9 months ago
Jiewei Qian 1fbd216dbc metadata: expose validation result additional as a getter
This permit downstream clients to retrieve the "source" text and do
their own formatting instead of relying on the format coded in
get_message().

Change-Id: Ia36cbd064ed0781bda76b09b064b97f6dc5e899e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5686730
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
10 months ago
Jiewei Qian 79cfa048c0 metadata: early terminate certain fields to avoid over extraction
This CL adds a "early terminate the field based on field value" parser
mechanism to end the field as soon as the field value provides an
unambiguous answer to the question we care about.

This is to prevent over-extraction over certain fields (specifically,
local modifications) which can either be a definitive answer (e.g. No
modification) or multi-line free-form texts (which may contain unknown
fields that we don't care about at this stage).

This mitigates over extraction of README.chromium files like:

```
Local Modifications:
None

How to Uprev:
Steps...
```

Where the old parser would extract "None\n\nHow to Uprev:\nSteps..."

This CL also refactors single line fields to use the same early
termination mechanism since single line field simply ends as soon as
the line is parsed.

Union[Something, None] is changed to Optional[Something] based on
styleguide.

Bug: b/324149233
Change-Id: I3fca80eaceb071263f8ae8730afda230fff0bbb0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5394917
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
1 year ago
Jiewei Qian 9e7d3cec55 metadata: sort discovered files and validation messages
Explicitly sorts the discovered metadata file, and defines
ordering of validation messages, so the tools will output
will be self-consistent.

Change-Id: I9b263a16b151c014e5950638f066376469c701df
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5379678
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Reviewed-by: Anne Redulla <aredulla@google.com>
1 year ago
Mike Frysinger 124bb8e53c switch to 4 space indent
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.

Reformatted using:
files=( $(
	git ls-tree -r --name-only HEAD | \
		grep -Ev -e '^(third_party|recipes)/' | \
		grep '\.py$';
	git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"

The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py

These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py

Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Anne Redulla 6715758ed9 [ssci] PEP8 formatting for metadata directory
All files in metadata/ are new, so they should follow the PEP-8 style.

Change-Id: I5d8424536c3d7b703e6b8087e0e2d70c06a1549c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4834909
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Rachael Newitt <renewitt@google.com>
2 years ago
Anne Redulla 427f0f43ad [ssci] Script to run validation on all metadata files
Adds script metadata/scan.py which can be used to search for and
validate Chromium dependency metadata files, given a repository
root directory.

Bug: b:277147404
Change-Id: Ibde0eeb7babe0b1e3f9c7f887bece629d390974a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4823596
Commit-Queue: Anne Redulla <aredulla@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
2 years ago
Anne Redulla c7aca34c8e Define main metadata validation functions
This is a reland of commit a1cfc693af

The original commit was reverted do to `ModuleNotFoundError`s. I believe this was due to not specifying `metadata` to be part of the `depot_tools` recipe bundle. I have updated `.gitattributes` for this, and also added `__init__.py` files.

I will put the changes to `presubmit_canned_checks.py` in a later CL, once I can confirm `metadata` is being bundled.

Original change's description:
> [ssci] Added CheckChromiumMetadataFiles in presubmit_canned_checks
>
> Bug: b:277147404
> Change-Id: I14a2f11b256bc85fdfe225443ef533c38463ca3e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4796694
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Rachael Newitt <renewitt@google.com>
> Commit-Queue: Anne Redulla <aredulla@google.com>

Bug: b:277147404
Change-Id: Ibd9efd5970a5393c157ca8763f97064d7c167803
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4803385
Reviewed-by: Rachael Newitt <renewitt@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 760f8bcfb9 Revert "[ssci] Added CheckChromiumMetadataFiles in presubmit_canned_checks"
This reverts commit a1cfc693af.

Reason for revert: causing presubmit errors downstream

Original change's description:
> [ssci] Added CheckChromiumMetadataFiles in presubmit_canned_checks
>
> Bug: b:277147404
> Change-Id: I14a2f11b256bc85fdfe225443ef533c38463ca3e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4796694
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Rachael Newitt <renewitt@google.com>
> Commit-Queue: Anne Redulla <aredulla@google.com>

Bug: b:277147404
Change-Id: I83f52494bc1a3a786505b8b74b2053269baa6e8e
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4803286
Commit-Queue: Anne Redulla <aredulla@google.com>
Auto-Submit: Anne Redulla <aredulla@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla a1cfc693af [ssci] Added CheckChromiumMetadataFiles in presubmit_canned_checks
Bug: b:277147404
Change-Id: I14a2f11b256bc85fdfe225443ef533c38463ca3e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4796694
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 3aeb682373 [ssci] Added validate method for single dependencies
Bug: b:277147404
Change-Id: I54c9c82d093cb11813e1c224da125b8d555f1b29
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4797050
Reviewed-by: Rachael Newitt <renewitt@google.com>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago
Anne Redulla 51690612da [ssci] Defined basic metadata fields
Bug: b:277147404
Change-Id: Iabba43add61ce5090be50ac8f0245f624028c44b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4776728
Reviewed-by: Rachael Newitt <renewitt@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Anne Redulla <aredulla@google.com>
2 years ago