Until we fix slow Window builds on developer machine,
It would probably be better to limit remote jobs to 1000.
Bug: b/335525655, b/336481263
Change-Id: I065e19480a27203da2b32ceb19050d0846fbc554
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5482550
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Junji Watanabe <jwata@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
If a child repository is not cloned, gclient will assume git_dependency
to match parent, which can be incorrect.
R=gavinmak
Change-Id: Icb464524f5dcf2e18ccae1db00c69963ffd9ef02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5506655
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
The number of revision objects must match the number of current objects
for a given path. The comma separated values for a revision object
must also be in the order of object_name, sha256sum, size_bytes, and
generation.
Example usage:
gclient setdep --revision=src/third_party/js_code_coverage@myobjectname,deadbeef,1223455,9438393984/myobjectname2,deadbefeaef,1223455,9438393984
Resulting diff:
diff --git a/DEPS b/DEPS
index b40bca8cdd89d..c2186185c1db5 100644
--- a/DEPS
+++ b/DEPS
@@ -553,10 +553,10 @@ deps = {
'bucket': 'chromium-nodejs',
'objects': [
{
- 'object_name': 'js_code_coverage/d538975c93eefc7bafd599b50f867e90c1ef17f3',
- 'sha256sum': '646bb00ced0a930b2eb1e4dbcfac18ebbb8f889bb80599e0254d9d6505427914',
- 'size_bytes': 1469185,
- 'generation': 1657780123604338,
+ 'object_name': 'myobjectname',
+ 'sha256sum': 'deadbeef',
+ 'size_bytes': 1223455,
+ 'generation': 9438393984,
},
],
},
Bug: b/324418194
Change-Id: Ibd824f7b51fa88f732c7197e2cc663b58de7479e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5503101
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
If gsutil is not downloaded, and if gsutil (or
download_from_google_storage) is called concurrently with n>2, it's
possible that those processes are doing the same work. There is also a
critical point where gsutil.py can fail with:
Destination path '/depot_tools_path/external_bin/gsutil/gsutil_4.68/d'
already exists error.
To avoid this problem, use FS locking around code that manipulates with
files.
R=jojwang@google.com
Bug: 338040708
Change-Id: Ib83aaa1e09628f878e512d79f2fa5221c2bcfd37
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5502531
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Auto-Submit: Josip Sokcevic <sokcevic@chromium.org>
The check relies on `git ls-tree` which is unavailable for non-git
workspaces.
Bug: 333744051
Change-Id: Ib9151a283c3bf543390f2cf06e0ce213051fb509
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5503095
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Most tarfiles have .getnames() return top level dirs as <top_level_dir_name>
and files as <top_level_dir_name>/filename.txt. But some tarfiles don't
return the top level directories when running .getnames(). Instead we'll
have to parse through each path name to retrieve the top level directories.
Bug: b/337331070, b/337330736
Change-Id: I8c0e05a2c0722c02eda55162bf51b15a9c21e34a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5499051
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Bug: 337720659
Change-Id: I7953bc95c6e8fb1d9ff794935811464a31d804a4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5499917
Reviewed-by: Alvin Jia <alvinjia@google.com>
Auto-Submit: Joel Hockey <joelhockey@chromium.org>
Owners-Override: Joel Hockey <joelhockey@chromium.org>
Commit-Queue: Joel Hockey <joelhockey@chromium.org>
Commit-Queue: Alvin Jia <alvinjia@google.com>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
This fixes the issue that presubmits were not catching unformatted C++
files.
Presubmit hooks run `git cl format --dry-run`, which is expected to
exit with a code of 2 if the patch needs formatting.
When `clang-format-diff.py` is run by `git cl format --dry-run`, it will
catch any non-zero exit codes and exit the entire `git cl` invocation
with code 1.
However, as of https://github.com/llvm/llvm-project/pull/70883,
`clang-format-diff.py` - now exits with a code of 1 when there are
formatting issues. This will cause the entire `git cl` invocation to
exit with code 1 - not code 2!
Fix the issue by specifying `error_ok=True` in the `RunCommand()` call
to ignore the new exit code returned by `clang format-diff.py`.
Bug: None
Change-Id: I874e5f0710155cd665dfeb4e60d375d6f945ca24
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5492806
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Michael Cui <mlcui@google.com>
get_config is implemented by caching git config --list -z, rather than
shelling out for every key one-by-one. This reduces git map-branches
time on my machine from 1.2s to 0.6s.
Change-Id: I999f4b893d248c2f71a36ec0f2594e0cdf08a248
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5498461
Commit-Queue: David Benjamin <davidben@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Even though we already know the parent is origin/main, every branch that
is parented to origin/main ends up shelling out to git in the upstream()
function, just to double-check that origin/main exists.
If the parent has already been registered, don't bother repeating the
work. This cuts out about half of the times that git map-branches shells
out to git and reduces it from 1.7s to 1.2s on my machine.
Change-Id: Ic5fdaaa5bc62ed8a3574f5a28f9f783093dacc2f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5498460
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Auto-Submit: David Benjamin <davidben@chromium.org>
Sometimes the object name can have more than one dot like
20.11.0/f9a337cfa0e2b92d3e5c671c26b454bd8e99769e.
Currently, the dotfiles would be .20_.*
This change makes it so that the dotfiles would be
.20_11_0_f9a337cfa0e2b92d3e5c671c26b454bd8e99769e.*
Bug: b/328065301
Change-Id: I043a4b5fd56e48b046cfdc9a2fde3204b1bb638c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5491861
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Replace uses of '[START_DIR]' and '[CACHE]' in comments with
api.path.start_dir and api.path.cache_dir. This doesn't actually keep
anything from breaking, but it ensures the comments stay correct.
Change-Id: I4087925a29e3e9a4cb47e044733ab6109cb659bd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5492788
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Rob Mohr <mohrr@google.com>
If symbolic-ref remote HEAD is not available, we skip setting it and
query remote Git using ls-remote. Such information is not stored, and
gclient will need to repeat it on the next invocation.
Instead, we can call set-head on symbolic-ref failure. While that's
slower operation than ls-remote, it saved in internal Git database and
can be reused on next gclient invocation.
We may be okay with hardcoding 'main' as default remote branch today,
but it's possible that some projects still use old default.
R=gavinmak@google.com
Change-Id: Ic4c826b888d96e367039bfc4b9bd2ba0d8b58b52
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5492789
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Use Path.joinpath() or the / operator instead of Path.join().
Eventually, config_types.Path will be replaced with pathlib.Path, which
doesn't have a join() method. In general, if the substitution was simple
the / operator was used. Otherwise, '.join(' was replaced with
'.joinpath('.
Change-Id: Iff2eb0abd9fdb0d3b3bca281c7c5436f932a5380
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5483206
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Rob Mohr <mohrr@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Look for both new and old paths when checking the buildtools/ dir.
Currently, there's different binaries for mac x64 and mac arm64.
Depending on the host cpu, only one is downloaded to buildtools/mac,
meaning that there is an overlap in buildtools/mac.
When we migrate to clang-format, buildtools/mac will be for mac x64
and buildtools/mac_arm64 will be for mac arm64.
- Verified locally on my macbook that the mac_arm64 path correctly
gets chosen.
Migration CL: https://chromium-review.googlesource.com/c/chromium/src/+/5484590
Bug: b/336843583,
Change-Id: I26f80dff0e39b7ae31ed5d0a1d8e436eb19fbb3d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5484138
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
The downloaded file may be a binary that needs to be executed
Bug: b/328065301, b/336843583
Change-Id: I0f6c959cbe3d5cf7e2717da72c98c51bbe98f7b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5484340
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
This version is 1.08 ± 0.04 times faster based on hyperfine results.
Bug: 40942309
Change-Id: If67e53bb778b88b0cb7c44ad86ba7438ab843ce3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5480171
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Gavin Mak <gavinmak@google.com>
The check relies on cl.GetRemoteBranch() and cl.GetRemoteUrl() that
both rely on git. Provide that info with Gerrit info from input_api
if available instead of always calling git.
Bug: 333744051
Change-Id: I915d6451d808c9bf871804d116ea884294ee7c84
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5479889
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Riley Wong <rgw@google.com>
A user was seeing an error about the output_file not existing. I was
not able to reproduce it on my workstation or sshrimp's.
Raise error if output file does not exist after downloading.
Bug: b/336509491, b/324418194
Change-Id: I4b1c2049d8369304cd2accd677f6fe2e59bcb513
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5483576
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
This is a reland of commit df8c52a549
Whats changed:
- Only edit the paths created after GetBuildtoolsPlatformBinaryPath()
- Fixed paths (the new paths needed an extra 'gn')
- Check if path isfile(), since the old buildtools/<platform>/gn will
still exist but becomes a directory
Verified locally with buildtools/linux64/gn/gn and
buildtools/linux64/gn
Original change's description:
> [depot_tools] Support both gn paths in gn.py
>
> Build in support for both buildtools/<platform>/gn and
> buildtools/<platform>/gn/gn preemptively.
> This will allow the libfuzzer builders in https://chromium-review.googlesource.com/c/chromium/src/+/5474162 to succeed.
>
> Bug: b/328065301
> Change-Id: I97b401cb1b3339cfa7962f60b891be05baac75d5
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5479888
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Bug: b/328065301
Change-Id: I54a9ae12cb51882e80823ab0c89efa0841025a9c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5482565
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
git cl upstream will run `git branch --set-upstream-to xxx` command
which modifies the git config. However, it doesn't invalidate config
cache. I think an ideal fix would be the git scm exposes method to
invalidate cache and the caller of git command needs to explicitly
invalidate the cache right after git command invocation. However,
I decide to take a "clever" approach here because this seems to
be the only place in git cl that requires explicit cache eviction.
It may not be worth to expose the existence of cache just for this
reason. The "clever" approach swaps the order of subseqent config
modification and config reading. The config modification will always
invalidate the cache. Therefore, It will ensure we are always reading
up-to-date upstream branch name from the fresh config cahce.
Bug: 40942309
Change-Id: I3786fd37799092ce269c7c4d3945c51d077335cb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5479950
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
This reverts commit df8c52a549.
Reason for revert: Found bugs
Original change's description:
> [depot_tools] Support both gn paths in gn.py
>
> Build in support for both buildtools/<platform>/gn and
> buildtools/<platform>/gn/gn preemptively.
> This will allow the libfuzzer builders in https://chromium-review.googlesource.com/c/chromium/src/+/5474162 to succeed.
>
> Bug: b/328065301
> Change-Id: I97b401cb1b3339cfa7962f60b891be05baac75d5
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5479888
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Bug: b/328065301
Change-Id: I2480e587ce314f7856edc7011fed945e01924371
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5482563
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Set the timeouts in autoninja for now, determine template rollout
problem after.
Bug: b/335525655
Change-Id: I0021b3031a9e8952acea465af94f61a4c50d774f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5477047
Reviewed-by: Ramy Medhat <abdelaal@google.com>
Commit-Queue: Michael Savigny <msavigny@google.com>
This CL changes CPEPrefix field to return None for the validate field
accessor.
There's little reason to return a special "unknown" string in property
accessor (which is used for tooling automation).
We still allow specifying "unknown" in that field.
Bug: b/321154076
Change-Id: Ib4cbc017d6b6df179ccfb008bd5ec9477913764b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5465016
Reviewed-by: Anne Redulla <aredulla@google.com>
Commit-Queue: Jiewei Qian <qjw@chromium.org>
Windows devs use luci-auth as the auth mechanism (as credshelper). By
default, credshelper-based auth is cached, but that doesn't work quite
correctly with luci-auth (it does it's own caching). It can cause
intermittent auth failures during a build, which can slow down build
performance.
Bug: b/335525655
Change-Id: Ic4ef25cfa8e0f37929170e306e8bf57e737a4a72
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5473742
Reviewed-by: Ben Segall <bentekkie@google.com>
Auto-Submit: Michael Savigny <msavigny@google.com>
Commit-Queue: Ben Segall <bentekkie@google.com>
This simplifies the administration of repositories which is currently
done via the Gerrit UI instead of the CLI.
Bug: None
Change-Id: I0af3d77fb611e0c89dc455a5a215129b5c51c5ea
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5472843
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Alexander Schulze <alexschulze@chromium.org>
config_types.Path will soon be replaced with pathlib.Path and
pathlib.Path doesn't have a join() method.
Bug: 329113288
Change-Id: I81bf607ffb41d84928af0eea06be78540b9912d9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5467697
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Auto-Submit: Rob Mohr <mohrr@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Stephanie Kim
Josip Sokcevic
Junji Watanabe
recipe-roller
Gavin Mak
Joel Hockey
mlcui
David Benjamin
Rob Mohr
Chan Li
Yiwei Zhang
Michael Savigny
Takuto Ikuta
Jiewei Qian
Joanna Wang
Alexander Schulze