Detect failure of scm.GIT.RefToRemoteRef and raise an error before
attempting to join its results as a string. This makes certain
repository errors diagnosable.
Fixed: 1470430
Change-Id: I19cd86d9ed088f5cd511e4c9d7dbe5844165c7cf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4754237
Auto-Submit: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
On NOP or incremental builds the reproxy startup/shutdown times can be a
significant portion of the total build time and it is important to
acknowledge these times.
Bug: b/294443556
Change-Id: Ic63331683632c8f447858b498f53d254b56ffe89
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4753472
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This info is only used when bootstrapping for the first time,
so move these vars to that function to avoid wasted effort.
Bug: 1469696
Change-Id: I21b97b2429cf2dac6242e2e86aa7668b09ed47c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4752396
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Avoid recomputing this value multiple times between vpython & cipd.
Bug: 1469696
Change-Id: If82d58180f94cb2833a5a177f7b1e431e08f2892
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4750040
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
This reverts commit 43083529de.
Reason for revert: crbug.com/1470122
Original change's description:
> New vpython to depot_tools
>
> This is a reland of https://crrev.com/c/4653897
> Fixed the issue for cipd wrapper. Now all environment variables should
> be perserved when invoking cipd: https://crrev.com/c/4669637
>
> 1. virtualenv field in the spec is ignored.
> 2. --vpython-tool removed support for delete and help subcommands.
> 3. --vpython-tool installed removed support for naming venv.
> 4. removed support for -vpython-interpreter.
> 5. removed support for searching interpreters in host PATH.
> 6. python 2.7 is available only if the binary is invoked as `vpython`.
> 7. fixed a bug that passes invalid vpython arguments to the script,
> which may be silently ignored.
> 8. python_version in the vpython_spec must specify a minor version
> (python_version: "3" is not valid anymore).
> 9. vpython now requires the cipd binary to be present in PATH, which is
> true already when using Swarming or depot_tools.
>
> Also updates the LUCI_OWNERS to add peep-software-deploy team.
>
> Bug:1415212
> Change-Id: Ie541a2a60bef829a976a13db9a6732b406c4d878
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4719827
> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
> Auto-Submit: Chenlin Fan <fancl@chromium.org>
> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Bug: 1415212
Change-Id: I6ca32066acd977a293f8b8f42697c383cc2a93fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4751179
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Chenlin Fan <fancl@chromium.org>
A lot of scripts like to compute the dirname of $0 to find itself to
load more files. Have some scripts export & resuse this value so we
avoid recalculating when not needed.
Bug: 1469696
Change-Id: I3aac2e6119dbfa3c68715208df2786f67cdce92e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4750430
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Other depot_tools scripts use dirname regardless of mingw,
so do that here too to simplify.
Bug: 1469696
Change-Id: Ia3c22a4eb72e62564eee1e8b18a88671c8b8b9a5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4748852
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
The chromium-style plugin was replaced last year with cl/495068108.
Change-Id: Ifce7cf5e22f10681ffe743572182d759ae51fed2
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4743665
Auto-Submit: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Other depot_tools scripts use dirname regardless of mingw,
so do that here too to simplify.
Bug: 1469696
Change-Id: I9a70975ef5614b1107ac87a5dc29b5ee1a06e298
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4750039
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Mike Frysinger <vapier@chromium.org>
This was added long ago to workaround svn behavior with symlinked dirs.
We haven't used svn in years, so drop this logic to simplify.
Bug: 1469696
Change-Id: Ic881ec74b1a3caa002a131be48259a56be30d4f4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4745419
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
This CL stops "git cl split" from printing which directories could
not be uploaded if upload fails. This information is redundant with
previous lines printed by "git cl split"
Bug:None
Change-Id: I39bc47e3b29d4ce777a65b6bdac28b19b672339f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4743774
Commit-Queue: Peter Kotwicz <pkotwicz@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
When running on macOS under python3 as part of PRESUBMIT_test.py,
a ResourceWarning is printed. This is due to the `open(..).read()`
idiom.
Use a `with` statement to ensure the file is properly closed.
Bug: none
Change-Id: I5643db0b77e6896b4b86177d2c1fe8dea04b58ae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4742422
Reviewed-by: Gavin Mak <gavinmak@google.com>
Auto-Submit: Sylvain Defresne <sdefresne@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
This is a reland of https://crrev.com/c/4653897
Fixed the issue for cipd wrapper. Now all environment variables should
be perserved when invoking cipd: https://crrev.com/c/4669637
1. virtualenv field in the spec is ignored.
2. --vpython-tool removed support for delete and help subcommands.
3. --vpython-tool installed removed support for naming venv.
4. removed support for -vpython-interpreter.
5. removed support for searching interpreters in host PATH.
6. python 2.7 is available only if the binary is invoked as `vpython`.
7. fixed a bug that passes invalid vpython arguments to the script,
which may be silently ignored.
8. python_version in the vpython_spec must specify a minor version
(python_version: "3" is not valid anymore).
9. vpython now requires the cipd binary to be present in PATH, which is
true already when using Swarming or depot_tools.
Also updates the LUCI_OWNERS to add peep-software-deploy team.
Bug:1415212
Change-Id: Ie541a2a60bef829a976a13db9a6732b406c4d878
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4719827
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Auto-Submit: Chenlin Fan <fancl@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
This CL prevents split_cl.py from uploading multiple CLs for the same
reviewer set.
BUG=1468350
Change-Id: I9c328589f7facfe10ee5066cc3d1cda007dd1d2a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4726781
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Peter Kotwicz <pkotwicz@chromium.org>
CrOS intends to provide a Bazel executable for our users in
chromite/bin/bazel in our tree. We'd like the "bazel" command in
depot_tools to call this executable.
This adds a new launcher to depot_tools which searches for that bazel
executable when located inside of a ChromiumOS checkout, and executes
it. When located outside of a ChromiumOS checkout, this launcher
"disappears", searching elsewhere in the PATH for another Bazel
executable.
Since other teams using depot_tools may want to start using Bazel in
the future, this launcher is intended to have shared ownership: other
teams are welcome to come add their search functions to the launcher
if they require the same functionality as us.
Bug: b:253268519
Change-Id: I61f6383d8b69b9eea622f37277678f898cc7fd6b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4718785
Reviewed-by: Shuhei Takahashi <nya@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Aaron Massey <aaronmassey@google.com>
Commit-Queue: Jack Rosenthal <jrosenth@chromium.org>
Auto-Submit: Jack Rosenthal <jrosenth@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
This makes hooks path behavior match deps path behavior.
Bug: 1468658
Change-Id: Ib6685cb52679249b389599ada549e2c4410e812c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4729084
Auto-Submit: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
On macOS the default limit on open file descriptors is really low
and causes ninja to fail with 'Too many open files' if run with a
value of -j greater than 200.
Running only 200 jobs however slow down the build considerably so
check whether the limit has been raised by the user or if it can
be raised programmatically. In the positive case, use a limit of
800 jobs, otherwise settle down on 200 jobs.
This should allow running ninja with a large number of jobs even
on macOS Ventura 13.5 which now requires to set the limit both in
/Library/LaunchDaemons/limit.maxfiles.plist and via ulimit -n.
Bug: 1467777
Change-Id: Ib8b7d0d1ee47d243c1872229c5340e7795c1b42e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4725183
Auto-Submit: Sylvain Defresne <sdefresne@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@google.com>
Commit-Queue: Sylvain Defresne <sdefresne@chromium.org>
The limit set in /Library/LaunchDaemons/limit.maxfiles.plist appears
to no longer be respected in macOS Ventura 13.5 and the OS forces a
limit of 256 file descriptors, causing build with autoninja to fail
on macOS running this most recent version of the OS.
Force the max number of process to 200 until a new working way to
increase the file descriptor limit is found. This will allow devs
working on macOS to be able to build (albeit slower than before).
Fixes the following build failure when goma is enabled and running
on macOS Ventura 13.5:
$ autoninja -C out/Debug-iphonesimulator chrome
ninja: Entering directory `out/Debug-iphonesimulator'
[0/3574] CXX ....oninja: fatal: pipe: Too many open files
Bug: 1467777
Change-Id: Ia7eaab552f7e6d26a2f48d72bb8235a70d6d442f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4720227
Reviewed-by: Dirk Pranke <dpranke@google.com>
Auto-Submit: Sylvain Defresne <sdefresne@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@google.com>
local_resource_fraction is set to 0.2 as goma limited the max local
racing subprocesses to num_cpu/5
racing_bias is set to 0.95 as experimentally that ensures reasonable
behaviour for cache hits.
This cl does not enable racing by default. These values are only used if
RBE_exec_strategy=racing is set
Test: Tested builds on my mac and linux cloudtop and got mostly remote exec
and cache hits with some local executions
Bug: b/288564971
Change-Id: Idb71c03f4addd23d0253dc5bf3564058cd212284
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4713186
Commit-Queue: Ben Segall <bentekkie@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
This is caused by the previous reproxy instance still having the log files open for writing. Bootstrap will kill that instance on startup of the next build but that only happens after we try and create the log dir
Bug: b/290230382
Change-Id: Idd88cfa9c571c0bca078f8bf0078fdb3941603ca
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4670328
Commit-Queue: Ben Segall <bentekkie@google.com>
Reviewed-by: Junji Watanabe <jwata@google.com>
Auto-Submit: Ben Segall <bentekkie@google.com>
mdproxy/bin/prodcertstatus confuses the automatic auth logic in reproxy
This will be fixed in the next release but for now we remove mdproxy
from the path for autoninja builds
Test: Installed mdproxy locally and verified that gcert is not called
Change-Id: I0912a3c8b00a0cda41a85a16f51b8f54b20f1974
Bug: b/292523514
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4712465
Auto-Submit: Ben Segall <bentekkie@google.com>
Reviewed-by: Ramy Medhat <abdelaal@google.com>
Commit-Queue: Ben Segall <bentekkie@google.com>
This was approved in the PDD for reclient metrics
Bug: b/290921206
Change-Id: Ib9a38f1143d3a05e76ce75fc4bc6e9299d4dafb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4705043
Reviewed-by: Andy Perelson <ajp@google.com>
Auto-Submit: Ben Segall <bentekkie@google.com>
Commit-Queue: Ben Segall <bentekkie@google.com>
The previous change to git-freeze added an accidental regression
where running `git freeze` from somewhere other than the repo root
would return a mysterious error:
"Failed to index some unindexed files. Nothing to freeze."
This is because `git add` always treats pathspecs as relative to the
current working directory. Fix this by changing `git add` to always
run from the repo root.
R=ajp@google.com
Change-Id: I451f26fe35a5c6e9f3b917a1d90bdadc7065244c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4703128
Commit-Queue: Andy Perelson <ajp@google.com>
Reviewed-by: Andy Perelson <ajp@google.com>
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
This reverts commit 3aa7489c3e.
Reason for revert: reproxy 0.110.0 includes a flag for fixing hardlink issue
Original change's description:
> Revert "reclient: Enable racing for chrome developers"
>
> This reverts commit 24ac2543c7.
>
> Reason for revert: hard links dont work on tmpfs b/288997481
>
> Original change's description:
> > reclient: Enable racing for chrome developers
> >
> > racing_bias=0.95 is a best guess based on benchmarks on my glinux and gwindows cloudtops, and my gmac laptop. I will monitor build performance data and adjust the bias based on that.
> >
> > Bug: b/288285261
> > Change-Id: Ie8652281f7bc815e64978bcd56a75e5ec28e2132
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4633180
> > Commit-Queue: Ben Segall <bentekkie@google.com>
> > Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
>
> Bug: b/288285261
> Change-Id: I7b262f63d4edd68447fcec0a781ef55923635d2c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4651968
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Ben Segall <bentekkie@google.com>
Bug: b/288285261
Change-Id: I8fb65c32adc7b22a747c30f408f1f34b2bb5f8f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4693625
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Ben Segall <bentekkie@google.com>
Kenneth Russell
Bruce Dawson
Arthur Milchior
Mike Frysinger
Junji Watanabe
Chenlin Fan
Gavin Mak
Ben Segall
Peter Kotwicz
Vadim Shtayura
Sylvain Defresne
Jack Rosenthal
recipe-roller
Struan Shrimpton
Joanna Wang
Anne Redulla
Chan Li
Robert Iannucci