Commit Graph

21 Commits (3be09c6d74ab3e69c99a4dfef5cac79d075b7142)

Author SHA1 Message Date
Junji Watanabe e68bbfc572 Prepend caffeinate to build commands on macOS
When building on macOS, it's better to avoid sleeping with caffeinate by
default. This CL prepends `caffeinate` to the build commands, which can
be opted out with `--no-caffeinate`.

Change-Id: Iacb72acb6523576305b38fe45bfef12f87121a93
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6596109
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
4 weeks ago
Arthur Milchior cd6685b6c0 [siso]Adding import
`shutil` was used in
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6420348
but not imported, leading to error message when executing
_is_google_corp_machine

Bug: None
Change-Id: I0bec8986f531a31905dabaadd20af68c1cd5536f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6449462
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Arthur Milchior <arthurmilchior@chromium.org>
3 months ago
Fumitoshi Ukai e7df27f40f siso: don't check backend.star if backend_config dir doesn't exist
Bug: 408114984
Change-Id: I354bcd7e3c0f0e6af6496054f5f295c927cfc8f0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6431647
Commit-Queue: Philipp Wollermann <philwo@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
3 months ago
Fumitoshi Ukai e1f6feb7ac siso: better error message when backend.star doesn't exist
Change-Id: I0c87b2c371f71b9777baaf2a4acc2111b30f3633
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6420348
Feels: Gary Tong <gatong@chromium.org>
Reviewed-by: Gary Tong <gatong@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
3 months ago
Mohamed Heikal 9b4d1e485d Integrate autoninja.py with fast_local_dev_server.py
- Starts the build server when the build starts.
- Writes tty filename to env variable.
- Tells the build server about the current build so it does not exit
  until autoninja does, even if idle.
- Cancels pending tasks on Ctrl+c.

Change-Id: I86bb9852bd0975f381b049b9ff21c38eef7cef9d
Bug: 370589852
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5917985
Reviewed-by: Junji Watanabe <jwata@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Mohamed Heikal <mheikal@chromium.org>
Commit-Queue: Mohamed Heikal <mheikal@chromium.org>
7 months ago
Fumitoshi Ukai f5e1092339 siso: print error if siso binary doesn't exist in third_party/siso
If build/config/siso/.sisoenv exists, but third_party/siso/cipd/siso
doesn't exist, it prints error

depot_tools/siso.py: Could not find .sisoenv under build/config/siso of the current project. Did you run gclient sync?

But it should print error as siso binary not in third_party/siso
instead.

Change-Id: I6a6a920b0e6d877551f0fa7c650ab7c8165180bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802659
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
10 months ago
Fumitoshi Ukai 06ab4cbb26 siso: call checkOutdir to prevent from running siso in ninja's out
checkOutdir should be called before run `siso ninja`
to not run `siso ninja` where `ninja` is used.

Change-Id: I55453b21d35396b59924be5398b47f67b592eca1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5802658
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
10 months ago
Fumitoshi Ukai 4eb5cc887d siso: allow run $SISO_PATH even if .sisoenv not found
When $SISO_PATH is set, allow to run it even if .sisoenv not found.
Useful for `siso fetch`, `siso recall` etc, which would not be
needed to run in chromium checkout dir.

Change-Id: If3bfcf879cc67dc41bedf17cd771c94de836c949
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5804427
Commit-Queue: Richard Wang <richardwa@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Richard Wang <richardwa@google.com>
10 months ago
Josip Sokcevic 43c6415bce Add a new possible siso path
We are relocating siso binary from third_party/siso/siso to
third_party/siso/cipd/siso to make it compatible with Cog, which
requires cipd packages to be placed in directories with no Git content.

Bug: 346837573
Change-Id: If40e6033e341dcedfee5670f07d68c14bd923f7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5625091
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Fumitoshi Ukai 951578030e siso: set PYTHONDONTWRITEBYTECODE=1 if not set
during build, lots of python scripts run and generates
lots of pycache *.pyc in workspace.
To keep workspace clean, set PYTHONDONTWRITEBYTECODE=1
in siso.py

Change-Id: I725c6745c44410eef4a780b8ffc30fdafd899334
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5606824
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
1 year ago
Fumitoshi Ukai 08d5eb6d68 Reland "siso.py check ninja marker"
This reverts siso.py in commit 6ed0d7b86c
siso.py doesn't affect `ninja -t commands`

If out dir was built by Ninja, don't build by Siso.
(still allow `siso query` etc)

Bug: b/338414465
Change-Id: I3b317ca40153cee3987390d8b62ac070b6f85243
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5524213
Commit-Queue: Andrew Grieve <agrieve@chromium.org>
Reviewed-by: Andrew Grieve <agrieve@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
1 year ago
Andrew Grieve 6ed0d7b86c Revert "Reland "siso.py: check ninja marker""
This reverts commit c9fb389f4b.

Reason for revert: Breaks ability to do "ninja -t commands"

Original change's description:
> Reland "siso.py: check ninja marker"
>
> This reverts commit b1c6497e19.
>
> Reason for revert: Fix var name
>
> ```
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/ninja
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> depot_tools/siso.py: out/ninja contains Ninja state file.
> Use `autoninja` to use reclient,
> or run `gn clean out/ninja` to switch from ninja to siso
>
> ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/siso base
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
>  4.63s init credentials
> reapi instance: projects/rbe-chrome-untrusted/instances/default_instance
>  1.63s loading fs state
>  3.42s load build.ninja
> 15.32s Regenerating ninja files
>  3.64s reloading
> build finished
> local:3 remote:0 cache:0 fallback:0 skip:3026
> reapi: ops: 17(err:1) / r:0(err:0) 0B / w:0(err:0) 0B
> fs: ops: 12092(err:192) / r:350(err:0) 189.04MiB / w:1(err:0) 13.96KiB
> 27.29s Build Succeeded: 3 steps - 0.11/s
>
> ```
>
> Original change's description:
> > Revert "siso.py: check ninja marker"
> >
> > This reverts commit 41bbfb8b90.
> >
> > Reason for revert:
> > ukai@ukai ...src/chromium/src % siso ninja -C out/siso
> > depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> > Traceback (most recent call last):
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
> >     sys.exit(main(sys.argv))
> >              ^^^^^^^^^^^^^^
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
> >     checkOutdir(args[1:])
> >   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
> >     for i, arg in enumerate(ninja_args):
> >                             ^^^^^^^^^^
> > NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
> >
> >
> > Original change's description:
> > > siso.py: check ninja marker
> > >
> > > If out dir was built by Ninja, don't build by Siso.
> > >
> > > (still allow `siso query` etc)
> > >
> > > Bug: b/338414465
> > > Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> > > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> > > Reviewed-by: Junji Watanabe <jwata@google.com>
> > > Commit-Queue: Junji Watanabe <jwata@google.com>
> > > Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> > > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
> >
> > Bug: b/338414465
> > Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
> > No-Presubmit: true
> > No-Tree-Checks: true
> > No-Try: true
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
> > Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> > Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
>
> Bug: b/338414465
> Change-Id: Idb18ad9191a062f83b538c1e2a76ee77200c4df0
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518397
> Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: I0a24db3675762b92e96ab37b629fa12b4af89240
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5522633
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Andrew Grieve <agrieve@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
1 year ago
Fumitoshi Ukai c9fb389f4b Reland "siso.py: check ninja marker"
This reverts commit b1c6497e19.

Reason for revert: Fix var name

```
ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/ninja
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
depot_tools/siso.py: out/ninja contains Ninja state file.
Use `autoninja` to use reclient,
or run `gn clean out/ninja` to switch from ninja to siso

ukai@ukai ...src/chromium/src % ~/src/depot_tools/depot_tools/siso ninja -C out/siso base
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
 4.63s init credentials
reapi instance: projects/rbe-chrome-untrusted/instances/default_instance
 1.63s loading fs state
 3.42s load build.ninja
15.32s Regenerating ninja files
 3.64s reloading
build finished
local:3 remote:0 cache:0 fallback:0 skip:3026
reapi: ops: 17(err:1) / r:0(err:0) 0B / w:0(err:0) 0B
fs: ops: 12092(err:192) / r:350(err:0) 189.04MiB / w:1(err:0) 13.96KiB
27.29s Build Succeeded: 3 steps - 0.11/s

```

Original change's description:
> Revert "siso.py: check ninja marker"
>
> This reverts commit 41bbfb8b90.
>
> Reason for revert:
> ukai@ukai ...src/chromium/src % siso ninja -C out/siso
> depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
> Traceback (most recent call last):
>   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
>     sys.exit(main(sys.argv))
>              ^^^^^^^^^^^^^^
>   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
>     checkOutdir(args[1:])
>   File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir
>     for i, arg in enumerate(ninja_args):
>                             ^^^^^^^^^^
> NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?
>
>
> Original change's description:
> > siso.py: check ninja marker
> >
> > If out dir was built by Ninja, don't build by Siso.
> >
> > (still allow `siso query` etc)
> >
> > Bug: b/338414465
> > Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> > Reviewed-by: Junji Watanabe <jwata@google.com>
> > Commit-Queue: Junji Watanabe <jwata@google.com>
> > Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> > Commit-Queue: Fumitoshi Ukai <ukai@google.com>
>
> Bug: b/338414465
> Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
> No-Presubmit: true
> No-Tree-Checks: true
> No-Try: true
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
> Reviewed-by: Philipp Wollermann <philwo@chromium.org>
> Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: Idb18ad9191a062f83b538c1e2a76ee77200c4df0
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518397
Reviewed-by: Philipp Wollermann <philwo@chromium.org>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
1 year ago
Fumitoshi Ukai b1c6497e19 Revert "siso.py: check ninja marker"
This reverts commit 41bbfb8b90.

Reason for revert: 
ukai@ukai ...src/chromium/src % siso ninja -C out/siso
depot_tools/siso.py: Using Siso binary from SISO_PATH: /usr/local/google/home/ukai/go/bin/siso.
Traceback (most recent call last):
  File "/usr/local/google/home/ukai/depot_tools/siso.py", line 115, in <module>
    sys.exit(main(sys.argv))
             ^^^^^^^^^^^^^^
  File "/usr/local/google/home/ukai/depot_tools/siso.py", line 104, in main
    checkOutdir(args[1:])
  File "/usr/local/google/home/ukai/depot_tools/siso.py", line 21, in checkOutdir   
    for i, arg in enumerate(ninja_args):
                            ^^^^^^^^^^
NameError: name 'ninja_args' is not defined. Did you mean: 'ninja_marker'?


Original change's description:
> siso.py: check ninja marker
>
> If out dir was built by Ninja, don't build by Siso.
>
> (still allow `siso query` etc)
>
> Bug: b/338414465
> Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
> Reviewed-by: Junji Watanabe <jwata@google.com>
> Commit-Queue: Junji Watanabe <jwata@google.com>
> Auto-Submit: Fumitoshi Ukai <ukai@google.com>
> Commit-Queue: Fumitoshi Ukai <ukai@google.com>

Bug: b/338414465
Change-Id: Ie69ca1b6c1d3dc47475d9356beab29a917356479
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5514902
Reviewed-by: Philipp Wollermann <philwo@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
1 year ago
Fumitoshi Ukai 41bbfb8b90 siso.py: check ninja marker
If out dir was built by Ninja, don't build by Siso.

(still allow `siso query` etc)

Bug: b/338414465
Change-Id: I48f8fbb5e83714fd3d27ff3a2bfffe929e4ef121
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5518396
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
1 year ago
Fumitoshi Ukai 9f668bbff9 siso: use stderr to print message from siso.py
siso binary will print json etc (e.g. `siso fs export`)
and usually used to pipe to jq or so.

but siso.py print message when $SISO_PATH is set
and corrupt json message.

use stderr for such message.

Change-Id: Ic35ecabc159fb2053a8a29816c276d914f8d2468
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5197566
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
1 year ago
Junji Watanabe 9ed72f91a9 [siso] Propagte SIGINT to siso process
siso's signal handler doesn't run as expected with siso/autosiso
wrappers.
This CL sets signal handlers for the Python wrappers to propagate
signals to siso.

Note that Siso immediately exists at the 2nd Ctrl-C.
So developers can stop Siso forcibly when they want.

Bug: b/308734805
Change-Id: Idd8cc1d99931d609cfc3bedfb4c0bb4319cd20b3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4996768
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
2 years ago
Mike Frysinger 124bb8e53c switch to 4 space indent
Leave the recipes/ code at 2 space to match the rest of the recipes
project in other repos.

Reformatted using:
files=( $(
	git ls-tree -r --name-only HEAD | \
		grep -Ev -e '^(third_party|recipes)/' | \
		grep '\.py$';
	git grep -l '#!/usr/bin/env.*python' | grep -v '\.py$'
) )
parallel ./yapf -i -- "${files[@]}"
~/chromiumos/chromite/contrib/reflow_overlong_comments "${files[@]}"

The files that still had strings that were too long were manually
reformatted because they were easy and only a few issues.
autoninja.py
clang_format.py
download_from_google_storage.py
fix_encoding.py
gclient_utils.py
git_cache.py
git_common.py
git_map_branches.py
git_reparent_branch.py
gn.py
my_activity.py
owners_finder.py
presubmit_canned_checks.py
reclient_helper.py
reclientreport.py
roll_dep.py
rustfmt.py
siso.py
split_cl.py
subcommand.py
subprocess2.py
swift_format.py
upload_to_google_storage.py

These files still had lines (strings) that were too long, so the pylint
warnings were suppressed with a TODO.
auth.py
gclient.py
gclient_eval.py
gclient_paths.py
gclient_scm.py
gerrit_util.py
git_cl.py
presubmit_canned_checks.py
presubmit_support.py
scm.py

Change-Id: Ia6535c4f2c48d46b589ec1e791dde6c6b2ea858f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4836379
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
2 years ago
Junji Watanabe 4480f863c5 [siso] check presence of .sisoenv file
`set([primary_solution_path, gclient_root_path, gclient_src_root_path])` doesn't guarantee order.
This is problematic when `SISO_PATH` is used because it doesn't check the presence of `third_party/siso/siso` binary.

This CL changes siso.py to check the presence of `.sisoenv` to detect the correct `base_dir`.

Change-Id: Ieb1acc44269037d43c793eb28cd2f12ad0179b17
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4625315
Auto-Submit: Junji Watanabe <jwata@google.com>
Commit-Queue: Junji Watanabe <jwata@google.com>
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
2 years ago
Richard Wang 4992184cd1 [autosiso] add SISO_PATH env var
Bug: b/287895730
Change-Id: Ibbf658f802d46a14fc82a622f7f853dcadc25e5f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4624217
Reviewed-by: Philipp Wollermann <philwo@google.com>
Commit-Queue: Richard Wang <richardwa@google.com>
2 years ago
Fumitoshi Ukai 3ca8d0d0ca add siso wrapper
Bug: b/277855544, b/277859568
Change-Id: I520834647881745db7a42fd357bc829a7c6779d8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4413517
Reviewed-by: Philipp Wollermann <philwo@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Junji Watanabe <jwata@google.com>
Commit-Queue: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Junji Watanabe <jwata@google.com>
Auto-Submit: Fumitoshi Ukai <ukai@google.com>
2 years ago