Only hanging process recorded so far is gsutil. Sending sigterm doesn't
result in successfully terminating the process. Instead, use sigkill.
R=gavinmak@google.com
Bug: 1255228
Change-Id: Iad354284422676df804c68cb6f18c7db723bf862
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3232250
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
It's generaly not useful and makes it harder to understand bot_update
logs. If we need to debug specific builders, we can set environment
variables in the recipe itself.
R=apolito@google.com
Bug: 1255228
Change-Id: I18fc6ee8cb15b30bc3a4422ddd04979189bacf64
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3221250
Commit-Queue: Anthony Polito <apolito@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
This change starts observers as soon as possible, and wraps the entire
block that can cause an exception in try..finally. This ensures that we
stop threads are stopped.
If we notice the same behavior, it could be worth to put a cap on number
of executions per observer, and shut the thread down once reached.
R=apolito@google.com
Bug: 1255228
Change-Id: I1f165267460da02b3bbba39022c262e6c29fe86b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3213451
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
This reverts commit 8031f2ad8d.
Reason for revert: didn't help
Original change's description:
> Update HEAD when checking out from git cache
>
> HEAD may be different than what git cache has. This change ensures the
> correct revision is used.
>
> R=gavinmak@google.com
>
> Bug: 1251783
> Change-Id: I4909a4d52ef70ee806c285f24693eca5173003d9
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3174363
> Auto-Submit: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
Bug: 1251783
Change-Id: I35549e47fc7cd2361321a6e8c6480ec2732215fd
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3180383
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
HEAD may be different than what git cache has. This change ensures the
correct revision is used.
R=gavinmak@google.com
Bug: 1251783
Change-Id: I4909a4d52ef70ee806c285f24693eca5173003d9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3174363
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
If sync fails, synced_solutions is not initialized, resulting in a
use-before-init error.
Bug: 1217352
Change-Id: I2367d0f8030faa571b5c2743d5d97aa4c0329a7b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2945663
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
If checked-out revision is less than 6 hours old, skip fetching latest
revision.
Bug: 1199853
Change-Id: Ifd2a61055de286fe6fd94a5d43f80a1885a6931c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2911015
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is a reland of 856ccef0e9
Use a tempfile instead of stdout for gclient revinfo output
Original change's description:
> bot_update: Don't use gclient sync output and rely on gclient revinfo.
>
> In preparation for skipping gclient sync if there are no DEPS changes.
>
> Bug: 1199853
> Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 1199853
Change-Id: Ib50c4e943cbb51e4a2463beee9b7d1dee824ad29
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910950
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 856ccef0e9.
Reason for revert: Probably broke chrome os waterfall
Original change's description:
> bot_update: Don't use gclient sync output and rely on gclient revinfo.
>
> In preparation for skipping gclient sync if there are no DEPS changes.
>
> Bug: 1199853
> Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 1199853
Change-Id: Ib893704dffc68b02dfa7f977e751e236003b7e0b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2910949
Auto-Submit: Stephen Martinis <martiniss@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Avi Drissman <avi@chromium.org>
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Owners-Override: Avi Drissman <avi@chromium.org>
In preparation for skipping gclient sync if there are no DEPS changes.
Bug: 1199853
Change-Id: Ib9b4ab803bc574a384c661765cee5e4c1de5baae
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2912259
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
I'm not saying this is why bot_update is failing, but I never trust python threads, especially not at scale. I also think the new implementation is easier to understand and less likely to have subtle mistakes if it is extended later
Change-Id: I29be55594a400fda845ea31893789104373fd0f6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2892185
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Anthony Polito <apolito@google.com>
This is a reland of 4c67f856f0
Issues should have been fixed by crrev.com/c/2838026
Original change's description:
> git-cache: Add option to fetch commits.
>
> Add option to git cache to fetch commits.
> And use it in bot_update and gclient sync to make sure
> the needed commits are present on the checkout.
>
> Change-Id: I9e90da9e3be6e7bacf714b22bf0b735463e655b6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2829942
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Change-Id: Ie5a29737f5a75d28bc7c5c2f6cb99ec7f87cd9e8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2841046
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 4c67f856f0.
Reason for revert: crbug.com/1200657
Original change's description:
> git-cache: Add option to fetch commits.
>
> Add option to git cache to fetch commits.
> And use it in bot_update and gclient sync to make sure
> the needed commits are present on the checkout.
>
> Change-Id: I9e90da9e3be6e7bacf714b22bf0b735463e655b6
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2829942
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Change-Id: I31a1defb26119bcb46894ab4084d626b65c45fed
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2837413
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Add option to git cache to fetch commits.
And use it in bot_update and gclient sync to make sure
the needed commits are present on the checkout.
Change-Id: I9e90da9e3be6e7bacf714b22bf0b735463e655b6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2829942
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 6d0c0ffef5.
Reason for revert: this appears to break builders if reference
is a commit hash:
git log --format=%B -n1 HEAD failed with code 128
Original change's description:
> Add previous_got_revision and previous_got_revision_cp to output
>
> As chrome browser infra works to improve CQ times, we would like to
> easily glance at builds and see how many new commits were checked out.
> Currently you have to open up bot_update stdout to see the previous
> HEAD revision and go from there. Having this conveniently outputted
> would be very convenient.
>
> If the bot has previously checked out the repo, output which revision
> and commit position were checked out.
>
> example LED build showing new output: https://ci.chromium.org/swarming/task/51592e46d6a70e10?server=chromium-swarm.appspot.com
>
> Change-Id: Ib8a6f152a593446ce4082477bc9ada44f1fd1fc3
> Bug: 1170568
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644829
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
TBR=infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 1170568
Change-Id: I50d6ef6802c2eb1614cca804391c7826a79a2cf9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2657023
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
As chrome browser infra works to improve CQ times, we would like to
easily glance at builds and see how many new commits were checked out.
Currently you have to open up bot_update stdout to see the previous
HEAD revision and go from there. Having this conveniently outputted
would be very convenient.
If the bot has previously checked out the repo, output which revision
and commit position were checked out.
example LED build showing new output: https://ci.chromium.org/swarming/task/51592e46d6a70e10?server=chromium-swarm.appspot.com
Change-Id: Ib8a6f152a593446ce4082477bc9ada44f1fd1fc3
Bug: 1170568
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2644829
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
If set, set git config fetch.uriprotocols to https. This will be
initially set by codesearch builders. Once tested, this env variable can
be removed.
R=ehmaldonaldo@chromium.org
Bug: 1136986
Change-Id: I1c3da87bdbc74d18585ed00ffdc2cf21b84b16fa
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2591808
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
The enforce_fetch kwarg was being ignored. This change adds the
--enforce_fetch to the resources/bot_update cmd.
Confirmed here through led: b852fa5e4bb77d6fb2aa69ec87ee57b5d2/+/steps/bot_update/0/logs/execution_details/0
Bug: 1129235
Change-Id: I78c5d62841fda6dafbb709301e26c5047df2d58d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2444114
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
This is a reland of 3774c589cb
Added missing options.enforce_fetch arg
Confirmed via a fake test roll with this CL's patch commit that the
bot_update step passes with this change:
https://chromium-review.googlesource.com/c/chromium/tools/build/+/2443649
Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Bug: 1129235
Change-Id: Ibbd4535f8529243d2dc1d4d32881d8a61efe374a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443574
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
This reverts commit 3774c589cb.
Reason for revert: ea31a9d6c2771701fcaa11b47a2d735c19/+/steps/bot_update/0/stdout Going to revert and test with led before landing another "fix"
Original change's description:
> Reland "Add optional enforce_fetch kwarg to fetch and refresh git cache"
>
> This is a reland of f38f54f2f1
>
> Original change's description:
> > Add optional enforce_fetch kwarg to fetch and refresh git cache
> >
> > Allows builders to do a force refresh of the git cache to ensure a new
> > fetch happens. android-binary-size needs this to get an accurate diff
> > of the passed in src revision and the applied patch:
> > https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
> >
> > Bug: 1129235
> > Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Josip Sokcevic <sokcevic@google.com>
>
> Bug: 1129235
> Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
TBR=tandrii@google.com,ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com
Change-Id: I305215eed4128b13c9b5b524f4cbf3203037d582
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443573
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
This is a reland of f38f54f2f1
Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Bug: 1129235
Change-Id: I034a91a8b57821d08b10042f7fc94d32350540ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443235
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
This reverts commit f38f54f2f1.
Reason for revert: Seeing errors about missing args 007f2e16445872d9237cec0f72bb239fd6/+/steps/bot_update/0/stdout
Original change's description:
> Add optional enforce_fetch kwarg to fetch and refresh git cache
>
> Allows builders to do a force refresh of the git cache to ensure a new
> fetch happens. android-binary-size needs this to get an accurate diff
> of the passed in src revision and the applied patch:
> https://bugs.chromium.org/p/chromium/issues/detail?id=1129235#c19
>
> Bug: 1129235
> Change-Id: I993521cdde2a491fc9d42d75ff28dedcd2cea64e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2438802
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
TBR=tandrii@google.com,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com,kimstephanie@google.com
Change-Id: Ib867742c624f085d3dd56422c2d63c15fdc84f06
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1129235
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2443231
Reviewed-by: Stephanie Kim <kimstephanie@google.com>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Map refs to remote refs when fetching refs from origin
(e.g. refs/heads/master -> refs/remotes/origin/master)
so that a detached HEAD is not necessary.
Bug: 1104182
Change-Id: Icb751505de1bf443e5d45c4b37425223dbf6e088
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2310849
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit b99e61c58a.
Reason for revert:
Doesn't work for refs/branch-heads/*
Original change's description:
> bot_update: Refactor branch to detach and checkout.
>
> Refactor _git_checkout to use information from
> get_target_branch_and_revision to know what branch to checkout after
> cloning.
>
> Bug: 1104182
> Change-Id: Ib3ba57ca0b6803f172b85121c2a4b123f17bfb8c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2303291
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=ehmaldonado@chromium.org,infra-scoped@luci-project-accounts.iam.gserviceaccount.com,sokcevic@google.com
Change-Id: If51423df99fd9c164f8e42e0220ee2d9bc2a39f6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 1104182
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2310293
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Refactor _git_checkout to use information from
get_target_branch_and_revision to know what branch to checkout after
cloning.
Bug: 1104182
Change-Id: Ib3ba57ca0b6803f172b85121c2a4b123f17bfb8c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2303291
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Checkout 'HEAD' instead. This is needed to have a detached HEAD, so we
can fetch refs/heads/main:refs/heads/main, where main is the default
branch.
Bug: 1104182
Change-Id: I7b7457b21b7d98d29583555b62e27e96e900adb4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2300533
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
File marker '.dirty_bot_checkout' indicates that bot_update checkout has
started. Such marker is removed on successful checkout.
If such marker is present before creation, it means previous run wasn't
successful and may have left workspace in a bad state. In that case,
delete working directory and continue with a clean checkout.
R=apolito@google.com, ehmaldonado@chromium.org
Bug: 1074260
Change-Id: Ic089eedc26f7c269d5342fae366899571d8f8d41
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2186273
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Anthony Polito <apolito@google.com>
Josip Sokcevic
Anthony Polito
Gavin Mak
Takuto Ikuta
Edward Lesmes
Stephen Martinis
Edward Lesmes
Stephanie Kim
Quinten Yearsley