Commit Graph

161 Commits (e164056d2a152a129256ebeefdf0c5cc254bbfe4)

Author SHA1 Message Date
Nodir Turakulov fbf01c5431 remove slave_build path
slave_build path, defined by infra_paths module, is unnecessarily tied
to our buildbot setup which we are trying to deprecate in favor of LUCI.
In practice it is always == start_dir and that's what we want in LUCI too.

Update all code that uses slave_build path to use start_dir instead.

R=iannucci@chromium.org, martiniss@chromium.org
BUG=662586

Change-Id: Id52aa535d6a4f52670d581bf56aab65f670b1861
Reviewed-on: https://chromium-review.googlesource.com/412075
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
9 years ago
Nodir Turakulov 52185b3782 Update infra_paths to use START_DIR
CURRENT_WORKING_DIR in path module was renamed to START_DIR. Update
infra_paths accordingly.

This includes recipe_engine roll.

R=iannucci@chromium.org, martiniss@chromium.org
BUG=662586

Change-Id: I913c16b1a185c5b753284ed934a5e60912887f0f
Reviewed-on: https://chromium-review.googlesource.com/412055
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
9 years ago
Dan Jacques 6ecbb4ac0b Add gsutil recipe module.
Add the "gsutil" recipe module to depot_tools. This module is copied
from "tools/build"'s recipe module set at:
d6205ab0c0e8e1920f4af1617b5b3767cd944492.

Two Chromium-specific functions have been removed:
- download_with_polling
- download_latest_file

It also adds a function, "gsutil_py_path", which mimics the function in
the "depot_tools" recipe module. Ultimately, the latter will be phased
out in favor of the module-specific function.

BUG=chromium:662895
TEST=expectations

Change-Id: Ie049d637593b3f02b9ff029fbd20fd3f7170faa6
Reviewed-on: https://chromium-review.googlesource.com/411970
Commit-Queue: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
9 years ago
Dan Jacques 260ab401aa Remote nested retries on checkout.
Remove nested retries on checkout. Since the outer loop handles retries
specially, we want to let it exclusively handle retries. This will
prevent the inner retry loop triggering a timeout condition (rather than
a failure condition) with the outer loop.

BUG=chromium:665116
TEST=None

Change-Id: I4bf0c08b75861bf1c95060452118c314a121f9f3
Reviewed-on: https://chromium-review.googlesource.com/411397
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Ryan Tseng <hinoka@chromium.org>
9 years ago
agable a98a6cd47d Remove safesync support from gclient[_scm].py
R=dpranke@chromium.org

Review-Url: https://codereview.chromium.org/2395013002
9 years ago
dnj b445ef568c bot_update: Use 'gclient' from same repository.
Currently, 'bot_update' uses the 'gclient' that is on the system path.
Now, it will use the 'gclient.py' that is in the same 'depot_tools'
checkout as the 'bot_update' recipe module.

Also don't ignore "git_cache" move errors.

BUG=664254,663990,663440
TEST=None

Review-Url: https://codereview.chromium.org/2492963002
9 years ago
nodir dfe6bbbb22 swarmbucket: git_cache must be in cwd
on swarmbucket, git_cache must be in cwd

R=vadimsh@chromium.org, martiniss@chromium.org

Review-Url: https://codereview.chromium.org/2473243003
9 years ago
martiniss 5da21889e5 Revert of fix swarmbucket infra path (patchset #3 id:40001 of https://codereview.chromium.org/2477033002/ )
Reason for revert:
Broke build CQ: https://luci-milo.appspot.com/swarming/task/324a5636a5cfdf10/steps/Uncaught_Exception/0/logs/exception/0

Original issue's description:
> fix swarmbucket infra path
>
> Do not rely on buildbot config which goes up 4 directories. Everything
> should happen in the workdir. Also exclude any project-specific paths.
>
> This fixes the problem that swarmbucket builds expect git_cache to be in
> /b/git_cache. It should be in the CWD/git_cache. This will fix Windows
> builders.
>
> R=vadimsh@chromium.org
> BUG=
>
> Committed: b8b5aaea92

TBR=vadimsh@chromium.org,nodir@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=

Review-Url: https://codereview.chromium.org/2484433002
9 years ago
nodir b8b5aaea92 fix swarmbucket infra path
Do not rely on buildbot config which goes up 4 directories. Everything
should happen in the workdir. Also exclude any project-specific paths.

This fixes the problem that swarmbucket builds expect git_cache to be in
/b/git_cache. It should be in the CWD/git_cache. This will fix Windows
builders.

R=vadimsh@chromium.org
BUG=

Review-Url: https://codereview.chromium.org/2477033002
9 years ago
recipe-roller e0b205e884 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

Please review the expectation changes, and LGTM as normal. The recipe roller
will *NOT* CQ the change itself, so you must CQ the change manually.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

recipe_engine:
  https://crrev.com/2c474706779466f2e18e83a632f6d6debf63787f Fix issue overriding bug properties.test_api.tryserver for Gerrit. (tandrii@chromium.org)

R=tandrii@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2473883003
9 years ago
tandrii 9701c7c16d Recipes: Stop using tryserver_gerrit in depot_tools.
Only changes how test properties are given, no change in functionality.
Should enable recipe roll of https://codereview.chromium.org/2464673002.

R=martiniss@chromium.org,machenbach@chromium.org
BUG=chromium:645616

Review-Url: https://codereview.chromium.org/2468373002
9 years ago
katthomas 86fe47d1a0 Make bot_update on win more resilient
Sometimes Windows has trouble deleting files. This can cause problems
when lockfiles are left in .git directories.

R=agable@google.com
BUG=659178

Review-Url: https://codereview.chromium.org/2454463002
9 years ago
katthomas b7c80888a6 Log error message if os.rename fails
R=agable@google.com
BUG=659178

Review-Url: https://codereview.chromium.org/2448983004
9 years ago
borenet d918da6427 Remove apply_gerrit_ref from bot_update recipe API
BUG=skia:5588

Review-Url: https://codereview.chromium.org/2453663004
9 years ago
tandrii 51a7b096ca bot_update: understand new Gerrit patch properties.
This change is backwards compatible, so no expectation changes expected
in downstream rolls.

Depends on https://codereview.chromium.org/2442173003 recipe_engine
change.

BUG=645616
R=martiniss@chromium.org,machenbach@chromium.org

Review-Url: https://codereview.chromium.org/2439373002
9 years ago
hinoka bc0b4c6e41 Change path to internal-try rietveld credentials
To match where it ended up being located.

BUG=642150

Review-Url: https://chromiumcodereview.appspot.com/2436853002
9 years ago
katthomas df66a34f68 Remove git lockfile flakiness on win (bot_update)
Hypothesis: Sometimes bot update fails because windows fails to delete
a lockfile associated with a git process.

Test: If this happens, let's delete that lockfile and try again.

BUG=651602

Review-Url: https://codereview.chromium.org/2382653005
9 years ago
tandrii bef52ff000 git recipe_module: git new-branch.
R=iannucci@chromium.org,qyearsley@chromium.org
BUG=

Review-Url: https://codereview.chromium.org/2369073002
9 years ago
hinoka b61b9b7583 Reland bot_update: add --auth-refresh-token-json passthrough for apply_issue
Reland of https://codereview.chromium.org/2294413002/ after a rebase fix.

BUG=642150

Review-Url: https://codereview.chromium.org/2353833002
9 years ago
hinoka ad842878e4 Revert of bot_update: add --auth-refresh-token-json passthrough for apply_issue (patchset #7 id:120001 of https://codereview.chromium.org/2294413002/ )
Reason for revert:
Pointing to the wrong file :(

Original issue's description:
> bot_update: add --auth-refresh-token-json passthrough for apply_issue
>
> BUG=642150
>
> Committed: e465667e78

TBR=vadimsh@chromium.org,martiniss@chromium.org,tandrii@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=642150

Review-Url: https://codereview.chromium.org/2350363003
9 years ago
hinoka e465667e78 bot_update: add --auth-refresh-token-json passthrough for apply_issue
BUG=642150

Review-Url: https://codereview.chromium.org/2294413002
9 years ago
vadimsh 6ac12ffd59 Make bot_update.py print git version it uses.
R=iannucci@chromium.org, dnj@chromium.org
BUG=647446

Review-Url: https://codereview.chromium.org/2344313002
9 years ago
tandrii 15a248123d Revert of bot_update: ensure correct depot_tools checkout is used. (patchset #2 id:20001 of https://codereview.chromium.org/2346973003/ )
Reason for revert:
broke win_chromium_x64_rel_ng/builds/281022
because of apply_issue.bat calls local python.

Original issue's description:
> bot_update: ensure correct depot_tools checkout is used.
>
> R=vadimsh@chromium.org,martiniss@chromium.org
> BUG=635641,647446
>
> Committed: 972ac50401

TBR=martiniss@chromium.org,vadimsh@chromium.org,iannucci@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=635641,647446

Review-Url: https://codereview.chromium.org/2349673002
9 years ago
tandrii 972ac50401 bot_update: ensure correct depot_tools checkout is used.
R=vadimsh@chromium.org,martiniss@chromium.org
BUG=635641,647446

Review-Url: https://codereview.chromium.org/2346973003
9 years ago
vadimsh 18ca30ca80 Teach bot_update to remove partially deleted git repos.
Also add logging to 'remove', helps to debug issues with 'build.dead' filling
up with crap.

R=iannucci@chromium.org, hinoka@chromium.org
BUG=647046

Review-Url: https://codereview.chromium.org/2342973002
9 years ago
recipe-roller 34aa4d98f1 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

Please review the expectation changes, and LGTM as normal. The recipe roller
will *NOT* CQ the change itself, so you must CQ the change manually.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

recipe_engine:
  https://crrev.com/82c60d376b779c86f78ce98884fd4f887382e9ec Add better documentation, trigger namedtuple. (dnj@chromium.org)

R=dnj@chromium.org
BUG=None

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2335163004
9 years ago
martiniss 33061f78a1 Add recipe-roller as an OWNER of recipe modules.
See https://codereview.chromium.org/2335163004 for why this is needed. The
roller can't land a non-trivial CL with the upstream author's LGTM, because
it's not an OWNER of the recipes in the repo it's rolling into.

Review-Url: https://codereview.chromium.org/2337263002
9 years ago
recipe-roller 1ce79108a3 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).

Please review the expectation changes, and LGTM as normal. The recipe roller
will *NOT* CQ the change itself, so you must CQ the change manually.

More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug
(or complain)

recipe_engine:
  https://crrev.com/dbdcd3b5c9096b3a770bdb61e15039115b6a8dd3 Revert of Add better documentation, trigger namedtuple. (patchset #3 id:40001 of https://codereview.chromium.org/2332833003/ ) (martiniss@chromium.org)

R=martiniss@chromium.org
BUG=None

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Review-Url: https://codereview.chromium.org/2336313002
9 years ago
rmistry ea9e93f28c Skip apply_gerrit rebase when there is a rebase failure.
Not doing so causes a failure in the 'git checkout $base_rev' step as visible here: https://build.chromium.org/p/client.skia.fyi/builders/Infra-PerCommit-Trybot/builds/4995
This causes the checkout to remain in the "rebase in progress" state.

BUG=chromium:645955
BUG=skia:5749

Committed: 9736fe8938
Review-Url: https://codereview.chromium.org/2328313002
9 years ago
rmistry b940385856 Revert of Skip apply_gerrit rebase when there is a rebase failure (patchset #2 id:20001 of https://codereview.chromium.org/2328313002/ )
Reason for revert:
The "git rebase --abort" should only happen when there is a failure during rebase not when it succeeds.

Original issue's description:
> Skip apply_gerrit rebase when there is a rebase failure.
>
> Not doing so causes a failure in the 'git checkout $base_rev' step as visible here: https://build.chromium.org/p/client.skia.fyi/builders/Infra-PerCommit-Trybot/builds/4995
> This causes the checkout to remain in the "rebase in progress" state.
>
>
> BUG=chromium:645955
> BUG=skia:5749
>
> Committed: 9736fe8938

TBR=agable@chromium.org,andybons@chromium.org,tandrii@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=chromium:645955

Review-Url: https://codereview.chromium.org/2331103002
9 years ago
rmistry 9736fe8938 Skip apply_gerrit rebase when there is a rebase failure.
Not doing so causes a failure in the 'git checkout $base_rev' step as visible here: https://build.chromium.org/p/client.skia.fyi/builders/Infra-PerCommit-Trybot/builds/4995
This causes the checkout to remain in the "rebase in progress" state.

BUG=chromium:645955
BUG=skia:5749

Review-Url: https://codereview.chromium.org/2328313002
9 years ago
iannucci d1d31d5d41 A bunch more little cleanups in bot_update.
R=agable@chromium.org, estaab@chromium.org, hinoka@chromium.org, martiniss@chromium.org
BUG=643885

Review-Url: https://codereview.chromium.org/2321203003
9 years ago
kjellander 9abf453c3a WebRTC: Remove references to src/third_party/libjingle/source/talk
A while back, we removed the last pieces of code in
src/third_party/libjingle/source/talk and it is no longer synced
in Chromium's DEPS file. Clean this up to avoid confusion in the
buildbot recipes.

BUG=webrtc:4256

Review-Url: https://codereview.chromium.org/2325043002
9 years ago
agable c958da4979 Switch bot_update to use exponential backoff on clone
R=iannucci@chromium.org
BUG=472386

Review-Url: https://codereview.chromium.org/2324073002
9 years ago
iannucci 3dd7436899 Remove unused PATCH_TOOL and resulting dead code from bot_update.
R=agable@chromium.org, estaab@chromium.org, hinoka@chromium.org, martiniss@chromium.org
BUG=643885

Review-Url: https://codereview.chromium.org/2328883002
9 years ago
agable 3454ae85da Reland "Delete lots of svn logic from bot_update"
This is a reland of 082267a659
which was reverted in 4dcad385f4

BUG=472386

Review-Url: https://codereview.chromium.org/2325513002
9 years ago
iannucci 0abcd2640a Fix typobug in bot_update.py.
Extra comma changes type from dict to tuple :(

TBR=agable@chromium.org, estaab@chromium.org, hinoka@chromium.org, martiniss@chromium.org
BUG=643885
NOPRESUBMIT=true
NOTRY=true

Review-Url: https://codereview.chromium.org/2325763002
9 years ago
iannucci 16e7f097a7 Remove bogus master/builder/slave parameters from bot_update.
Bot update now always runs regardless of environmental context.

R=estaab@chromium.org, hinoka@chromium.org, martiniss@chromium.org, tandrii@chromium.org
BUG=643885

Review-Url: https://codereview.chromium.org/2308193002
9 years ago
iannucci 990ab1b42c Remove force parameter from bot_update.ensure_checkout.
This is the simplest change I can make here. The bot_update.py script already
ignores the value of force (and sets it to True in all cases). This is an
expectations-only change.

I'll refactor the content of the bot_update.py script to account for this
removed option in a followup CL as well.

R=agable@chromium.org, estaab@chromium.org, hinoka@chromium.org, martiniss@chromium.org
BUG=643885

Review-Url: https://codereview.chromium.org/2316323002
9 years ago
bpastene 4dcad385f4 Revert of Delete lots of svn logic from bot_update (patchset #5 id:80001 of https://codereview.chromium.org/2280213002/ )
Reason for revert:
Everything's purple :(

Original issue's description:
> Delete lots of svn logic from bot_update
>
> R=hinoka@chromium.org
> BUG=472386
>
> Committed: 082267a659

TBR=hinoka@chromium.org,hinoka@google.com,iannucci@chromium.org,agable@chromium.org
# Skipping CQ checks because original CL landed less than 1 days ago.
NOPRESUBMIT=true
NOTREECHECKS=true
NOTRY=true
BUG=472386

Review-Url: https://codereview.chromium.org/2320653003
9 years ago
agable 082267a659 Delete lots of svn logic from bot_update
R=hinoka@chromium.org
BUG=472386

Review-Url: https://codereview.chromium.org/2280213002
9 years ago
andybons f0b0378318 Remove gerrit_rebase_patch_ref flag from bot_update API
BUG=612417

Review-Url: https://codereview.chromium.org/2317213002
9 years ago
agable fbfa601efd Remove svn from gclient recipe module example
R=martiniss@chromium.org
BUG=640251

Review-Url: https://codereview.chromium.org/2318843002
9 years ago
iannucci 045d107c23 Remove patch_url from bot_update.
R=hinoka@chromium.org, martiniss@chromium.org, tandrii@chromium.org
BUG=643885

Review-Url: https://codereview.chromium.org/2310113003
9 years ago
iannucci 95f3d317e4 Just disable force to ensure that bot_update really is running everywhere.
This makes it so that regardless of what force is set to in the recipe, if you
invoke bot_update.ensure_checkout, it will actually run.

Removal of the force option (and related options) to come shortly.

R=estaab@chromium.org, hinoka@chromium.org, martiniss@chromium.org, tandrii@chromium.org
BUG=643885

Review-Url: https://codereview.chromium.org/2311673002
9 years ago
tandrii 4cb6fb7fbd bot_update: don't forget to apply Gerrit patches for DEPS-ed in projects.
R=machenbach@chromium.org,andybons@chromium.org
BUG=602906
TEST=manual run of Angle recipe locally

Review-Url: https://codereview.chromium.org/2299693003
9 years ago
andybons 6911b06750 Make rebase and sync the default for Gerrit updates.
BUG=602906
R=tandrii@chromium.org

Recipe-Tryjob-Bypass-Reason: tandrii@ takes responsibility.
Review-Url: https://codereview.chromium.org/2298883002
9 years ago
agable c8f3a3498e Remove non-trivial SVN config from gclient recipe module
R=martiniss@google.com
BUG=640251
Recipe-Tryjob-Bypass-Reason: can't mock expectation changes downstream

Review-Url: https://codereview.chromium.org/2285213004
9 years ago
agable 9741c590a4 Remove defunct SVN-based gclient configs
The URL helper functions were only used by this file,
the svn custom_deps urls aren't referenced in any DEPS
files I can find anymore, and the oilpan and buildspec
configs aren't used by any recipes.

R=hinoka@chromium.org
BUG=640251

Review-Url: https://codereview.chromium.org/2291583003
9 years ago
agable 7090f652d1 Remove all references to GIT_MODE from depot_tools
BUG=640251,472386

Review-Url: https://codereview.chromium.org/2286793003
9 years ago