Commit Graph

1563 Commits (ad39f9d8f8d117ef63fc7d50b207a14017f05c08)

Author SHA1 Message Date
Edward Lemur 488362624b metrics: Add a mechanism to notify users when we want to collect additional metrics.
When we change the version number in metrics_utils:

If the user is not a Googler, or has opted out explicitly, nothing happens
and we still don't collect metrics.

If we're collecting metrics from the user, we stop collecting metrics
and display a notice telling them what has changed.
That notice will be displayed ten times, after which we will
resume collecting metrics. A notice telling them we're collecting metrics
will still be displayed.

Bug: None
Change-Id: If1cc12b2fc06f0d6237714c4f182367b1afdf9fb
Reviewed-on: https://chromium-review.googlesource.com/c/1285395
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov c68a1753c5 git cl try-results: delete Rietveld support.
R=ehmaldonado

Bug: 770408
Change-Id: I94ff37f198937dc7d341a859a71becd91cf3a706
Reviewed-on: https://chromium-review.googlesource.com/c/1279140
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov dd672fb3c5 git cl description: rename gerrit test.
R=ehmaldonado

Bug: 770408
Change-Id: I211ad8e67e9486b974eed7694436afb15ecf21a2
Reviewed-on: https://chromium-review.googlesource.com/c/1279136
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov 642641d030 git cl comments: remove Rietveld support.
R=ehmaldonado

Bug: 770408
Change-Id: Ib3120b5fd09c35be691c025d0d69ceda60f2ec7f
Reviewed-on: https://chromium-review.googlesource.com/c/1279135
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov feec80e971 git cl: no longer support --rietveld flag when forcing codereview.
R=ehmaldonado

Bug: 770408
Change-Id: I980487aacd115535d0ca855cd1edfcfc18fc5cbe
Reviewed-on: https://chromium-review.googlesource.com/c/1279138
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov 2f727917ac git cl: add reviewers and ccs to git push command if possible.
Also fix a typo in ValidateEmail function which didn't support
email addresses with '-' in them, e.g., infra-dev@chromium.org.

R=ehmaldonado

Bug: 875089
Change-Id: I2d73c1473527c9bf62e25e9f88250196b783fcb0
Reviewed-on: https://chromium-review.googlesource.com/c/1242849
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov 9f274436bd git cl: partially remove Rietveld support for uploading code.
Also deletes lots of tests which provided coverage for Rietveld only,
and hence no longer useful.

R=ehmaldonado

Bug: 770408
Change-Id: I31195f7819a52d1063ed28064a74fd70fbc39357
Reviewed-on: https://chromium-review.googlesource.com/c/1279133
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov 03da150665 git cl: delete Rietveld support for tryjobs.
R=ehmaldonado

Bug: 770408
Change-Id: Id070e6084b6beba7309bdc35f0f750decd77e0c9
Reviewed-on: https://chromium-review.googlesource.com/c/1279132
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov f5569d2608 git cl: start deleting Rietveld support.
R=ehmaldonado

Bug: 770408
Change-Id: I5020596ada0c6fef2797fed6b78e7256a35ffc40
Reviewed-on: https://chromium-review.googlesource.com/c/1279131
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov ba7b0a466d git cl: find which reviewers and ccs are OK for Gerrit before upload.
This adds O(reviewers+cc) RPC to Gerrit before upload,
though done concurrently in up to 10 threads. This information isn't
used yet, but will be in follow up CLs.

R=ehmaldonado

Bug: 877717
Change-Id: I2374f249ee874a71089244309e50e8e88a3dee7d
Reviewed-on: https://chromium-review.googlesource.com/c/1242847
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 76988a8c52 git cl: compute reviewers and ccs before pushing change to Gerrit.
R=ehmaldonado

Bug: 877717
Change-Id: Iccfba594369d8c726fd9b077d3ac18046757e4a2
Reviewed-on: https://chromium-review.googlesource.com/c/1242845
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 0c3bd49069 bot_update: Don't use apply_patch_on_gclient.
It has been True by default for a while, and there is no need to override it.

Bug: 891917
Change-Id: I2598a2230b0ea38a647a533757331c541b871971
Reviewed-on: https://chromium-review.googlesource.com/c/1260057
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Eric Boren 2fb6310237 Reland gerrit_util: Support OAuth2 bearer tokens in CookieAuthenticator
Bug: skia:8394
Change-Id: I928eaec8459a5905360ce760c6361e4554852b44
Reviewed-on: https://chromium-review.googlesource.com/c/1261796
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Eric Boren <borenet@chromium.org>
7 years ago
Andrii Shyshkalov d9fdc1f5b5 git cl: save description before doing any RPCs during upload.
R=smut@google.com

Bug: 881860
Change-Id: I59c337384891f02dbd2cc776318803ac556843ca
Reviewed-on: https://chromium-review.googlesource.com/1247584
Reviewed-by: smut <smut@google.com>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov c4a7356a46 git cl: cache also LABELS data on a change.
This removes 1 RPC when tbr-ing second or later patchset.

Since tests cover TBR only in first upload, there are
no test changes. Since adding test is real PITA here,
keep them as is.

R=ehmaldonado

Change-Id: I90801b06727ce2131a703d6cd6c24ffad798bc8a
Reviewed-on: https://chromium-review.googlesource.com/1242212
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 8e64369342 git cl: delete old support code for Git Numberer with Rietveld.
R=ehmaldonado

Change-Id: Ie5522283e84bdec9d98e236577eaad568816519b
Reviewed-on: https://chromium-review.googlesource.com/1242211
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov d4c867355a git cl: use explicit Gerrit mirrors on 404s during upload.
This will cycle through known gerrit mirrors for chromium-review host
if Gerrit HTTP RPC results in 404 after successful initial
git push refs/for/refs/...

Tested locally by intentionally using wrong change number (2000000):

    $ PATH=`pwd`:$PATH git cl upload --bypass-hooks
        ...
        remote: Processing changes: refs: 1, new: 1, done
        remote: SUCCESS
        remote: New Changes:
        remote:   https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1227440

                * [new branch]        ea9aea5faa -> refs/for/refs/heads/master%wip,m=Initial_upload,hashtag=git-cl

        WARNING:root:404 NotFound error occurred while querying POST https://chromium-review.googlesource.com/a/changes/chromium%2Ftools%2Fdepot_tools~2000000/revisions/current/review: Not Found
        WARNING:root:404 NotFound error occurred while querying POST https://ap1-mirror-chromium-review.googlesource.com/a/changes/chromium%2Ftools%2Fdepot_tools~2000000/revisions/current/review: Not Found
        WARNING:root:404 NotFound error occurred while querying POST https://us1-mirror-chromium-review.googlesource.com/a/changes/chromium%2Ftools%2Fdepot_tools~2000000/revisions/current/review: Not Found

        ^C

Bug: 881860
Change-Id: Iac7dbe4e35052007650a7a2646a394caed6bd400
Reviewed-on: https://chromium-review.googlesource.com/1227441
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 21a51b3921 gclient: set the push url to the actual repository instead of the mirror.
Bug: 884434
Change-Id: Id156709e14d1c3506a1236bc1e59f8468381a402
Reviewed-on: https://chromium-review.googlesource.com/1232257
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 647e1e79eb Reland "gclient: delete unversioned directory before adding cipd dep for the same path"
This is a reland of 67ef3f67e8

We no longer call cipd unconditionally.

Original change's description:
> gclient: delete unversioned directory before adding cipd dep for the same path
>
> Bug: 882611
> Change-Id: I46e41cc9693b90874b5d6569a12ec638eaac1050
> Reviewed-on: https://chromium-review.googlesource.com/1228655
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

Bug: 882611
Change-Id: I683bfc62bd1eebfec0853583f96f3981c2c6bdf2
Reviewed-on: https://chromium-review.googlesource.com/1232891
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov 07b5283a4e Revert "gclient: delete unversioned directory before adding cipd dep for the same path"
This reverts commit 67ef3f67e8.

Reason for revert: might have broken GCE/ccompute fleet.

Original change's description:
> gclient: delete unversioned directory before adding cipd dep for the same path
> 
> Bug: 882611
> Change-Id: I46e41cc9693b90874b5d6569a12ec638eaac1050
> Reviewed-on: https://chromium-review.googlesource.com/1228655
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>

TBR=tandrii@chromium.org,ehmaldonado@chromium.org

Change-Id: Ia2172b1b0d236357edf93dacf57f3bf7d80d5968
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 882611
Reviewed-on: https://chromium-review.googlesource.com/1231699
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Edward Lemur 67ef3f67e8 gclient: delete unversioned directory before adding cipd dep for the same path
Bug: 882611
Change-Id: I46e41cc9693b90874b5d6569a12ec638eaac1050
Reviewed-on: https://chromium-review.googlesource.com/1228655
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Sergiy Byelozyorov 1aa405fd85 Add an option to disable adding CC emails automatically
R=ehmaldonado@chromium.org, machenbach@chromium.org

Bug: 878303
Change-Id: Ia90001025babe692e481eb74fdbb4a66ce9a22f8
Reviewed-on: https://chromium-review.googlesource.com/1230074
Commit-Queue: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Eric Boren 0526335226 Revert "gerrit_util: Support OAuth2 bearer tokens in CookieAuthenticator"
This reverts commit 18b4479cee.

Reason for revert: Breaking WebRTC roller: https://luci-milo.appspot.com/buildbot/client.webrtc.fyi/Auto-roll%20-%20WebRTC%20DEPS/12788

Original change's description:
> gerrit_util: Support OAuth2 bearer tokens in CookieAuthenticator
> 
> Change-Id: I9fd3e2fa9ea16d7f057290e77a81f7009c7c9d91
> Reviewed-on: https://chromium-review.googlesource.com/1221826
> Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Eric Boren <borenet@chromium.org>

TBR=vadimsh@chromium.org,dnj@chromium.org,borenet@chromium.org,tandrii@chromium.org

Change-Id: I93bbd2d09cd2b27d9a7aa98fc15d60c5033a38cd
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/1231433
Reviewed-by: Eric Boren <borenet@chromium.org>
Commit-Queue: Eric Boren <borenet@chromium.org>
7 years ago
Eric Boren 18b4479cee gerrit_util: Support OAuth2 bearer tokens in CookieAuthenticator
Change-Id: I9fd3e2fa9ea16d7f057290e77a81f7009c7c9d91
Reviewed-on: https://chromium-review.googlesource.com/1221826
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Eric Boren <borenet@chromium.org>
7 years ago
Leszek Swirski 45b20c4140 [git_cl] Fix git cl comments json stdout printing
Print "git cl comments" json to stdout when the output file is "-"

Change-Id: I91021317e52019a0d499cccc4f40f062127b300e
Reviewed-on: https://chromium-review.googlesource.com/1228094
Auto-Submit: Leszek Swirski <leszeks@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Vadim Shtayura 2174136d25 [cipd] Add presubmit support for checking *.digests file.
Not enabling it yet, since PRESUBMIT.py uses presubmit_canned_checks.py from
depot_tools at HEAD, not from a CL (thus enabling this check in this CL will
make it fail the presubmit).

R=nodir@chromium.org
BUG=870166

Change-Id: I97802ec37f1f7513dfca3950f7f38a5c51ab0350
Reviewed-on: https://chromium-review.googlesource.com/1227432
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
7 years ago
Corentin Wallez a68660d0bb gclient: add use_relative_hooks
When a recursive dependency has use_relative_paths it also makes sense
to have the hooks working directory by the dependency's directory.
Otherwise if a hook uses one of the relative dependencies it is impossible
to know which path prefix to use.

However we cannot change the behavior of hooks with use_relative_paths
because it would break existing projects that use_relative_paths but
hardcoded the prefix for hooks. Instead we add a second boolean,
use_relative_hooks that triggers the behavior.

Adds tests for the new behavior and a test for existing interactio
between hooks and recursedeps.

BUG=chromium:875245

Change-Id: Ie4c526baa425ff887b3be54e0feca7c597ededec
Reviewed-on: https://chromium-review.googlesource.com/1213327
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Corentin Wallez aca984cbf1 Add test for a weirdness of gclient use_relative_paths and recurse_deps
BUG=

Change-Id: I1f8be18b821d45564bbabbd4e90eb2624d51a3f9
Reviewed-on: https://chromium-review.googlesource.com/1213582
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
7 years ago
Corentin Wallez 82849ccc5a gclient_test: use write() for hook tests
BUG=

Change-Id: I7561d252e10ae9b376e1b61ea3846cc8fc08f472
Reviewed-on: https://chromium-review.googlesource.com/1213323
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Corentin Wallez <cwallez@chromium.org>
7 years ago
Daniel Cheng 74fda7147c [owners.py] require stricter naming conventions for file: includes
Files that control ownership have special ownership rules, to disallow
someone from self-approving a CL that adds themselves to a file that
controls ownership. owners.py now requires that they be named OWNERS or
end with a suffix of _OWNERS to ensure the special ownership rules are
correctly applied.

Bug: 801315
Change-Id: I083a2d15bdc9c749e3838fb1c983286d5a7c4af9
Reviewed-on: https://chromium-review.googlesource.com/1204917
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Daniel Cheng <dcheng@chromium.org>
7 years ago
Andrii Shyshkalov 889677c044 git cl: now fully use project~changenum when querying Gerrit.
R=ehmaldonado

Bug: 876910
Change-Id: Ibc631bf049440d158de757da0a225ed28f892891
Reviewed-on: https://chromium-review.googlesource.com/1194314
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 03e0ed26ea git cl: use project~change_number when querying Gerrit for change Info.
R=ehmaldonado

Bug: 876910
Change-Id: I218a8198d06e51ac2eb57636bbccca9aadc7d81a
Reviewed-on: https://chromium-review.googlesource.com/1194312
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Edward Lemur 5cc2afd9b8 gclient_eval: Handle non-string variables.
Bug: 877902
Change-Id: I77a1f213318d1e09ae050c2ea0113b2976bd259e
Reviewed-on: https://chromium-review.googlesource.com/1191928
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov f57841b78b git cl patch: cleanup tests.
* remove tests which weren't actually testing any new code path.
* remove Rietveld tests, except 1.
* change default to Gerrit.

R=ehmaldonado

Bug: 719954
Change-Id: I27e59036bbafb870cadc4d6208d21ba17c48c8c0
Reviewed-on: https://chromium-review.googlesource.com/1188992
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 2d0e03cb5d git cl: safe fallback for Gerrit RPCs even if gerrit project is unknown.
R=ehmaldonado

Bug: 876910
Change-Id: I258efa0a95f20b80819bc89cccfb5e0a29e09d5c
Reviewed-on: https://chromium-review.googlesource.com/1188989
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov 1e82867e3d git cl: restart using project~number on Gerrit.
This fixes Gerrit project detection based on remote URL,
accounting for potential 'a/' prefix in the URL path component,
which isn't part of the Gerrit project name.

R=ehmaldonado, mmoss

Bug: 876964, 876910
Change-Id: I473ae8c6c9e0f2034b350901abd67db151e0a3d3
Reviewed-on: https://chromium-review.googlesource.com/1187573
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 6365bc4446 git cl: temporary stop using project~number on Gerrit.
Broke release process.

R=ehmaldonado, mmoss

Bug: 876964, 876910
Change-Id: I02ea424632f5c5522af0010adce1c993e2610b48
Reviewed-on: https://chromium-review.googlesource.com/1187548
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 81db1d5032 git cl: cache GetRemoteUrl result.
Sadly, this makes maintaining one test handling covering
exceptional circumstance very complex, so it was removed.

I've decoupled test that ensures that GetRemoteUrl works from
upload codepath.

R=ehmaldonado@chromium.org

Bug: 876910
Change-Id: I39de410c72d893e73492d5c3fc8f60a6ebc4f11f
Reviewed-on: https://chromium-review.googlesource.com/1186142
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 0ec9d15571 git cl: use project~number on Gerrit for better routing when setting reviewers
R=ehmaldonado@chromium.org

Testing
  patched my own depot_tools in $PATH and uploaded
  https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1186072
  which set reviewers and ccs as expected.

Bug: 876910
Change-Id: I43c0f2284941cf703133bb51132226d4a0472d8e
Reviewed-on: https://chromium-review.googlesource.com/1186068
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov b7214602e3 git cl: remove unused issue kwarg.
R=ehmaldonado

Change-Id: I5be27cb025f8237cef0186c90eb7b99d000076d9
Reviewed-on: https://chromium-review.googlesource.com/1185986
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 7de5f08c45 bot_update: remove --gerrit_repo and --gerrit_ref.
Bug: 865882
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: release_scripts
Recipe-Nontrivial-Roll: skia
Recipe-Nontrivial-Roll: skiabuildbot
Change-Id: I4a04c4c28c7dc63ad282f54604c300bbfe654f11
Reviewed-on: https://chromium-review.googlesource.com/1157687
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Vadim Shtayura 3d429cf513 [cipd] Check CIPD client hash against pinned SHA256 during updates.
Linux and OSX only for now. This also rolls CIPD client to a version that
supports pinned hashes (v2.2.5).

CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since
it makes no sense when pinning hashes of the binaries at specific version on
the specific backend.

Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output
consistently.

R=iannucci@chromium.org, nodir@chromium.org
BUG=870166, 874586

Change-Id: Iac67fbb6b5d07dcd81d44536737b03b146f1ad14
Reviewed-on: https://chromium-review.googlesource.com/1176727
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
7 years ago
Robert Sesek 16a396e431 Clarify OWNERS error if using TBR.
The message was previously not clear that if the entire CL has any change
to an OWNERS file, TBR will not apply.

Change-Id: Ib30a93f80196f45b02a29701faab7cf581e37f50
Reviewed-on: https://chromium-review.googlesource.com/1176321
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
Commit-Queue: Robert Sesek <rsesek@chromium.org>
7 years ago
Vadim Shtayura 5e5c2173ba Revert "[cipd] Check CIPD client hash against pinned SHA256 during updates."
This reverts commit eebc3d8232.

Reason for revert: crbug.com/874586

Original change's description:
> [cipd] Check CIPD client hash against pinned SHA256 during updates.
> 
> Linux and OSX only for now. This also rolls CIPD client to a version that
> supports pinned hashes (v2.2.5).
> 
> CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since
> it makes no sense when pinning hashes of the binaries at specific version on
> the specific backend.
> 
> Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output
> consistently.
> 
> R=​iannucci@chromium.org, nodir@chromium.org
> BUG=870166
> 
> Change-Id: I9e61f9f8fbdcf10985c52828b2bfbec64b4234f0
> Reviewed-on: https://chromium-review.googlesource.com/1171957
> Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
> Reviewed-by: Nodir Turakulov <nodir@chromium.org>

TBR=iannucci@chromium.org,vadimsh@chromium.org,nodir@chromium.org

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: 870166
Change-Id: I9aa8e7a7f07520aa69d366c76e4dbccae345bc00
Reviewed-on: https://chromium-review.googlesource.com/1175294
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
7 years ago
Vadim Shtayura eebc3d8232 [cipd] Check CIPD client hash against pinned SHA256 during updates.
Linux and OSX only for now. This also rolls CIPD client to a version that
supports pinned hashes (v2.2.5).

CIPD_CLIENT_VER and CIPD_CLIENT_SRV are no longer supported as env vars, since
it makes no sense when pinning hashes of the binaries at specific version on
the specific backend.

Also somewhat cleanup 'cipd' script to use "${VAR}", stderr and colored output
consistently.

R=iannucci@chromium.org, nodir@chromium.org
BUG=870166

Change-Id: I9e61f9f8fbdcf10985c52828b2bfbec64b4234f0
Reviewed-on: https://chromium-review.googlesource.com/1171957
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
7 years ago
Edward Lemur 6a4e31b488 gclient_scm: Add option to specify the branch the patch is based on.
Also add shortcuts, so we don't have to look through all the branches
if the patch is based on a common branch (i.e. master, infra/config, lkgr)


Bug: 870279
Change-Id: I625a8481dccac9a475b096b926e6fab7efe676b0
Reviewed-on: https://chromium-review.googlesource.com/1161094
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Edward Lemur dd5051fa52 metrics: Deal with permission denied when writing metrics.cfg.
If we don't have permission to create the metrics.cfg file, print a
notice and disable metrics collection.

Bug: 870231
Change-Id: I784e988ed021daef0fb07c08f1da44718581b1b9
Reviewed-on: https://chromium-review.googlesource.com/1166322
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 6f812e132d metrics: Separate metrics collection and print notice logic.
As it is right now, when a function is collecting metrics it prints
the notice and calls sys.exit() to ensure the notice is the last thing
that is printed.

git-cl split has to call git-cl upload multiple times, but once it has
been called once, it exits, so only the first cl is uploaded.

This separates metrics collection from notice printing, so that the
function that is collecting metrics behaves like a function that isn't.
It also makes sure we don't collect metrics for multiple functions at
the same time.

Bug: 868280
Change-Id: Ic58ebe7d19e09ed85fa8b0af76dcbf608ee4c9bc
Reviewed-on: https://chromium-review.googlesource.com/1153503
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 3e65878e5e Add --patch_ref to bot_update.
This doesn't expose --patch_ref in the bot_update recipe_module yet,
though.

Bug: 865882
Change-Id: I26abe59d2f6a8a3dd3945cce0bf13b238a7fffd3
Reviewed-on: https://chromium-review.googlesource.com/1151105
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Vadim Shtayura 7e50ee376b [cipd] Try to bootstrap CIPD from scratch if selfupdate fails.
When updating the CIPD client to be identified by SHA256 hash, old clients (that
have no idea about SHA256) fail during 'selfupdate'.

We'll roll our SHA256 support in two stages:
  1. Deploy new client that understand SHA256 using its SHA1 name, so
     self-update from old clients works.
  2. Deploy same (or newer) client using its SHA256 name. This would work since
     the client doing the self-update already understands SHA256 at this point.

But we can't guarantee that ALL depot_tools deployments will update through
stages (1) and (2) sequentially. Some of them may skip (1) and end-up directly
in (2), failing on 'selfupdate'.

This CL makes sure they can recover from this state by rebootstraping the client
from scratch (this works with SHA256 hashes).

R=nodir@chromium.org, iannucci@chromium.org
BUG=821194

Change-Id: I27dece19e0305b5b2d6f8b0130631c1bf5f6499c
Reviewed-on: https://chromium-review.googlesource.com/1149454
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
7 years ago
Edward Lemur ca7d881540 Reland "Reland "gclient_scm: Use cherry-picking instead of rebasing.""
Abort any cherry-picks before applying the patch, so that if the bots are in a
bad state, we don't fail.

Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
>
> The failures were caused by:
>  1 - When one change (call it #2) has been uploaded on top of another (#1),
>      and (#1) has already landed, git cherry-pick complains that the range
>      '<merge-base>..<change #2>' contains empty commits, since the contents
>      of (#1) are already present in the tree.
>  2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
>      so a failure made all further CLs in that bot fail.
>
> This CL fixes it and prevents further regressions.
>
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
>
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 850812
Change-Id: Ic65bda67c792bd7af5ec013a62d9615d1498eb3a
Reviewed-on: https://chromium-review.googlesource.com/1142805
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 5ba1e9caee git cl: Start reporting metrics.
Bug: 832386
Change-Id: Iccb71c020aea31db33fc16050cb1100ccf5a7fbb
Reviewed-on: https://chromium-review.googlesource.com/1145902
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 60b9b6fb91 git_hyper_blame_test: Fix pylint issue.
TBR=agable@chromium.org
Bug: 863669
Change-Id: Id641269f023acc1a89e83e9e5951e933eae93d40
Reviewed-on: https://chromium-review.googlesource.com/1145720
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 2c984a03ca Revert "Reland "gclient_scm: Use cherry-picking instead of rebasing.""
This reverts commit f9afc77f28.

Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=864301#c10

Original change's description:
> Reland "gclient_scm: Use cherry-picking instead of rebasing."
> 
> The failures were caused by:
>  1 - When one change (call it #2) has been uploaded on top of another (#1),
>      and (#1) has already landed, git cherry-pick complains that the range
>      '<merge-base>..<change #2>' contains empty commits, since the contents
>      of (#1) are already present in the tree.
>  2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
>      so a failure made all further CLs in that bot fail.
> 
> This CL fixes it and prevents further regressions.
> 
> Original change's description:
> > gclient_scm: Use cherry-picking instead of rebasing.
> >
> > Currently gclient might include extra commits when applying patches.
> > For example, in this case we checkout |patch| and rebase it on top of |base|,
> > thus including an |extra commit| that we shouldn't.
> >
> > o master
> > |
> > . o patch
> > |/
> > o extra commit
> > |
> > o base (what gclient synced src at)
> >
> > This change uses the merge-base between |patch| and |master| to cherry-pick only
> > the changes belonging to the patch.
> >
> > Bug: 850812
> > Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> > Reviewed-on: https://chromium-review.googlesource.com/1137052
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
> 
> Bug: 850812
> Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
> Reviewed-on: https://chromium-review.googlesource.com/1139554
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=agable@chromium.org,ehmaldonado@chromium.org,jbudorick@chromium.org

Change-Id: I2138757899abd6eb9bece2dac3a25676effafdf3
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812
Reviewed-on: https://chromium-review.googlesource.com/1142624
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur f9afc77f28 Reland "gclient_scm: Use cherry-picking instead of rebasing."
The failures were caused by:
 1 - When one change (call it #2) has been uploaded on top of another (#1),
     and (#1) has already landed, git cherry-pick complains that the range
     '<merge-base>..<change #2>' contains empty commits, since the contents
     of (#1) are already present in the tree.
 2 - We did not abort the cherry-picking when 'git cherry-pick' failed,
     so a failure made all further CLs in that bot fail.

This CL fixes it and prevents further regressions.

Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
>
> Currently gclient might include extra commits when applying patches.
> For example, in this case we checkout |patch| and rebase it on top of |base|,
> thus including an |extra commit| that we shouldn't.
>
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
>
> This change uses the merge-base between |patch| and |master| to cherry-pick only
> the changes belonging to the patch.
>
> Bug: 850812
> Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> Reviewed-on: https://chromium-review.googlesource.com/1137052
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

Bug: 850812
Change-Id: I83f38d0a258df3f5cd89e277f0d648badff29a22
Reviewed-on: https://chromium-review.googlesource.com/1139554
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 302bb847d3 gclient: run cipd ensure in paranoid mode.
Bug: 860251
Change-Id: I0acaafcfe1d214d102b6a17b29b76aadcc1542c2
Reviewed-on: https://chromium-review.googlesource.com/1132345
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
7 years ago
Edward Lemur a11fc9b2bd git_cache: Add tests for reset fetch config.
Bug: 862547
Change-Id: I56b096cbf3358c06aa5c285594330af118ab17fd
Reviewed-on: https://chromium-review.googlesource.com/1138714
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur 7fa0f19f73 metrics: Fix countdown to metrics collection.
We were printing the notice when the decorator was called, so that if
several functions are decorated, the countdown will increase more than once
during each execution.

This change fixes that.

Bug: 832386
Change-Id: I5d6dd2d793137e1e2c83b6cc765b6245080b9d91
Reviewed-on: https://chromium-review.googlesource.com/1141073
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 3298e7b592 gclient: Start reporting metrics.
Bug: 832386
Change-Id: I6d1167802f077bcd67bf004ccc417661d8fff3c7
Reviewed-on: https://chromium-review.googlesource.com/1135903
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 40bacee96a Revert "gclient_scm: Use cherry-picking instead of rebasing."
This reverts commit fb78b368de.

Reason for revert: https://bugs.chromium.org/p/chromium/issues/detail?id=864301

Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
> 
> Currently gclient might include extra commits when applying patches.
> For example, in this case we checkout |patch| and rebase it on top of |base|,
> thus including an |extra commit| that we shouldn't.
> 
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
> 
> This change uses the merge-base between |patch| and |master| to cherry-pick only
> the changes belonging to the patch.
> 
> Bug: 850812
> Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
> Reviewed-on: https://chromium-review.googlesource.com/1137052
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

TBR=agable@chromium.org,ehmaldonado@chromium.org

Change-Id: I57299e60e58eac5656dc88937c622d0d14c4ba37
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812
Reviewed-on: https://chromium-review.googlesource.com/1139553
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur fb78b368de gclient_scm: Use cherry-picking instead of rebasing.
Currently gclient might include extra commits when applying patches.
For example, in this case we checkout |patch| and rebase it on top of |base|,
thus including an |extra commit| that we shouldn't.

o master
|
. o patch
|/
o extra commit
|
o base (what gclient synced src at)

This change uses the merge-base between |patch| and |master| to cherry-pick only
the changes belonging to the patch.

Bug: 850812
Change-Id: I138192f96bc62b1bb19b0e1ad952c8f8c67631c4
Reviewed-on: https://chromium-review.googlesource.com/1137052
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Joey Scarr 8d3925b164 Add support to gclient for skipping CIPD deps.
I want to use this in the Codesearch builder to replace the hack in
crrev.com/c/1126693.

Bug: 860397
Change-Id: I174a684cbff0f993b5c657bc2e32105cb49d253a
Reviewed-on: https://chromium-review.googlesource.com/1132266
Commit-Queue: Joey Scarr <jsca@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 579c98612b Reset the fetch config in the bots before each run.
If a ref is incorrectly set, it will affect later runs.
For example, [1] caused a bad refspec to be added to the fetch config,
(+refs/branch-heads/refs/branch-heads/6.7:refs/branch-heads/refs/branch-heads/6.7)
which affects all subsequent runs in the same bot: [2], [3].

This makes bot_update clear the fetch config before each run, so it is more
robust against these kind of failures.

[1] https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8942033142534034848%2F%2B%2Fsteps%2Fbot_update%2F0%2Fstdout
[2] https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8941307871443322944%2F%2B%2Fsteps%2Fbot_update%2F0%2Fstdout
[3] https://logs.chromium.org/v/?s=v8%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8941333453153279680%2F%2B%2Fsteps%2Fbot_update%2F0%2Fstdout

Bug: 862547
Change-Id: I2f849c604656e81ebd7377465d287226b8bdea1a
Reviewed-on: https://chromium-review.googlesource.com/1135645
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
7 years ago
John Budorick 882c91ed97 Revert "gclient: Add support for the branch:revision format."
This reverts commit d328b47eb1.

Reason for revert: suspect this caused https://bugs.chromium.org/p/chromium/issues/detail?id=863211

Original change's description:
> gclient: Add support for the branch:revision format.
> 
> Bug: 850812, 853032
> Change-Id: I597acbde2b3c229813b7eba8fcba52d5877130b2
> Reviewed-on: https://chromium-review.googlesource.com/1119235
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

TBR=agable@chromium.org,tandrii@chromium.org,ehmaldonado@chromium.org

Change-Id: I1e4c00b83a2840cc5a87621469da866dad58a20c
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812, 853032
Reviewed-on: https://chromium-review.googlesource.com/1135893
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur d328b47eb1 gclient: Add support for the branch:revision format.
Bug: 850812, 853032
Change-Id: I597acbde2b3c229813b7eba8fcba52d5877130b2
Reviewed-on: https://chromium-review.googlesource.com/1119235
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur 32e3d1e37c Add a library for monitoring.
Change-Id: I64c3d143186be938042c12e2455bdb034d3bcba8
Reviewed-on: https://chromium-review.googlesource.com/1079639
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur d64781e17a gclient_scm: Fetch refs/changes/* when syncing.
When a git mirror is configured, it fetches only refs/heads/*,
and possibly branch-heads and tags, but not refs/changes.

When gclient attempts to sync refs/changes/* from a mirror,
it fails, since the mirror has no such objects.

See for example [1], where the DEPS entry for builtools was
modified to sync to a CL, but all the bots failed to sync to it.

This change modifies gclient to always fetch refs/changes/*
directly from the repo, even when a mirror is configured.

[1] https://chromium-review.googlesource.com/c/chromium/src/+/1119098/6

Bug: 858894

Change-Id: I71bb313e4325a81b2985db1d00c70a8844dc7c22
Reviewed-on: https://chromium-review.googlesource.com/1119525
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur a92b961190 gclient setdep: Add support for CIPD versions
gclient setdep used to assume that the version of a CIPD package was always
of the form 'version:<something>'.
However, the following are all valid:
  * SHA1 hashes that specify a single instance
  * tags (tagname:tagvalue)
  * References (e.g. 'latest') that are dynamically resolved by the CIPD server.

This change adds support for all of them:
  * GetCIPD simply returns whatever the 'version' entry is, without assuming
    it begins with 'version:'.
  * SetCIPD sets 'version' ta whatever it is passed, without prepending 'version:'
    at the beginning.

Bug: 858978
Change-Id: I53669c5df7fb51cde42e0af271139b5719a47622
Reviewed-on: https://chromium-review.googlesource.com/1120943
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Robert Iannucci a19649b86b Clean up cache_dir handling in gclient/git_cache/bot_update.
This CL makes a couple changes:
  * The goal here is to be able to specify git cache entirely from the
    environment variable $GIT_CACHE_PATH and not require special treatment from
    bot_update. Eventually this will be specified at the swarming task level
    instead of in the recipe (i.e. "cached git" will eventually be an
    implementation detail of git on the bots and completely transparent to
    all other software).
  * Removal of the general --cache-dir option from gclient. This option was
    error-prone; it doesn't actually make sense to configure this on
    a per-invocation basis. The sole exception was `gclient config`, which
    now allows this option to be set directly.
  * Consolidation of GitWrapper.cache_dir and GitWrapper._GetMirror; previously
    these two things could disagree, leading to weird intermediate states. Now
    they're the same value.

R=agable@chromium.org, ehmaldonado@chromium.org, tandrii@chromium.org

Bug: 823434
Change-Id: I14adc7619b5fc10768ce32be2651c6215ba94aff
Reviewed-on: https://chromium-review.googlesource.com/1105473
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
7 years ago
Yoshisato Yanagisawa 406de133ef Add gclient_utils to InputAPI export.
Let me make them exported via InputAPI to be used by our repo.

Change-Id: I1a0c7f49ab5a3c00b2617062444d67bbb9600837
Reviewed-on: https://chromium-review.googlesource.com/1117958
Commit-Queue: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
7 years ago
John Budorick c35aba51d3 gclient: escape cipd variables when flattening DEPS.
crrev.com/c/1097623 broke the official bots because the cipd variable
escaping present in the normal DEPS file wasn't present in the flattened
DEPS file.

Change-Id: Iaae8b95a9d8c5ce0ed80071169e731a24e98b4c4
Reviewed-on: https://chromium-review.googlesource.com/1112935
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur e4e15044c3 gclient: Don't include CIPD package name in hierarchy.
It causes flakiness, since the package that is responsible for
printing the dependency is not always the same.

Bug: None
Change-Id: I887d740d6a903cb0d5e6c080b76feab78a54effd
Reviewed-on: https://chromium-review.googlesource.com/1116191
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 7ccf2f0c21 gclient: Report what dependencies were not processed.
Report the dependencies that were not synced, due to their, or their parent's
condition evaluating to False.

Bug: 853010
Change-Id: I375703a1b91e3c3e31e444b0df1c95ecae17b6ba
Reviewed-on: https://chromium-review.googlesource.com/1111113
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Andrii Shyshkalov 77094f7fea Reland of "Wipe out shallow checkout support from bot_update."
Originally-Reviewed-on: https://chromium-review.googlesource.com/1111107
Led-test: https://ci.chromium.org/swarming/task/3e51ed0e31211710?server=chromium-swarm.appspot.com

TBR=hinoka, iannucci

Bug: 855137
Change-Id: I157ef6b9ee2a7ce45e4a474853ae769a82cc4c99
Reviewed-on: https://chromium-review.googlesource.com/1114234
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 612d62d0ca Revert "bot_update: Don't force solutions to be unmanaged."
This reverts commit c48318a35b.

Reason for revert: likely broke ChromeOS

Original change's description:
> bot_update: Don't force solutions to be unmanaged.
> 
> gclient should be the one syncing the solutions to the appropriate
> revisions.
> 
> Bug: 850812, 853032
> Change-Id: Ieefc5661627d4864deb0d4e7053168a99da29d29
> Reviewed-on: https://chromium-review.googlesource.com/1102833
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=hinoka@chromium.org,agable@chromium.org,ehmaldonado@chromium.org

Change-Id: I7e110108a9687d703ea8be39cc83e0888f7813c6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812, 853032, 855700
Reviewed-on: https://chromium-review.googlesource.com/1111925
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 565fa6a407 Revert "Wipe out shallow checkout support from bot_update."
This reverts commit 568b7a8998.

Reason for revert: temporary revert to revert previous CL.

Original change's description:
> Wipe out shallow checkout support from bot_update.
> 
> Bug: 855137
> Change-Id: I60b8d69f9ab353c12120d4216e3a12518f99732c
> Reviewed-on: https://chromium-review.googlesource.com/1111107
> Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
> Reviewed-by: Ryan Tseng <hinoka@chromium.org>

TBR=iannucci@chromium.org,hinoka@chromium.org,tandrii@chromium.org

Change-Id: Iede67717be448be9bdd7c5dd4895fbab9fd65c66
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 855137, 855700
Reviewed-on: https://chromium-review.googlesource.com/1111924
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Andrii Shyshkalov 568b7a8998 Wipe out shallow checkout support from bot_update.
Bug: 855137
Change-Id: I60b8d69f9ab353c12120d4216e3a12518f99732c
Reviewed-on: https://chromium-review.googlesource.com/1111107
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
7 years ago
Edward Lemur c48318a35b bot_update: Don't force solutions to be unmanaged.
gclient should be the one syncing the solutions to the appropriate
revisions.

Bug: 850812, 853032
Change-Id: Ieefc5661627d4864deb0d4e7053168a99da29d29
Reviewed-on: https://chromium-review.googlesource.com/1102833
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur e42137040a gclient: Make gclient respect unmanaged dependencies when syncing.
gclient syncs unmanaged dependencies when '--revision' files are passed,
regardless of whether the dependency appears in the revision flags or not.

Bug: 853032
Change-Id: I66de0865e1103d92524f565f4392dae24a5648e1
Reviewed-on: https://chromium-review.googlesource.com/1105416
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
John Budorick 4099daa97b gclient: Use posixpath-style separators for cipd subdirs on all platforms.
Bug: 854219
Change-Id: Ibd83135dcb96979f8ab989a248e3e2cf59b9dd43
Reviewed-on: https://chromium-review.googlesource.com/1106698
Commit-Queue: John Budorick <jbudorick@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Michael Moss d683d7ced4 Revert "gclient: Don't allow None URLs (except in .gclient files)"
This reverts commit 0c91147d50.

Reason for revert: This is causing 'gclient revinfo' to fail on the release builders, and appears to be somehow related to the "--output-json" flag.

Original change's description:
> gclient: Don't allow None URLs (except in .gclient files)
> 
> This reverts commit crrev.com/4e9b50ab86b9b9f8ebf0b9ba6bd4954217ebeff9
> and thus relands the following commits:
> 
>   ebdd0db493b20f0abeab8960e6ea0ceb7c6b379a: "gclient: Remove URLs from hierarchy."
>   54a5c2ba8ac2f9b8f4a32fe79913f13545e4aab9: "gclient: Refactor PrintRevInfo"
>   083eb25f9acbe034db94a1bd5c1659125b6ebf98: "gclient: Don't allow URL to be None."
> 
> When a None URL is specified in a .gclient file, and a DEPS file is
> given, the DEPS file is treated as a .gclient file and its dependencies
> are added.
> 
> Bug: 839925
> 
> Change-Id: I1068b66487874bfa0a788bf9da5273714b6ad39e
> Reviewed-on: https://chromium-review.googlesource.com/1083340
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Reviewed-by: Michael Moss <mmoss@chromium.org>

TBR=agable@chromium.org,mmoss@chromium.org,ehmaldonado@chromium.org

Change-Id: I46785bd272b16b3672e553b6443cee6d6b370ec1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 839925, 853093
Reviewed-on: https://chromium-review.googlesource.com/1101978
Reviewed-by: Michael Moss <mmoss@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
7 years ago
Andrii Shyshkalov 690d8d437b Revert "gclient_scm: Use cherry-picking instead of rebasing."
This reverts commit c912114140.

Reason for revert: broke patch application on infra/config https://crbug.com/853032

Original change's description:
> gclient_scm: Use cherry-picking instead of rebasing.
> 
> We have a problem when in this situation, we checkout |patch| and rebase it on
> top of |base|, thus including an |extra commit| that we shouldn't.
> 
> o master
> |
> . o patch
> |/
> o extra commit
> |
> o base (what gclient synced src at)
> 
> This does merge-base between |patch| and |master|, and cherry-picks only the
> changes belonging to the patch.
> 
> Bug: 850812
> Change-Id: Id09ae1682e53b69ed49b2fb649310de6a6a8a29e
> Reviewed-on: https://chromium-review.googlesource.com/1098228
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>

TBR=agable@chromium.org,ehmaldonado@chromium.org

Change-Id: Ib3feeee2f44f5441713383f1dbf08db16fae4717
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 850812, 853032
Reviewed-on: https://chromium-review.googlesource.com/1101977
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Edward Lemur c912114140 gclient_scm: Use cherry-picking instead of rebasing.
We have a problem when in this situation, we checkout |patch| and rebase it on
top of |base|, thus including an |extra commit| that we shouldn't.

o master
|
. o patch
|/
o extra commit
|
o base (what gclient synced src at)

This does merge-base between |patch| and |master|, and cherry-picks only the
changes belonging to the patch.

Bug: 850812
Change-Id: Id09ae1682e53b69ed49b2fb649310de6a6a8a29e
Reviewed-on: https://chromium-review.googlesource.com/1098228
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur 0c91147d50 gclient: Don't allow None URLs (except in .gclient files)
This reverts commit crrev.com/4e9b50ab86b9b9f8ebf0b9ba6bd4954217ebeff9
and thus relands the following commits:

  ebdd0db493b20f0abeab8960e6ea0ceb7c6b379a: "gclient: Remove URLs from hierarchy."
  54a5c2ba8ac2f9b8f4a32fe79913f13545e4aab9: "gclient: Refactor PrintRevInfo"
  083eb25f9acbe034db94a1bd5c1659125b6ebf98: "gclient: Don't allow URL to be None."

When a None URL is specified in a .gclient file, and a DEPS file is
given, the DEPS file is treated as a .gclient file and its dependencies
are added.

Bug: 839925

Change-Id: I1068b66487874bfa0a788bf9da5273714b6ad39e
Reviewed-on: https://chromium-review.googlesource.com/1083340
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
7 years ago
Edward Lemur fbb06aaab1 gclient: Use only recursedeps to decide whether to process a dependency.
Bug: 839925
Change-Id: If86cfbdbea795b6da53dec1ea6e44c5af8a3b846
Reviewed-on: https://chromium-review.googlesource.com/1086270
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur e05f18d477 Reland "Reland "Expand variables in gclient flattened output.""
This is a reland of ff62224443
should_process was set to None in the previous attempt, so CIPD dependencies were not processed.
This CL fixed that.

Original change's description:
> Reland "Expand variables in gclient flattened output."
>
> This is a reland of a32f98e652
>
> Original change's description:
> > Expand variables in gclient flattened output.
> >
> > Bug: 848990
> > Change-Id: I0ad7e4f965973edbc5a335bd30f9cbd7b04abff2
> > Reviewed-on: https://chromium-review.googlesource.com/1085996
> > Reviewed-by: Michael Moss <mmoss@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
>
> Tbr: agable@chromium.org
> Bug: 848990
> Change-Id: I7843544b79b2ab7e2046c187d13ea3eb65fc1b7d
> Reviewed-on: https://chromium-review.googlesource.com/1085975
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Bug: 848990
Change-Id: Ic804be1b84bf8402a741a4189b60372075dfb6f3
Reviewed-on: https://chromium-review.googlesource.com/1087368
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur efe4f897c2 gclient: Add a test to make sure we're syncing CIPD dependencies.
Bug: 849374
Change-Id: I5654ed65843c6dae6b7dbee10a40aa35e16803dc
Reviewed-on: https://chromium-review.googlesource.com/1087390
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
John Budorick b990770386 Revert "Reland "Expand variables in gclient flattened output.""
This reverts commit ff62224443.

Reason for revert: spec revert for crbug.com/849374

Original change's description:
> Reland "Expand variables in gclient flattened output."
> 
> This is a reland of a32f98e652
> 
> Original change's description:
> > Expand variables in gclient flattened output.
> >
> > Bug: 848990
> > Change-Id: I0ad7e4f965973edbc5a335bd30f9cbd7b04abff2
> > Reviewed-on: https://chromium-review.googlesource.com/1085996
> > Reviewed-by: Michael Moss <mmoss@chromium.org>
> > Reviewed-by: Aaron Gable <agable@chromium.org>
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> 
> Tbr: agable@chromium.org
> Bug: 848990
> Change-Id: I7843544b79b2ab7e2046c187d13ea3eb65fc1b7d
> Reviewed-on: https://chromium-review.googlesource.com/1085975
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=agable@chromium.org,mmoss@chromium.org,ehmaldonado@chromium.org

Change-Id: I89ffb80893be72f60043e8a6e02f6c0fb6fe5cc1
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 848990
Reviewed-on: https://chromium-review.googlesource.com/1086747
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: John Budorick <jbudorick@chromium.org>
7 years ago
Edward Lemur ff62224443 Reland "Expand variables in gclient flattened output."
This is a reland of a32f98e652

Original change's description:
> Expand variables in gclient flattened output.
>
> Bug: 848990
> Change-Id: I0ad7e4f965973edbc5a335bd30f9cbd7b04abff2
> Reviewed-on: https://chromium-review.googlesource.com/1085996
> Reviewed-by: Michael Moss <mmoss@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

Tbr: agable@chromium.org
Bug: 848990
Change-Id: I7843544b79b2ab7e2046c187d13ea3eb65fc1b7d
Reviewed-on: https://chromium-review.googlesource.com/1085975
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lesmes 01cb51055e Revert "Expand variables in gclient flattened output."
This reverts commit a32f98e652.

Reason for revert:
Doesn't work when there are variables which definition includes other variables, e.g.:

'cros_download_vm': '"{cros_board}" == "amd64_generic"',

Original change's description:
> Expand variables in gclient flattened output.
> 
> Bug: 848990
> Change-Id: I0ad7e4f965973edbc5a335bd30f9cbd7b04abff2
> Reviewed-on: https://chromium-review.googlesource.com/1085996
> Reviewed-by: Michael Moss <mmoss@chromium.org>
> Reviewed-by: Aaron Gable <agable@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>

TBR=agable@chromium.org,mmoss@chromium.org,ehmaldonado@chromium.org

Change-Id: I7e81000e92aa352e45b420845bb1dcc8572f1962
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 848990
Reviewed-on: https://chromium-review.googlesource.com/1085974
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur a32f98e652 Expand variables in gclient flattened output.
Bug: 848990
Change-Id: I0ad7e4f965973edbc5a335bd30f9cbd7b04abff2
Reviewed-on: https://chromium-review.googlesource.com/1085996
Reviewed-by: Michael Moss <mmoss@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Aaron Gable 4b23a2c6d7 Remove more Rietveld support from presubmit
This simplifies future removals of unnecessary code from git-cl.

Bug: 770408
Change-Id: I20329fb6d93b349bf2e9f4aebc795b3f7ef0b646
Reviewed-on: https://chromium-review.googlesource.com/1070588
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
7 years ago
Michael Moss 4e9b50ab86 Revert recent gclient.py changes to fix Chrome releases.
Reason for revert: This is breaking official release scripts, which have long
used "None" URL functionality to allow gclient commands to run against
"local" DEPS (buildspec) files. Please do not remove this behavior.

This reverts the following commits:
  ebdd0db493b20f0abeab8960e6ea0ceb7c6b379a: "gclient: Remove URLs from hierarchy."
  54a5c2ba8ac2f9b8f4a32fe79913f13545e4aab9: "gclient: Refactor PrintRevInfo"
  083eb25f9acbe034db94a1bd5c1659125b6ebf98: "gclient: Don't allow URL to be None."

BUG=846194

TBR=agable@chromium.org,ehmaldonado@chromium.org

Change-Id: Ibdd5581889bd4afd86474199c7b64555f01bbbca
Reviewed-on: https://chromium-review.googlesource.com/1070893
Commit-Queue: Michael Moss <mmoss@chromium.org>
Reviewed-by: Michael Moss <mmoss@chromium.org>
7 years ago
Edward Lemur 54a5c2ba8a gclient: Refactor PrintRevInfo
Bug: 839925
Change-Id: I9d1b2572676a7d61862382f6882622521d28ef97
Reviewed-on: https://chromium-review.googlesource.com/1068250
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 428e7c14de roll-dep: Add tests.
Bug: 829134
Change-Id: Ifa4dc5572a17c9f2a3ab4bb04765106b1563ecae
Reviewed-on: https://chromium-review.googlesource.com/1065081
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 083eb25f9a gclient: Don't allow URL to be None.
Bug: 839925
Change-Id: I5a559db846ac19e3c61e77f3019c7d5362beccba
Reviewed-on: https://chromium-review.googlesource.com/1063029
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 4727c64750 gclient: Fix getdep and setdep and add tests.
Bug: 843917, 839925
Change-Id: I84e52c95fc91d67375d4d954018f057561896914
Reviewed-on: https://chromium-review.googlesource.com/1064005
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur 23a358764d gclient: Fix for extra custom_deps.
Custom deps not present in DEPS files cause errors when syncing, since
we add them as strings in postprocess_deps, but deps_to_objects expects
a dictionary.

TBR=agable@chromium.org

Bug: 839925
Change-Id: Ic08a83e8692f1bf90d4456c72fe99493363ba747
Reviewed-on: https://chromium-review.googlesource.com/1063326
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
7 years ago
Edward Lemur ed1bb34fc5 gclient: Remove condition_value.
It not used anywhere.

Bug: 839925
Change-Id: Iad07b548744f2c58d34427f3e2225d8c75926eea
Reviewed-on: https://chromium-review.googlesource.com/1060632
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago
Edward Lemur 16f4bad914 gclient: Merge hook_os with hooks and deps_os with os.
This is done in gclient_eval, so we can remove all code
that deals with deps_os and hooks_os from gclient.

Bug: 839925
Change-Id: I491819207a712d62008ff010e313add87d22c937
Reviewed-on: https://chromium-review.googlesource.com/1058375
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
7 years ago