This reverts commit fa573785df.
Reason for revert: broke CQ on branches.
Example: https://ci.chromium.org/p/chromium/builders/luci.chromium.try/chromium_presubmit/28220
Original change's description:
> bot_update: allow rebasing the patch onto an older revision.
>
> When applying a rebase, we normally go from an older base commit to a newer one.
>
> A ---- B ---- C ---- D -origin/master
> \
> E -branch
>
> In this case, `git rebase D` would certainly work as expected. However,
> writing `git rebase B` would NOT get us to the following state:
>
> A ---- B ---- C ---- D -origin/master
> \
> E' -branch
>
> In fact, it would have no effect.
>
> This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
> explains the general invocation as
> > `git rebase --onto <graft-point> <exclude-from> <include-from>`
> > If you don’t specify --onto, <graft-point> defaults to <exclude-from>
>
> So what's happening is, by writing `git rebase B` we're rebasing onto B,
> excluding commits that are in B. Commit C is not "in" B so it is kept and we're
> back to the starting point.
>
> So I suggest to change the invocation to `git rebase --onto B origin/master`,
> which rebases onto B, excluding commits that are in origin/master. This works
> more generally and allows rebasing "backwards".
>
> Bug: None
> Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
> Reviewed-on: https://chromium-review.googlesource.com/904004
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Oleh Prypin <oprypin@chromium.org>
TBR=iannucci@chromium.org,tandrii@chromium.org,oprypin@chromium.org
Change-Id: I7243641d84428bf4504ee798bf31ad0afbfd4865
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: None
Reviewed-on: https://chromium-review.googlesource.com/905862
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This will be used when running infrastructure code locally
(e.g. recipes) to correctly interact with LUCI service APIs
(such as luci-scheduler.appspot.com).
It is also generically useful to explore and interact with LUCI
service APIs on the command line (for debugging/scripting).
R=tandrii@chromium.org, vadimsh@chromium.org
Bug: 808677
Change-Id: I41cfd4cc7e2d245d3a5d2be83f9879f92a8d1bca
Reviewed-on: https://chromium-review.googlesource.com/905457
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Goal: reduce number of git fetches from remote to the minimum possible.
Minimum possible is either:
* 0 if desired revision is pinned (!= HEAD) and already exists in cache
* 1 if desired revision is HEAD
* >= 2 if revision is pinned but we talk temporarily
out-of-date git server not yet having it.
This CL achieves the above.
No fetch:
https://ci.chromium.org/swarming/task/39dad20230c84110?server=chromium-swarm.appspot.com
bot_update takes <1min, but it used to take at least 2. The steps are now like this:
...
===Running git cat-file -e 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
In directory: /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
===Succeeded in 0.0 mins===
===Running /b/swarming/w/ir/cipd_bin_packages/bin/python -u /b/swarming/w/ir/recipe-checkout-dir/depot_tools/git_cache.py exists --quiet --cache-dir /b/swarming/w/ir/cache/git https://chromium.googlesource.com/chromium/src.git===
In directory: /b/swarming/w/ir/cache/builder/linux
/b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
===Succeeded in 0.0 mins===
===Running git remote set-url origin /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src===
In directory: /b/swarming/w/ir/cache/builder/linux/src
===Succeeded in 0.0 mins===
===Running git fetch origin===
In directory: /b/swarming/w/ir/cache/builder/linux/src
From /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
003eedbfee47..5210107a20e8 master -> origin/master
===Succeeded in 0.0 mins===
===Running git checkout --force 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
...
Fetch & success:
https://ci.chromium.org/swarming/task/39db022b874ffd10?server=chromium-swarm.appspot.com
Fetch & retry because missing rev:
https://ci.chromium.org/swarming/task/39db04ce8b4b8910?server=chromium-swarm.appspot.com
Fetch because rev=HEAD:
https://ci.chromium.org/swarming/task/39db0c0dd5ab5d10?server=chromium-swarm.appspot.com
This has already been adopted in gclient.
Bug:
Change-Id: Id99892b62719fdf3f7e6e59058986d1500384f8d
Reviewed-on: https://chromium-review.googlesource.com/771591
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
This CL adds target_cpu and target_cpu_only variables. The logic is:
* If target_cpu is defined in .gclient, use it. Otherwise, initialize it as an
empty list.
* If not target_cpu_only, add the host arch to the list too.
detect_host_arch.py is copied directly from the Chromium repo.
BUG=807986
R=dpranke
Change-Id: I27621cbc81ad6a844648525863b92ffdd3b1d03a
Reviewed-on: https://chromium-review.googlesource.com/902441
Commit-Queue: Thomas Anderson <thomasanderson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
When applying a rebase, we normally go from an older base commit to a newer one.
A ---- B ---- C ---- D -origin/master
\
E -branch
In this case, `git rebase D` would certainly work as expected. However,
writing `git rebase B` would NOT get us to the following state:
A ---- B ---- C ---- D -origin/master
\
E' -branch
In fact, it would have no effect.
This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
explains the general invocation as
> `git rebase --onto <graft-point> <exclude-from> <include-from>`
> If you don’t specify --onto, <graft-point> defaults to <exclude-from>
So what's happening is, by writing `git rebase B` we're rebasing onto B,
excluding commits that are in B. Commit C is not "in" B so it is kept and we're
back to the starting point.
So I suggest to change the invocation to `git rebase --onto B origin/master`,
which rebases onto B, excluding commits that are in origin/master. This works
more generally and allows rebasing "backwards".
Bug: None
Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
Reviewed-on: https://chromium-review.googlesource.com/904004
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Oleh Prypin <oprypin@chromium.org>
The first call to gsutil causes a network call and makes the download
script slow also in the most optimal cases. This CL refactors the
download script and moves the first gsutil call after checking locally
if sha1s match.
1) This turns the input acquisition into a generator and buffers the
files and sha1s in a list before multithreading.
2) This sequentially checks the sha1s and files and bails out early if
all match. In Chrome-land, we usually call this script with only one
file. There are some cases with around 4. This could also be
parallelized if the need arises.
3) The initial gsutil check, which ensures gsutil is updated, is moved
right in front of the multithreaded downloads.
The performance of one call to download_from_google_storage for an
existing 500MB file is 2.3s before this CL and 1.2s after (most of the
remaining time left is spent for making sha1sum).
Example for full gclient runhooks (when everything is up-to-date):
Chromium: 32s before, 12s after
V8: 12s before, 3s after
Bug: 776311
Change-Id: Ia7715a6af84b1b336455ea88494d399bdb050317
Reviewed-on: https://chromium-review.googlesource.com/897562
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
This is needed for run_presubmit.py to work.
BUG=pdfium:971
Change-Id: Ib598fbfd6eab3c0b7628301eb3bd54c548a4885e
Reviewed-on: https://chromium-review.googlesource.com/898039
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Lei Zhang <thestig@chromium.org>
Flattening currently ignores custom_vars, but it looks like that was
just an attempt to avoid including all the runtime-calculated values
(e.g. the host OS) in the flattened output, and not because there's any
real issue with supporting explicit overrides.
This is useful for changing default values in the flattened buildspecs
(e.g. to enable internal checkouts by default).
R=dpranke@google.com, iannucci@google.com
Bug: 570091,807318
Change-Id: I6d743f33cf7e3e202d5e68d21657f74f0e4c001b
Reviewed-on: https://chromium-review.googlesource.com/895927
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
This will allow us to show the progress when cloning a new repo when running
commands like gclient sync.
R=agable@chromium.org
Bug: 722686
Change-Id: I268cba343ea4c3c024292c9341d5009aa112b184
Reviewed-on: https://chromium-review.googlesource.com/890524
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Add an option to not store stdout when running commands unless it's needed.
R=agable@chromium.org
Bug: 722686
Change-Id: I402c83767097d53e588ba3e8bca89291712a572f
Reviewed-on: https://chromium-review.googlesource.com/888584
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
I don't know of any use-case where someone would want to
use git-cl-patch to pull a (e.g.) chromium change into
their (e.g.) v8 checkout.
Bug: 803918
Change-Id: Id53f1cc3ab97e623d0098bb366a573b18b54e91a
Reviewed-on: https://chromium-review.googlesource.com/876930
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
Context:
https://bugs.chromium.org/p/chromium/issues/detail?id=800355#c22
Prepare git-cl-try to a new format of builders-map.appspot.com response
where each entry contains buckets, as opposed to masters.
Bug: 800355
Change-Id: I5a90c6c4860a7e1fca843067c477a272d782cba1
Reviewed-on: https://chromium-review.googlesource.com/885100
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This fixes the case where a new workdir is created from a repository
which was previously created with gclient-new-workdir.py and that
repository is later deleted, rendering the symbolic links created by
gclient-new-workdir.py invalid.
It also fixes support_cow() returning 'True' in that case because 'cp
--reflink' does not fail when copying a symlink.
Bug: 721585
Change-Id: I84f4d80b8698af0da346be559d46a328ec0d0e9b
Reviewed-on: https://chromium-review.googlesource.com/842402
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
In particular this should:
* Now ignore `-S`, which breaks all VirtualEnvs
* Properly parse the python command line in more cases
* No longer ignore PYTHONPATH (essential for launcher scripts in src.git
which set PYTHONPATH before invoking python)
38335de5a Roll recipe dependencies (trivial).
c20873b37 Roll luci-go (1 commit)
4d90f2197 Roll recipe dependencies (trivial).
79d3339be Roll recipe dependencies (trivial).
9f9a14311 On workflow launched error, log run ID
4c5a25ed7 Roll recipe dependencies (trivial).
d7160f243 Roll luci-go (4 commits)
ec1a77841 [dockerbuild] Add tlslite.
bfbf1dfda Roll infra/luci/ a9cc99df8..ff3140a58 (49 commits)
978202c4c Roll recipe dependencies (trivial).
23b594dff Revert golang.org/x/sys to 52ba35ddf9b65dc3ad27218743aa2444fe13b690
516c58022 [Monorail] Find protoc and plugin at their new locations.
413b8f38d [buildbucket] access: add Makefile
073a90247 Roll recipe dependencies (trivial).
454664529 Add github.com/VividCortex/mysqlerr to Go dependencies
715ed81d4 [som] update relnotes for weekly push
b17e9543a [buildbucket] Add buildbucket_template_canary swarming tag.
ebb7b62de Roll recipe dependencies (trivial).
a1b7ba838 Roll recipe dependencies (trivial).
56ccb3b11 Roll luci-go (2 commits)
d3c73486c [Findit] Move UpdateAnalysisResult from swarming_util.py to services/test_failure/test_failure_analysis
4a63f0820 [Monorail] Set up Polymer dev environment in Monorail.
2ee6775c1 Roll recipe dependencies (trivial).
54837b79e Roll recipe dependencies (trivial).
31e8d5ef0 Roll recipe dependencies (trivial).
8c103759a Roll recipe dependencies (trivial).
717f4ac15 Roll recipe dependencies (trivial).
55c3a19eb Roll luci-go (3 commits)
8b6a95944 Set build_exit_status on api.goma.stop.
b9384470c Roll recipe dependencies (trivial).
37c8d15de Roll recipe dependencies (trivial).
80475628c Roll recipe dependencies (trivial).
7870bfba3 Roll recipe dependencies (trivial).
0a0d75d59 Roll recipe dependencies (trivial).
37043c2b3 Roll luci-go
8b66950f2 Roll luci-go [9 commits]
99c8a8c40 [Findit] Merge and move logic of getting step log to build_util.py
9183a2f90 [event-pipeine] update docs with new bq package
81b4a9688 Roll recipe dependencies (trivial).
047e4b422 CIPD: Upgrade to Polymer 2.0
48027dc0a Roll recipe dependencies (trivial).
b799468f8 Roll recipe dependencies (trivial).
6fa85358e [test-results] Fix single test results chunked rendering
f3bc5ff7c [Predator] Monitor Predator-Wrong-CLs and Predator-Wrong-Components bugs daily.
af626d23d [vpython] Fix optional debugging -log-level argument.
036cbdaa5 Roll recipe dependencies (trivial).
3504421a7 [Findit] Move swarmbot related logic to a new module.
6a21e50cd Rename AnalyzerProgress to FunctionProgress
e9e7b8d57 Roll recipe dependencies (trivial).
ec3390c68 Move tricium/analyzers/ to tricium/functions/
ca9010a1b Roll recipe dependencies (trivial).
d80df7710 Roll recipe dependencies (trivial).
0e1aabf13 Roll recipe dependencies (trivial).
92f33daa7 Roll recipe dependencies (trivial).
24115742d Roll luci-go [5 commits]
1259b0d5b [bqschemaupdater] remove directory
b858620ca Roll recipe dependencies (trivial).
fb4916c75 [doc] update Event Pipeline docs
065dd65b0 [Findit] Fix Type error when extract build info.
9642319bc [Monorail] Catch empty JSON search response
39bd53125 [Findit] Remove duplicated function in try_job.py
8bd31a4f1 Download and build Linux tarball
77f25312e [Findit] Flake Analyzer - Fixing reset to clear suspects
739c48078 [Findit] Flake Analyzer - Fix empty confidence
a2073847d Roll recipe dependencies (trivial).
df4214209 [eventupload] complete move to luci
bc0bab29a [luci-migration] add got_revision to build group key
fb0c38e34 Roll infra/go/src/go.chromium.org/luci/ 94f59dfc7..067307a03 (1 commit)
06c9e5e1e [bqschemaupdater] remove old test-results schema
a3d7ae5cf Roll recipe dependencies (trivial).
1154ed49c Roll recipe dependencies (trivial).
bfcdd50d5 [infra/libs/bigquery] add README
f0ff49524 Roll recipe dependencies (trivial).
75c72ee4f Roll recipe dependencies (trivial).
27573df51 Roll recipe dependencies (trivial).
eacc34857 Roll recipe dependencies (trivial).
d3f8862ca Roll recipe dependencies (trivial).
fc48386c2 Roll recipe dependencies (trivial).
5948d8de6 Updates dev server config whitelist.
479540994 Roll recipe dependencies (trivial).
efa87446b Roll recipe dependencies (trivial).
97cbabb5b [Findit] Fix bug in TestHeuristicResult.
296b77750 Roll recipe dependencies (trivial).
f1123e9a7 [Findit] Flake Analyzer - NextCommitPositionPipeline
779c5a4be [Findit] Flake Analzyer - Skip try jobs for chromium.sandbox
5f8211e2d Roll recipe dependencies (trivial).
ede641a68 [Findit] Fix a bug in heuristic pipeline for test failures.
926916c58 Roll recipe dependencies (trivial).
015ff2c3c [led] Add -canary flag to get-builder.
a1f61935f Roll luci-go DEPS.
da6e85b70 Roll recipe dependencies (trivial).
40574c593 [cipd] Allow '.' in package names.
af84863d1 Roll recipe dependencies (trivial).
959c58536 Roll recipe dependencies (trivial).
a5a8297ab [Monorail] Add Release Notes doc and link.
04cd58a49 [test-results] Replace instead of append divs that already exist.
5d3455659 Start publishing nightly Linux tarballs
R=kbr@chromium.org, nodir@chromium.org, vadimsh@chromium.org
Bug: 800662
Change-Id: Ie32f291acd725a6cd977c907e465dba660dfb855
Reviewed-on: https://chromium-review.googlesource.com/874154
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
The default description in the editor when you invoke "git cl upload"
includes recent commit descriptions, some blank lines and comments, and
"Bug: ". The upload script properly strips out the comments but was
leaving in the "Bug:" line. This leads to CLs with blank Bug: lines,
which is sloppy. It also means that if a user changes their mind and
closes the upload editor without adding any content... they would still
have a CL created.
This change removes the "Bug: " line if nothing has been added to it,
thus avoiding uploading a change with an empty description, and avoiding
having meaningless Bug: lines.
Change-Id: I18c2094e5712d31729899c0ebd0a52de97dc4e7f
Reviewed-on: https://chromium-review.googlesource.com/861920
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
This patch adds the standard issue option (-i/--issue) to
"git cl try" command.
This is useful for maintaining FlagExpectations with fyi bots.
Runtime-flag features often see new failures, and a good number
of tests are flaky. Maintainers can upload a CL and use cron to
run try jobs to collect multiple tests results to find new
failures and flaky tests.
Change-Id: I069593950023ddae814faf3074c33de4c98cb70e
Reviewed-on: https://chromium-review.googlesource.com/850792
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Koji Ishii <kojii@chromium.org>
This should pick up a fix for where vpython attempts to read
`~/.vpython` folders as spec files (and fails).
R=tandrii@chromium.org
Bug: 795146
Change-Id: I261944c3d7e3e8703adc4986c88ae742b801d457
Reviewed-on: https://chromium-review.googlesource.com/851847
Commit-Queue: Aaron Gable <agable@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
If you have too many bytes of untracked+ignored data in your repo then
gclient sync will fail, confidently telling you how many bytes of this
type you have. The problem is, it lies. It reports the level as soon as
it is exceeded, even though it has not finished totaling up the file
sizes. This change makes the reporting accurate.
I noticed this problem because "gclient sync" told me I had 1049.9 MB
of untracked data. I deleted some and it then said I had 1563.9 MB of
untracked data. Hmmm.
Bug: 376099
Change-Id: I797a4279ad666d83c4a5fbd54bdf7ddbf0e7ad84
Reviewed-on: https://chromium-review.googlesource.com/850854
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
This will allow all uses of this class to correctly move into
gsutil's personal vpython environment. In particular, the
get_toolchain_if_necessary script is pretty sneaky, and adds
depot_tools to sys.path to import and use this class. This
means that if get_toolchain_if_necessary is invoked in a vanilla
vpython environment (or an environment otherwise incompatible with
gsutil.py), it's use of this class will fail.
R=hinoka@chromium.org, tandrii@chromium.org
Bug: 795146
Change-Id: I87309a8b878b465cda7f53db0a41f2f9e6da6f50
Reviewed-on: https://chromium-review.googlesource.com/849663
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Suppose the final paragraph of a commit message looks like this:
"""
And here's the final paragraph.
Bug: 1234
TBR=soandso
Change-Id: deadbeef
"""
In this case, we don't want to lose the Bug and Change-Id footers,
so we process the whole final paragraph. *But* we'd also like to
help the user get things formatted correctly. This change lets
git_footers notice this situation, and insert a newline before the
first well-formed footer (Bug: in this case), so that the set of
well- and mal-formed footers are separated from the rest of the
malformed body text.
In the rare case where the last line of the last non-trailer paragraph
is a url, this will also visibly push the url into the block of
trailers (where it doesn't belong), prompting the user to fix it.
A more comprehensive fix for that particular case is coming later.
Bug: 766234
Change-Id: I6ae0072fff68ddf06e6f43b70f9a82a7f247f4ab
Reviewed-on: https://chromium-review.googlesource.com/849481
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
It's unlikely that anyone would ever intend to have a git trailer
whose key is "http", but since URLs are often long, it is rather
likely that someone would put a URL on a line all by itself. When
that happens, don't accidentally interpret it as a footer.
R=iannucci, tandrii
Bug: 766234
Change-Id: I3101119c4e49e20339487618cc1719452b026d90
Reviewed-on: https://chromium-review.googlesource.com/849484
Commit-Queue: Aaron Gable <agable@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Currently the BUG= line is being concatenated at the end, preventing
Bugdroid from picking it up and bugs from being updated.
BUG=797444
Change-Id: I7242db76a9552658d94fda10ec564b230f4f0037
Reviewed-on: https://chromium-review.googlesource.com/843014
Commit-Queue: Marc-Antoine Ruel <maruel@chromium.org>
Reviewed-by: Marc-Antoine Ruel <maruel@chromium.org>
`download_from_google_storage --config` is currently failing. It still uses gsutil 4.26 but 4.28 has been added recently:
https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/801355
Using 4.28 fixes the bug.
Bug: 796937
Change-Id: I299da7ede8087d832cd6e193d8fc4198a6878c6c
Reviewed-on: https://chromium-review.googlesource.com/840007
Reviewed-by: Jochen Eisinger <jochen@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Joao da Silva <joaodasilva@chromium.org>
recipe-roller
Tom Anderson
Andrii Shyshkalov
Robert Iannucci
Oleh Prypin
Michael Moss
John Budorick
Aaron Gable
Michael Achenbach
Sergey Berezin
Lei Zhang
Edward Lemur
Benjamin Pastene
Nicolas Boichat
Nodir Turakulov
Henrique Ferreiro
Bruce Dawson
Vadim Shtayura
Koji Ishii
Mostyn Bramley-Moore
Takuto Ikuta
Kenneth Russell
Joao da Silva
Mun Yong Jang