This allows the cipd client to process refs with slashes in them.
Also it is compiled with cgo on OSX, which should help with DNS
resolution errors that happen on some machines.
R=vadimsh@chromium.org
Bug: 811974, 803673
Change-Id: If298db5cfe0e10e3734deb4215bf82b95ff5e09e
Reviewed-on: https://chromium-review.googlesource.com/940521
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
When hooks multiple cipd packages under same directory, gclient would
pop up error that same directory name appears multiple times in deps
section. This CL overrides verify_validity for CipdDependency to
support multiple packages hooked in same directory.
Bug:812386
Change-Id: Ia4f1fe0e3c8481c9b06c1d22b6e98d98e1e4c309
Reviewed-on: https://chromium-review.googlesource.com/920686
Commit-Queue: Shenghua Zhang <shenghuazhang@chromium.org>
Reviewed-by: Aaron Gable <agable@chromium.org>
For now 'version' of packages only support string like 'version:27.0.5'.
It is helpful to support version to match vars value, e.g.
Var('tool_version'), especially useful when version numbers could be
modified via api. This change won't affect versions with string format.
Change-Id: I40b389b528f04712798b08f1455869cab2ec54c9
Reviewed-on: https://chromium-review.googlesource.com/935666
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Shenghua Zhang <shenghuazhang@chromium.org>
Over last two weeks of running tryjobs on CLs of others, I found
this very useful. This CL also simplifies code.
R=agable@chromium.org
Change-Id: I5c7ce0d311b1ca024b92227dbec54e5197205c62
Reviewed-on: https://chromium-review.googlesource.com/930742
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
This will allow users on unsupported platforms to successfully bypass vpython
via the recognized environment variable.
R=tandrii@chromium.org, vadimsh@chromium.org
Change-Id: Id63019443321d9f63550fc11e7a4ae3f684b17e7
Reviewed-on: https://chromium-review.googlesource.com/927695
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Python multiprocessing is using sockets to communicate. UNIX socket names are
bound by PATH_MAX which includes temporary directory name.
gsutil uses multiprocessing, which creates file sockets in the tmpdir.
On some testing configurations, the tmpdir is set to a very long path, which
causes gsutil to fail with AF_UNIX path too long errors.
To prevent multiprocess failures, this CL overrides temporary directory
dictated by the environment to /tmp.
Bug: 812581
Change-Id: Idcd99d13b2e20b3095111fa26ec4e242848c8848
Reviewed-on: https://chromium-review.googlesource.com/923170
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Alexander Alekseev <alemate@chromium.org>
A handy usage pattern for autoninja.bat that I was not aware of is to go
autoninja -C out\Default chrome && chrome
This will build chrome and then run it, but only run it if the build
succeeds. The addition of post_build_ninja_summary.py broke this by
losing the error code. This change fixes it by using black magic to
set an error code in the failure case.
Bug: chromium:787983
Change-Id: Ib87fd1799816e19d56de76e08e0f9688be903d80
Reviewed-on: https://chromium-review.googlesource.com/916705
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
post_build_ninja_summary.py gives a summary of a ninja build. It can be
run standalone or it can be run automatically by autoninja. This CL
updates the Python script and the autoninja bash script to make this
work on Linux. This includes removing a zero-value assert, and ensuring
that .so files get categorized as such.
Change-Id: I2d59ab129f5ce70117beeb119719f8432bfbab7c
Reviewed-on: https://chromium-review.googlesource.com/915053
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
When someone runs "gclient diff", they expect to be diffing their
local state against the state of a clean gclient checkout.
Up until now, "gclient diff" has been doing... not that. It's been
assuming that the sha1 in DEPS is approximately the head of the
remote's master branch, finding the merge-base between the local
content and that remote master, and diffing against that merge-base.
This makes basically no sense in the context of gclient. With
gclient, we already know exactly what should be checked out: it's
whatever is in DEPS.
This CL updates the behavior of "gclient diff" to do the right thing.
Bug: 731701
Change-Id: I23a43cbc0d6bc7772495097ff615d4d070f5ad19
Reviewed-on: https://chromium-review.googlesource.com/914717
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
Previously, it counted the number of UTF-8 bytes when spacing out the
table, not the number of code points.
Bug: 808905
Change-Id: Ice5504089e0f7097e108c6dfbbb810620b9dfc94
Reviewed-on: https://chromium-review.googlesource.com/901142
Commit-Queue: Matt Giuca <mgiuca@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Although we want to prevent dfgs from untar'ing files to a parent
or sibling of its target directory, normal files that just happen
to have ".." in their name (i.e. not preceding a path separator) are
okay.
R=hinoka
Bug: 807286
Change-Id: Ibdc2c3615c4778ef66abceb532a4f671fbdab8ef
Reviewed-on: https://chromium-review.googlesource.com/912430
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
Previously, if you used custom metadata keys in a test commit (like
GitRepo.AUTHOR_NAME), they would also be treated as filenames, and fail
because they aren't strings. (This wasn't a problem because it isn't
currently used in any tests.)
Bug: 808941
Change-Id: Id7c4fa5822741925beba591ea587bd8ebbf2e478
Reviewed-on: https://chromium-review.googlesource.com/901122
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
The purpose of this CL is to remove confusion around whether or
not git-cl support the similarity flags when talking to Gerrit.
It does not. However, because Rietveld support has not been
fully ripped out of git-cl, this change does modify some of the
Rietveld-specific code to remove similarity support from everywhere.
Bug: 800902
Change-Id: I8703ed60d6889c7a36400b8a9f8f26de4669020a
Reviewed-on: https://chromium-review.googlesource.com/912389
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
post_build_ninja_summary.py analyzes the .ninja_log file in an output
directory to summarize build performance. It lists the most expensive
build steps, and build-step types, printing both their elapsed time and,
more importantly, their "weighted" time which is a measure of their
contribution to the total build time.
For each time segment within the build the running build steps
accumulate weighted time that is proportional to the elapsed time
divided by the number of tasks running. If a thousand compilation steps
are running in parallel then they will each be "charged" for 1/1,000th
of the elapsed time. If a single link step is running alone then it is
charged with all of the elapsed time.
Bug: chromium:787983
Change-Id: Id5aea715f798a16415dd0365a27f0051202668e5
Reviewed-on: https://chromium-review.googlesource.com/871988
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Erik Staab <estaab@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
After the initial gclient sync, "fetch --no-history" was adding a
fetchspec to the config for the main repository that causes all tags to
be downloaded in future. This means that after fetching a repository
with no history, the very next git pull (or other remote update
operation) will download all the tags, including all the history
referred to by those tags, leaving you with an almost-complete checkout
of the top-level repository that takes up much more space.
Fix this by skipping the "git config" command that requests the tags
when using no-history.
Change-Id: Ia31baa6a90df014ebc1e8c2ed6e05de3ad27d415
Reviewed-on: https://chromium-review.googlesource.com/909653
Reviewed-by: Aaron Gable <agable@chromium.org>
Commit-Queue: Richard Coles <torne@chromium.org>
No code changes, just rebuilt with 1.9.4 that includes potential fix for
deadlocks on Windows.
R=iannucci@chromium.org
BUG=810072
Change-Id: Ic39de760657e86abb5667b6fd9d200e74b140aeb
Reviewed-on: https://chromium-review.googlesource.com/909725
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Vadim Shtayura <vadimsh@chromium.org>
Also, some lines went over 80chars, so I ran 'yapf --style chromium' which
formatted some unrelated things.
BUG=905285
R=dpranke
Change-Id: Iee5f46d88a6e9782612cc4f9e5a2cb72d62ab6af
Reviewed-on: https://chromium-review.googlesource.com/907736
Commit-Queue: Thomas Anderson <thomasanderson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
This is already rolled to all buildbot machines; was waiting to roll
vpython in depot_tools last, but it appears that their builds may
actually depend on the version here.
f88d8d9aa [buildbucket] cancel swarming tasks when deleting builds
0c7d29eb5 Roll recipe dependencies (trivial).
58cd44f90 Roll luci-go.
b3dbb75ba Roll recipe dependencies (trivial).
57743d010 Revert "Remove temporary code used to add https to hosts"
4592a0a20 [buildbucket] refactor task cancelation
76f45a19b Roll recipe dependencies (trivial).
b2fb6be8e [som] Fix some FE unit tests.
57f45c589 [som] Update RELNOTES.md
c7bc0b942 Roll recipe dependencies (trivial).
f91ebe64a SoM: Add autosuggested labels back.
b2de6ae83 [Findit] Translate test failure to event schema
52adb6e67 Roll recipe dependencies (trivial).
efa035c1d [Monorail] Follow-up fix for presubmit browser compat fix
bf5820217 [prpc,luci-auth] Set builds to use static linking.
959f0f9bc SoM: Update ChOpsUI version.
91e92e1e6 Roll luci-go.
ebc1baca9 Roll recipe dependencies (trivial).
47b35eab2 In set review request, use patch set number
886cc525e [buildbucket] no kitchen output file => Infra failure
d73861ca3 [Predator] Increase soft memory for uma sampling profiler.
cd84bd1c5 [buildbucket] refactor build_run_result loading
011e82c92 [Monorail] Fix two issues introduced by presubmit Safari/IE fix
a004798d1 [luci-migration] Surface Infra Failures information.
3a20d2ded Roll recipe dependencies (trivial).
7d82e1b8b Roll recipe dependencies (trivial).
a2eb2e90f [Findit] Flake Analyzer - Run more try jobs.
9a3bd613e Roll recipe dependencies (trivial).
42f912526 Roll recipe dependencies (trivial).
66acb1cd1 Update path in example input in GitFileIsolator test
18b65029f Add a note to the GitRef comment in Tricium AnalyzeRequest
78bd9ceaf [Tricium] Proofread and reformat comments
11d51e0f0 Remove temporary code used to add https to hosts
b6876f848 Fix two minor spelling issues in isolate.go
9aeb8d71f Roll recipe dependencies (trivial).
fffcbaf25 Roll recipe dependencies (trivial).
2d91b6863 [Findit] Instrument bug filing to use CustomFields instead of labels
5a87eb159 Roll luci-go and luci/gae.
37175a98c Roll recipe dependencies (trivial).
965cb361b Revert "Use git.exe and python.exe from %PATH% on Windows"
3ca5b0adb Roll luci-go DEPS
0ae3fcdc2 [test-results] Parse build_id form parameters from uploads.
dd5056295 Roll recipe dependencies (trivial).
896ada564 Roll recipe dependencies (trivial).
71ed7ee94 [som] Add monitoring metrics for alert *groups*
549b91906 Make submodules respect 'condition' for inclusion.
f39b717ce Roll recipe dependencies (trivial).
19eb6271e Roll recipe dependencies (trivial).
75aa8fc6a [Findit] Move swarming related logic only for flake analyzer to a separate module.
6429566ed [Findit] Translate models to BQ schema for test flake & compile failure
8d1731533 Roll recipe dependencies (trivial).
7c8adc760 Roll recipe dependencies (trivial).
a1844f202 [Monorail] Refine the order that we peel off autolinking trailing delimeters.
5409e9f46 Roll recipe dependencies (trivial).
c69c8d781 bot_setup: do not initialize swarming client on multibots
66388a79d Roll recipe dependencies (trivial).
61163aa41 Roll recipe dependencies (trivial).
4c5009d2a Roll recipe dependencies (trivial).
03f5cac69 Use system python for preallocated Swarming bots.
3d0fe245d Roll recipe dependencies (trivial).
0c35aa4a3 [Findit] miscellaneous bug fixes
1cb856336 Roll recipe dependencies (trivial).
f1e5e0076 [doc] Document permissions for cross-project joins
bb4241f99 [Monorail] Fix typo in SQL declarations.
b8e22ed36 Roll recipe dependencies (trivial).
7f8d1f1ab [buildbucket] make timing metrics cumulative
1d7d9ffa0 Roll recipe dependencies (trivial).
6afb9a8c8 Roll recipe dependencies (trivial).
bd4151e59 Roll recipe dependencies (trivial).
d450c3faf Roll recipe dependencies (trivial).
783bc9322 Roll recipe dependencies (trivial).
f2770a571 Roll recipe dependencies (trivial).
e9b8e600b Roll recipe dependencies (trivial).
03db6cde2 [Findit] move all gtest specific logic to services/gtest.py
c4884b963 Use GCE service account when fetching Swarming bot code.
a4f61f59e Roll recipe dependencies (trivial).
1ebe658ce Gen some ssh keys on cros container startup.
382263cf8 Roll recipe dependencies (trivial).
26b8b693e Point swarm*-c7 to chrome-swarming.appspot.com
d0b138cd0 [Predator] Strip parentheses in function name when matching with crash_state.
b35020f73 Allow force-version and migrate-to to accept versions greater than 9
2f0a1df32 Roll recipe dependencies (trivial).
5c77e779e Roll recipe dependencies (trivial).
6cb457523 [Monorail] Add instructions for running one unit test
64291222f Roll recipe dependencies (trivial).
e50870d52 [Monorail] Fix presubmit JS incompatibility
7d8342ee3 Roll luci-go DEPS.
afaeeb0fc [Tricium] Include notify: NONE when posting to Gerrit
a2985c216 [Findit] Refactoring BigQuery schema for analysis events
f8272181b [Monorail] Make autocomplete dropdown accessible for ChromeVox users.
d31763ca4 [Findit] Add bigquery helper that interacts with BQ REST API
d4dc0c29a mac_toolchain: add support for legacy Xcode 8.*
a304231ca Roll recipe dependencies (trivial).
6ec369965 Roll recipe dependencies (trivial).
4d60ec8f8 Roll recipe dependencies (trivial).
07e3fd5ed Roll recipe dependencies (trivial).
390212979 [Findit] Move gtest related logic to gtest.py (1/2)
d5e83d77d Roll recipe dependencies (trivial).
5df250555 Generalize android_docker image building for CrOS as well.
ed978bda5 [buildbucket] use swarming's timestamps
a14f4bfb9 Roll recipe dependencies (trivial).
68f043ea7 Roll recipe dependencies (trivial).
a926b8f75 [Monorail] Autolink text in bigger issues.
63a39573b [som] Update RELNOTES.md
3ae74f454 Roll recipe dependencies (trivial).
4d5af48ff Roll recipe dependencies (trivial).
9d0c9b8b3 Roll recipe dependencies (trivial).
cd4db1329 Add approval_id to FieldDef table.
3ad142a2c [som] Increase client RPC timeouts to 1 minute
eaaa18b94 Roll recipe dependencies (trivial).
09b3ab3ff Roll recipe dependencies (trivial).
64bcfd437 Roll recipe dependencies (trivial).
2faa1f2ac Roll recipe dependencies (trivial).
2e0feb96a Roll recipe dependencies (trivial).
86e40f0c0 Bootstrap gn with clang
9c305f17f [Findit] Ignore IntelliJ project files
17bd7606d [bbroll] Fix other link :)
94c06b37a Add IssueSnapshot table SQL schema and join tables
b0f8ad63f [Findit] Flake Analyzer - New DetermineApproximatePassRatePipeline
89dfd42af Roll recipe dependencies (trivial).
00ac3227d Roll recipe dependencies (trivial).
aab58f01d Roll recipe dependencies (trivial).
c643aedfe [bbroll] Fix canary links.
52f211da7 Monorail: Allow specifying one label in an email address for ticket alerts.
dd7df86a2 [bq events] Update docs with comments about SoM and test-results tables.
TBR=vadimsh@chromium.org
Bug: 808434
Change-Id: Id16a8758257ad0c531b0a590e923b3b4d3442397
Reviewed-on: https://chromium-review.googlesource.com/905830
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
This reverts commit fa573785df.
Reason for revert: broke CQ on branches.
Example: https://ci.chromium.org/p/chromium/builders/luci.chromium.try/chromium_presubmit/28220
Original change's description:
> bot_update: allow rebasing the patch onto an older revision.
>
> When applying a rebase, we normally go from an older base commit to a newer one.
>
> A ---- B ---- C ---- D -origin/master
> \
> E -branch
>
> In this case, `git rebase D` would certainly work as expected. However,
> writing `git rebase B` would NOT get us to the following state:
>
> A ---- B ---- C ---- D -origin/master
> \
> E' -branch
>
> In fact, it would have no effect.
>
> This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
> explains the general invocation as
> > `git rebase --onto <graft-point> <exclude-from> <include-from>`
> > If you don’t specify --onto, <graft-point> defaults to <exclude-from>
>
> So what's happening is, by writing `git rebase B` we're rebasing onto B,
> excluding commits that are in B. Commit C is not "in" B so it is kept and we're
> back to the starting point.
>
> So I suggest to change the invocation to `git rebase --onto B origin/master`,
> which rebases onto B, excluding commits that are in origin/master. This works
> more generally and allows rebasing "backwards".
>
> Bug: None
> Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
> Reviewed-on: https://chromium-review.googlesource.com/904004
> Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
> Commit-Queue: Oleh Prypin <oprypin@chromium.org>
TBR=iannucci@chromium.org,tandrii@chromium.org,oprypin@chromium.org
Change-Id: I7243641d84428bf4504ee798bf31ad0afbfd4865
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: None
Reviewed-on: https://chromium-review.googlesource.com/905862
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
This will be used when running infrastructure code locally
(e.g. recipes) to correctly interact with LUCI service APIs
(such as luci-scheduler.appspot.com).
It is also generically useful to explore and interact with LUCI
service APIs on the command line (for debugging/scripting).
R=tandrii@chromium.org, vadimsh@chromium.org
Bug: 808677
Change-Id: I41cfd4cc7e2d245d3a5d2be83f9879f92a8d1bca
Reviewed-on: https://chromium-review.googlesource.com/905457
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Goal: reduce number of git fetches from remote to the minimum possible.
Minimum possible is either:
* 0 if desired revision is pinned (!= HEAD) and already exists in cache
* 1 if desired revision is HEAD
* >= 2 if revision is pinned but we talk temporarily
out-of-date git server not yet having it.
This CL achieves the above.
No fetch:
https://ci.chromium.org/swarming/task/39dad20230c84110?server=chromium-swarm.appspot.com
bot_update takes <1min, but it used to take at least 2. The steps are now like this:
...
===Running git cat-file -e 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
In directory: /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
===Succeeded in 0.0 mins===
===Running /b/swarming/w/ir/cipd_bin_packages/bin/python -u /b/swarming/w/ir/recipe-checkout-dir/depot_tools/git_cache.py exists --quiet --cache-dir /b/swarming/w/ir/cache/git https://chromium.googlesource.com/chromium/src.git===
In directory: /b/swarming/w/ir/cache/builder/linux
/b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
===Succeeded in 0.0 mins===
===Running git remote set-url origin /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src===
In directory: /b/swarming/w/ir/cache/builder/linux/src
===Succeeded in 0.0 mins===
===Running git fetch origin===
In directory: /b/swarming/w/ir/cache/builder/linux/src
From /b/swarming/w/ir/cache/git/chromium.googlesource.com-chromium-src
003eedbfee47..5210107a20e8 master -> origin/master
===Succeeded in 0.0 mins===
===Running git checkout --force 33bf8a94dcd5e0abbdf83e4afaed24b5180e3eb2===
...
Fetch & success:
https://ci.chromium.org/swarming/task/39db022b874ffd10?server=chromium-swarm.appspot.com
Fetch & retry because missing rev:
https://ci.chromium.org/swarming/task/39db04ce8b4b8910?server=chromium-swarm.appspot.com
Fetch because rev=HEAD:
https://ci.chromium.org/swarming/task/39db0c0dd5ab5d10?server=chromium-swarm.appspot.com
This has already been adopted in gclient.
Bug:
Change-Id: Id99892b62719fdf3f7e6e59058986d1500384f8d
Reviewed-on: https://chromium-review.googlesource.com/771591
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Jao-ke Chin-Lee <jchinlee@chromium.org>
This CL adds target_cpu and target_cpu_only variables. The logic is:
* If target_cpu is defined in .gclient, use it. Otherwise, initialize it as an
empty list.
* If not target_cpu_only, add the host arch to the list too.
detect_host_arch.py is copied directly from the Chromium repo.
BUG=807986
R=dpranke
Change-Id: I27621cbc81ad6a844648525863b92ffdd3b1d03a
Reviewed-on: https://chromium-review.googlesource.com/902441
Commit-Queue: Thomas Anderson <thomasanderson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
When applying a rebase, we normally go from an older base commit to a newer one.
A ---- B ---- C ---- D -origin/master
\
E -branch
In this case, `git rebase D` would certainly work as expected. However,
writing `git rebase B` would NOT get us to the following state:
A ---- B ---- C ---- D -origin/master
\
E' -branch
In fact, it would have no effect.
This article http://matthew-brett.github.io/pydagogue/rebase_without_tears.html
explains the general invocation as
> `git rebase --onto <graft-point> <exclude-from> <include-from>`
> If you don’t specify --onto, <graft-point> defaults to <exclude-from>
So what's happening is, by writing `git rebase B` we're rebasing onto B,
excluding commits that are in B. Commit C is not "in" B so it is kept and we're
back to the starting point.
So I suggest to change the invocation to `git rebase --onto B origin/master`,
which rebases onto B, excluding commits that are in origin/master. This works
more generally and allows rebasing "backwards".
Bug: None
Change-Id: I68e4d805811530b585550bc75099354fef4e9c15
Reviewed-on: https://chromium-review.googlesource.com/904004
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Oleh Prypin <oprypin@chromium.org>
The first call to gsutil causes a network call and makes the download
script slow also in the most optimal cases. This CL refactors the
download script and moves the first gsutil call after checking locally
if sha1s match.
1) This turns the input acquisition into a generator and buffers the
files and sha1s in a list before multithreading.
2) This sequentially checks the sha1s and files and bails out early if
all match. In Chrome-land, we usually call this script with only one
file. There are some cases with around 4. This could also be
parallelized if the need arises.
3) The initial gsutil check, which ensures gsutil is updated, is moved
right in front of the multithreaded downloads.
The performance of one call to download_from_google_storage for an
existing 500MB file is 2.3s before this CL and 1.2s after (most of the
remaining time left is spent for making sha1sum).
Example for full gclient runhooks (when everything is up-to-date):
Chromium: 32s before, 12s after
V8: 12s before, 3s after
Bug: 776311
Change-Id: Ia7715a6af84b1b336455ea88494d399bdb050317
Reviewed-on: https://chromium-review.googlesource.com/897562
Commit-Queue: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Sergiy Byelozyorov <sergiyb@chromium.org>
Reviewed-by: Ryan Tseng <hinoka@chromium.org>
This is needed for run_presubmit.py to work.
BUG=pdfium:971
Change-Id: Ib598fbfd6eab3c0b7628301eb3bd54c548a4885e
Reviewed-on: https://chromium-review.googlesource.com/898039
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Lei Zhang <thestig@chromium.org>
Robert Iannucci
Jao-ke Chin-Lee
Aaron Gable
Michael Achenbach
Shenghua Zhang
Andrii Shyshkalov
Daniel Bratell
recipe-roller
Henrique Ferreiro
Nodir Turakulov
John Budorick
Alexander Alekseev
Bruce Dawson
Matt Giuca
Torne (Richard Coles)
Vadim Shtayura
Tom Anderson
Edward Lemur
Oleh Prypin
Michael Moss
Sergey Berezin
Lei Zhang