This reverts commit b5c7f4b46c.
Reason for revert: missing a replace for urlparse.urlparse
Original change's description:
> Drop py2 support in gerrit and git related files
>
> python3 is the only supported version of python in depot_tools.
>
> Bug: 1475402
> Change-Id: Ie4ee18d297081b3aa0206b8d7ce6461819bff0ca
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4809560
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Commit-Queue: Gavin Mak <gavinmak@google.com>
Bug: 1475402
Change-Id: Idd00fdfe0b3d62785da2789a7dfcc9fbc79b6385
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4811623
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
python3 is the only supported version of python in depot_tools.
Bug: 1475402
Change-Id: Ie4ee18d297081b3aa0206b8d7ce6461819bff0ca
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4809560
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Gavin Mak <gavinmak@google.com>
Upon Gerrit team's suggestion, up the retry time and number of retries
to handle increased replication delay.
Bug:b/285164390
Change-Id: If833ae9bb0f8c276b761970a8b5f96ec217d11b9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4621268
Commit-Queue: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Auto-Submit: Gavin Mak <gavinmak@google.com>
https://crrev.com/c/4420526 put a 10s timeout on all calls made via
this script.
But it appears there are a variety of endpoints that take more than
10s, the latest being "SubmitChange", eg:
https://ci.chromium.org/ui/p/chromeos/builders/release/release-R113-15393.B-orchestrator/25/overview
So instead of applying the timeout to everything, and then opting
certain long-running calls out of it, this just opts-in the
"GetChanges" call to the timeout. This should cover the failure mode
that was originally solved for chrome's bots. And also bump the
timeout to 30s since we don't trust all get-changes queries to
reliably finish in under 10s.
This also bumps the step level timeout for the query in tryserver
recipe_mod to 8 min since there could be ~5min worth of sleeping +
150s worth of waiting+timing-out. So worst case the step will now
take 8min to fully exhaust all timeouts/sleeps.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: b/278083716
Change-Id: Ib366e004e0bb07297ba732590d488cae779e38ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4426524
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
When gerrit's availability drops, chrome's builds see an excessive
amount of failures, eg:
https://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359780/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359594/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359564/overviewhttps://ci.chromium.org/ui/p/chromium/builders/try/linux-rel/1359630/overview
Seemingly all failures occur in either the `gerrit fetch current CL
info` step or the `gerrit changes` step. Both steps have a 60s timeout
and shell out to depot_tools' gerrit_client.py. That script essentially
makes a single http call to gerrit. That request has no configured
timeout. So when gerrit's MIA and the call hangs indefinitely, so too
will the step hang. 60s after that, the step timeout is reached, and the
entire build crashes with an infra-failure.
However, one single retry has been shown to sufficiently work around
at least one instance of that failure:
dea9a6eca26f690ce10a959701a8fb51ad/+/build.proto
So this incorporates timeouts into the requests made by
gerrit_util.py. Each request is given a 10s timeout, which should be
enough for most/all gerrit calls. (Both steps have a p90 of less than
1sec.) When a timeout is reached, the script will automatically retry
up to 4 times.
This also bumps the timeouts of the step calls to gerrit_client.py to
6min since the script can now take up to 5 minutes to fail, ie:
the sequence of sleeps is roughly 10s, 20s, 40s, 80s, 160s, which is
about 5min. So a 6min timeout should cover all those retries.
This also passes in "--verbose" to all step calls to this script, so
the logging that prints out the retry + sleep log lines will be
visible in build logs.
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Bug: 1432638
Change-Id: I9dc47f4beeda3783ae4f9152bd29ee441ac3e197
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/4420526
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Gerrit rejects requests with 400 Bad request if query string `q`
contains any spaces. Replace spaces with a plus sign solves the problem.
R=thakis@chromium.org
Bug: 1199692
Change-Id: Ic13dda378527594c6cf57b8cb2edf740517811ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2832653
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
We've had reliability issues with gerrit, primarily related to
429 status codes but also DDoS bans. The light DDoS bans can be
very short lived and we extend retries to handle this. Longer
term bans can take up to an hour to lift.
BUG=chromium:1071590
TEST=./gerrit_client_test.py
Change-Id: Iaf68c0d9cc7375aa58367ec0447d54a99f8ebf39
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2153089
Commit-Queue: George Engelbrecht <engeg@google.com>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Mike Frysinger <vapier@chromium.org>
Decode the result of reading http.cookiefile from git config
Bug: 1066992
Change-Id: I1d4f2c8e54f717ef61ab4e630f3f66206c56047e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2132774
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This is a reland of 5bfa3ae88d
Replace cStringIO with StringIO and add tests.
Original change's description:
> gerrit_util: Refactor ReadHttpResponse and add more tests.
>
> Bug: 1016601
> Change-Id: Ie6afc5b1ea29888b0bf40bdb39b2b492d2d0494c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1880014
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 1016601
Change-Id: I0c83a83202169b6a1acc60bdf6f4cd00eac6e2a6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1884461
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This reverts commit 5bfa3ae88d.
Reason for revert:
Fails when fetching status for cl 1708084
Original change's description:
> gerrit_util: Refactor ReadHttpResponse and add more tests.
>
> Bug: 1016601
> Change-Id: Ie6afc5b1ea29888b0bf40bdb39b2b492d2d0494c
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1880014
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=ehmaldonado@chromium.org,apolito@google.com
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 1016601
Change-Id: I84bc378ed5f58e911e0900b1a5dbea70dc06ade1
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1883677
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Gavin Mak
Ben Pastene
Aravind Vasudevan
Josip Sokcevic
Edward Lesmes
George Engelbrecht
Edward Lemur