Commit Graph

29 Commits (a3a95d439bafcdb4c46da65d9be8e8fd5358c41a)

Author SHA1 Message Date
Aravind Vasudevan 23ddab2235 Reland "Break make_encoded_file into two functions"
This is a reland of commit 5e49eda5c4

Original change's description:
> Break make_encoded_file into two functions
>
> The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.
>
> Recipe-Manual-Change: chromiumos
> Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Greg Edelston <gredelston@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Recipe-Manual-Change: chromiumos
Change-Id: I321f797ae4a00deed7920ee6f80d666954b07c7d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3625907
Reviewed-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Takuto Ikuta 8a87603683 Revert "Reland "Break make_encoded_file into two functions""
This reverts commit 41c57603f9.

Reason for revert:
This broke depot tools CQ.
e.g.
https://ci.chromium.org/ui/p/infra/builders/try/ChromiumOS%20downstream%20Recipe%20Roll%20tester%20from%20Depot%20Tools/b8815683872106857201/overview

Please prepare roll CL on chromeos side when you reland this.

Original change's description:
> Reland "Break make_encoded_file into two functions"
>
> This is a reland of commit 5e49eda5c4
>
> Original change's description:
> > Break make_encoded_file into two functions
> >
> > The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.
> >
> > Recipe-Manual-Change: chromiumos
> > Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
> > Reviewed-by: Gavin Mak <gavinmak@google.com>
> > Reviewed-by: Greg Edelston <gredelston@google.com>
> > Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
>
> Recipe-Manual-Change: chromiumos
> Change-Id: I2de85eca71cc45f503c25888297ba881a5c662a8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3610973
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Commit-Queue: Greg Edelston <gredelston@google.com>

Change-Id: I7718ab01f4249c7dc2d978c12fb3a26e3ece91b6
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3612484
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
3 years ago
Aravind Vasudevan 41c57603f9 Reland "Break make_encoded_file into two functions"
This is a reland of commit 5e49eda5c4

Original change's description:
> Break make_encoded_file into two functions
>
> The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.
>
> Recipe-Manual-Change: chromiumos
> Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Greg Edelston <gredelston@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Recipe-Manual-Change: chromiumos
Change-Id: I2de85eca71cc45f503c25888297ba881a5c662a8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3610973
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
3 years ago
Aravind Vasudevan ada0ebd47d Revert "Break make_encoded_file into two functions"
This reverts commit 5e49eda5c4.

Reason for revert: chromiumos rollout fails

Original change's description:
> Break make_encoded_file into two functions
>
> The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.
>
> Recipe-Manual-Change: chromiumos
> Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
> Reviewed-by: Gavin Mak <gavinmak@google.com>
> Reviewed-by: Greg Edelston <gredelston@google.com>
> Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>

Change-Id: Ibfb51614f9e19b575b99704aa4ca33c5525ab294
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3602168
Auto-Submit: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
3 years ago
Aravind Vasudevan 5e49eda5c4 Break make_encoded_file into two functions
The python3 fallback for `make_encoded_file()` produces different results on py2 and py3 when bytes are passed. This CL makes it clear breaking it into two methods, one for text and another for bytes. This CL also adds a fallback in `download_file` for py3 to return bytes when it cannot decode the downloaded file into utf-8.

Recipe-Manual-Change: chromiumos
Change-Id: I3d313e430c852e179825bc24bf4a58ce84440b2a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3595026
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Aravind Vasudevan <aravindvasudev@google.com>
3 years ago
Josip Sokcevic 274ada0567 Reland "Use py3 in gitiles recipe module"
This is a reland of 0faae1c845

Original change's description:
> Use py3 in gitiles recipe module
>
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: chrome_release
> Recipe-Nontrivial-Roll: chromiumos
> Bug: 1289280
> Change-Id: I182f0e7ad1da58c1c707edd15675f6d30c5faed8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3438327
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Bug: 1289280
Change-Id: I9ba63a7cd4fea99c2043fae546c63ef942c82802
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3445785
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Josip Sokcevic 2721cc95da Revert "Use py3 in gitiles recipe module"
This reverts commit 0faae1c845.

Reason for revert: broke dawn and some chromeos builds

Original change's description:
> Use py3 in gitiles recipe module
>
> Recipe-Nontrivial-Roll: build
> Recipe-Nontrivial-Roll: chrome_release
> Recipe-Nontrivial-Roll: chromiumos
> Bug: 1289280
> Change-Id: I182f0e7ad1da58c1c707edd15675f6d30c5faed8
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3438327
> Reviewed-by: Josip Sokcevic <sokcevic@google.com>
> Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
> Commit-Queue: Gavin Mak <gavinmak@google.com>

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Bug: 1289280
Change-Id: I09f2860bac9fbac4c2edb0b77baea3802b077958
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3444000
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
3 years ago
Gavin Mak 0faae1c845 Use py3 in gitiles recipe module
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Bug: 1289280
Change-Id: I182f0e7ad1da58c1c707edd15675f6d30c5faed8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3438327
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com>
Commit-Queue: Gavin Mak <gavinmak@google.com>
3 years ago
Scott Lee 376ce660a5 Remove tandrii@ from OWNERS under depot_tools
Missing changes from https://chromium-review.googlesource.com/q/owner:tandrii%2540google.com+%2522remove+tandrii%2540%2522
(and nodir@)

Change-Id: I9ae573f2b01bcc5e65e17202b90bc724210cd971
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3252670
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Scott Lee <ddoman@chromium.org>
4 years ago
Gavin Mak d6d5c46cf4 Add PYTHON_VERSION_COMPATIBILITY to gitiles and bot_update
Setting PYTHON_VERSION_COMPATIBILITY = PY2+3 makes "recipe.py test" run
tests in both py2 and py3.

Bug: 1227140
Change-Id: I56a47f0491cbade172e1dc1239cbf6c2a0faa64f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3116850
Commit-Queue: Gavin Mak <gavinmak@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
4 years ago
Josip Sokcevic ce332a47a9 Use main as default branch in all recipes
Recipe-Manual-Change: build
Recipe-Nontrivial-Roll: build_limited
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: infra
Change-Id: I89008bc07b1c3de13d9c6dd64e723fe6dc8d01d6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3015262
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
4 years ago
Gavin Mak a138f082de Add python3 support to gitiles recipe module
This change omits python3 support for resources/gerrit_client.py

Bug: 1227140
Change-Id: Ibc8d9f1fbd28008959991688ced5818f0188f905
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/3012233
Commit-Queue: Anthony Polito <apolito@google.com>
Reviewed-by: Anthony Polito <apolito@google.com>
4 years ago
Josip Sokcevic 848a5361d7 Stop using /usr/bin/python directly
Use /usr/bin/env python instead

R=dpranke@google.com

Bug: 1191100
Change-Id: I8de37ea7a02e7236d3bd36d4bae8cf4ffcf75ab9
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2780631
Auto-Submit: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Dirk Pranke <dpranke@google.com>
Reviewed-by: Dirk Pranke <dpranke@google.com>
4 years ago
Quinten Yearsley b674f8a277 Edit depot_tools recipe_modules docstrings for consistency
Change-Id: I454e9c90d3159598ae34d97d22c96b9d20d55f80
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2521656
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
5 years ago
Andrii Shyshkalov 7b7bdcd075 tandrii: switch OWNERS to @google.com account.
R=tikuta
TBR=tandrii@chromium.org

Change-Id: I8ce4eb78e8ec2cf4c6d816b8a5ca63dbb0de66c6
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2411985
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Quinten Yearsley 925cedba7b Run spellchecker on depot_tools
This was made by running `codespell` and `scspell`
and then checking the results.

Change-Id: I169fd5b40294f83015075b4a899fbca263821f25
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2144602
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Auto-Submit: Quinten Yearsley <qyearsley@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Marc-Antoine Ruel 8e57b4bc55 python3 improvements
Ran:
  vi $(git grep --name-only iteritems | grep -v third_party)
  vi $(git grep --name-only itervalues | grep -v third_party)
  vi $(git grep --name-only 'print ' | grep -v third_party)

and edited the files quickly with adhoc macros. Then ran in recipes/:
  ./recipes.py test train

There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.

Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>
6 years ago
recipe-roller b712f77706 Roll recipe dependencies (nontrivial).
This is an automated CL created by the recipe roller. This CL rolls recipe
changes from upstream projects (e.g. depot_tools) into downstream projects
(e.g. tools/build).


Please review the expectation changes, and LGTM+CQ.


More info is at https://goo.gl/zkKdpD. Use https://goo.gl/noib3a to file a bug.
recipe_engine:
  https://crrev.com/b5922385482112ce31df891af71854c81662d69e Remove jsonResult defaulting to null (martiniss@google.com)


R=recipe-roller@chromium.org, tandrii@chromium.org

Recipe-Tryjob-Bypass-Reason: Autoroller
Bugdroid-Send-Email: False
Change-Id: I9791f5b80afe6529a6321eccbd4ae7d1a3f18a81
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1638960
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@chromium.org>
6 years ago
Stephen Martinis 9afc6490c1 Manual roll for new run logic
Manual roll for https://crrev.com/c/1239574/

TBR=iannucci

Recipe-Manual-Change: build
Recipe-Manual-Change: infra
Recipe-Manual-Change: build_limited_scripts_slave
Recipe-Manual-Change: release_scripts
Recipe-Manual-Change: skiabuildbot
Recipe-Manual-Change: skia
Bug: 646142
Change-Id: I56f156c05a695cfb00b9e0a7f335d516153a70c7
Reviewed-on: https://chromium-review.googlesource.com/c/1305041
Commit-Queue: Stephen Martinis <martiniss@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>
7 years ago
Nodir Turakulov 4e3920bef0 [gclient] Start actively using repo_path_map
Use repo_path_map in all places where patch_projects is currently used.
Add gclient.get_repo_path function to resolve path to a repo given its URL.

infra.git requires a non-trivial roll because of incorrect expectations.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: infra
Bug: 877161
Change-Id: Iaea0a3f14609bf0c18d4ecdf8564153b8052584d
Reviewed-on: https://chromium-review.googlesource.com/1208363
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Nodir Turakulov f215ae6ed0 [gitiles] Add parse_repo_url function
Add a function to parse a Gitiles repo URL supporting various edge cases.

R=tandrii@chromium.org

Bug: 877161
Change-Id: Ib4d6a28a2ace2e594f1a4ca31460a446fc6fa052
Reviewed-on: https://chromium-review.googlesource.com/1204914
Commit-Queue: Nodir Turakulov <nodir@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Robert Iannucci a4dec94a1a Add download_archive to gitiles module.
R=phosek@chromium.org, tandrii@chromium.org, vadimsh@chromium.org

Bug: 790650
Change-Id: Ia878004d6983dbbad882ec683da2e0db5e727c66
Reviewed-on: https://chromium-review.googlesource.com/1116073
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@chromium.org>
7 years ago
Nico Weber 6ceb345389 Remove dnj from depot_tools OWNERS
Add some other people to recipes/OWNERS

Change-Id: I554d1fd354e628d2e62629be7ce20e899364c4a9
Reviewed-on: https://chromium-review.googlesource.com/984972
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
7 years ago
Michael Moss 7de54ef0a2 Fix handling of potential empty gitiles result.
This fixes an issue introduced by
crrev.com/b40a45149a73a6b00e92fd1d2df4c0b1543d0005 where a 404 result
might return None contents.

R=agable@google.com, phajdan@google.com

Bug:774355
Change-Id: I993a029adb8d92cb27e043cd3d9b17cc8aee2432
Reviewed-on: https://chromium-review.googlesource.com/717937
Commit-Queue: Michael Moss <mmoss@chromium.org>
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
8 years ago
Michael Moss b40a45149a Allow specifying alternative "success" codes in gitiles requests.
This is needed in the release recipes because there are times when we
want to check that a file is _not_ there yet (i.e. 404 response).

R=agable@chromium.org, dnj@chromium.org

Change-Id: Iedf646d6c73ebf898e90afd2009840b1e5c5b1ab
Reviewed-on: https://chromium-review.googlesource.com/709916
Reviewed-by: Paweł Hajdan Jr. <phajdan.jr@chromium.org>
Commit-Queue: Michael Moss <mmoss@chromium.org>
8 years ago
Aaron Gable ae895075ab Use doseq=True when re-encoding url query parameters
The query parameters are parsed out of a string using
urlparse.parse_qs, which produces a dictionary whose values
are lists of values.

The query parameters are then stuffed back into a string
using urlencode.urlencode, which uses percent-encoding to
encode lists as literal sequences of brackets and commas,
resulting in parse_qs and urlencode not being inverse
operations.

This change adds the doseq flag to urlencode, thus making
it encode each value from the list of values as a separate
key=value pair, thus making it a true inverse of parse_qs.

R=dnj@chromium.org

Change-Id: Ic4631cc672f9e42ba371306efdda71892d625119
Reviewed-on: https://chromium-review.googlesource.com/575398
Reviewed-by: Daniel Jacques <dnj@chromium.org>
Commit-Queue: Aaron Gable <agable@chromium.org>
8 years ago
Robert Iannucci 56038f517d rename example.py -> examples/full.py for all modules
R=dnj@chromium.org, nodir@chromium.org, phajdan.jr@chromium.org

Bug:
Change-Id: I47d8f9311b6ee71583fb989888a1e4d28b5495df
Reviewed-on: https://chromium-review.googlesource.com/520731
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Nodir Turakulov <nodir@chromium.org>
8 years ago
Dan Jacques 69e760e4a5 Switch to recipe engine "url" module.
BUG=None
TEST=expectations
R=iannucci@chromium.org

Change-Id: I43a65405c957cb6dddd64f61846b926d81046752
Reviewed-on: https://chromium-review.googlesource.com/505278
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
8 years ago
Dan Jacques 9d2b4b235c Move tools/build "url" and "gitiles" modules.
Add "tools/build"'s "url" and "gitiles" recipe modules into
"depot_tools". "gitiles" had an explicit dependency on "depot_tools", so
this formalizes this dependency.

To support "url", we vendor "requests" from "tools/build" into
"depot_tools" for it.

BUG=None
TEST=expectations

Change-Id: I1ba634ae264b5c9d3adee20db3397c83a6c52c45
Reviewed-on: https://chromium-review.googlesource.com/483316
Reviewed-by: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Daniel Jacques <dnj@chromium.org>
8 years ago