This will allow us to separate the 'populate' and 'upload' phases
in the git cache updater recipe, which will allow us to do all the
'populate' steps in parallel, but then limit the parallelism of the
`pack/gc/upload` operations.
R=tandrii@chromium.org
Change-Id: I8b8a9155f86350be37ed5a67c592ff1fec4d42ff
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1830857
Auto-Submit: Robbie Iannucci <iannucci@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
This is a reland of e656e98831
Original change's description:
> gclient: Add a GCLIENT_PY3 env var to allow users to execute gclient using Python 3.
>
> Bug: 1008202
> Change-Id: I23dd70b72089a09c2a7cdc74e2a63c135a924ebe
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1825996
> Reviewed-by: Anthony Polito <apolito@google.com>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Bug: 1008202
Change-Id: I5181a7f8efb39bdd6adc99db90663cf3cf3cd0bf
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1828334
Reviewed-by: Anthony Polito <apolito@google.com>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
CheckCallAndFilter isn't part of the global namespace, and as a result,
this call was silently failing.
BUG=1007553
Change-Id: Ie2f3135fc3d56be189d8be0fe996a068e0645705
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1823664
Auto-Submit: Robert Liao <robliao@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Robert Liao <robliao@chromium.org>
I know that sometimes imports can have side-effects,
so unused imports shouldn't always be removed, but these
ones look like they could be.
Change-Id: Iea9f82afa99b0ea35f29a28f20ce0493b579cfee
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1819860
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
This reverts commit 82ae4b4b7d.
Reason for revert: didn't help. Something else limits 'git gc' by 2GB.
However, even so, total amount of .pack files for chromium/src is <7.5GB.
Original change's description:
> git-cache: don't limit pack files to 2GiB.
>
> This is only relevant to 1 builder which updates git cache bundles.
> Since some time in 2018-2019, it started creating several 2GiB packs
> for chromium/src. I verified that the setting affecting it is
> core.deltaBaseCacheLimit
>
> Log file of a build which I observed while ssd-ed into the machine (internal):
> https://logs.chromium.org/logs/infra-internal/buildbucket/cr-buildbucket.appspot.com/8901682638839779824/+/steps/Updating_https:__chromium.googlesource.com_chromium_src/0/stdout
> Git version: version:2.21.0.chromium16
>
> R=ehmaldonado
>
> Change-Id: I52cadfb9f34faea09a57d53387cab7e0538362b9
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1818076
> Auto-Submit: Andrii Shyshkalov <tandrii@google.com>
> Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=tandrii@google.com,ehmaldonado@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Change-Id: I24915555c703bc92fda921524820405de5d69f7d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1817101
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
There are many functions in git_cl that appear unused, although
it's possible that some of them are used in other places, where
git_cl may be imported as a library.
This CL just removes some of the private ones, which at least
were not supposed to be used from other places.
Things removed here:
- _get_commiter_timestamp
- _git_amend_head
- GitCookiesChecker._get_usernames_of_domain
- constants MERGE_BRANCH, CHERRY_PICK_BRANCH;
the comment says that these are supposed to
be used in CMDland, but apparently they aren't.
contextlib which is only used by this, is also
removed
- CodereviewSettingsFileMock in git_cl_test;
it is not used anywhere in the test, and I
assume that nobody's importing and using
git_cl_test...
git_cl_test still passes, git cl seems to still work
Change-Id: Ib8eee0dfb181c8cda04c0894d2daea52b3ad42ac
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1818621
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Quinten Yearsley <qyearsley@chromium.org>
Note that on line 504 I also fixed the test on windows by allowing \. Also properly escaped /.
Errors:
C:\Google\depot_tools\tests\gclient_scm_test.py:42: SyntaxWarning: invalid escape sequence \[
TIMESTAMP_RE = re.compile('\[[0-9]{1,2}:[0-9]{2}:[0-9]{2}\] (.*)', re.DOTALL)
C:\Google\depot_tools\tests\gclient_scm_test.py:504: SyntaxWarning: invalid escape sequence \.
"breaking lock.*\.git/index\.lock")
Bug: 958138
Change-Id: I19ef98538a1c172d7139ca2d7097a7fbc33ebe80
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1815866
Commit-Queue: Raul Tambre <raul@tambre.ee>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
This will be used on internal cache updater.
For instance, I've just compressed chromium/src resulting bootstrap
files from 20GiB to 7.5 GiB.
R=ehmaldonado
Change-Id: I15411700eb2ac3a26d1c658a12288cc49e48fd48
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1802877
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Andrii Shyshkalov <tandrii@google.com>
This is a reland of 3f79763629.
Last time it broke a skia test. The test has since been fixed
in https://crbug.com/1005388
Original change's description:
> Set core.trustctime=false during git checkout operations.
>
> This should be faster and more general than running
> `git index --refresh` over all git repositories we can find.
>
> See "Proposal: Setting core.trustctime=false on Chromium's bots"
> on infra-dev (linked to from bug 330461 comment 45).
>
> Bug: 330461,642711,746763,740301.
> Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Nico Weber <thakis@chromium.org>
Recipe-Nontrivial-Roll: chromiumos
Bug: 330461, 642711, 746763, 740301.
Change-Id: I5d09a93689e53e4339bff7b1173df938aee73a4d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1810400
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
This reverts commit 3f79763629.
Reason for revert:
Breaks Skia https://skia-review.googlesource.com/c/buildbot/+/242590
Original change's description:
> Set core.trustctime=false during git checkout operations.
>
> This should be faster and more general than running
> `git index --refresh` over all git repositories we can find.
>
> See "Proposal: Setting core.trustctime=false on Chromium's bots"
> on infra-dev (linked to from bug 330461 comment 45).
>
> Bug: 330461,642711,746763,740301.
> Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
> Commit-Queue: Nico Weber <thakis@chromium.org>
TBR=thakis@chromium.org,tandrii@google.com
Change-Id: I908ce92748f7b516efed5f2c02138ecfee306c88
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Bug: 330461, 642711, 746763, 740301.
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1811537
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Nico Weber <thakis@chromium.org>
These are mostly about rolling in Python 3 encoding fixes.
Bug: None
Test: `repo sync` works w/Python 3
Change-Id: I7d749a3a62523a7c802fb0ed7dcd383868df1d6f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809420
Commit-Queue: Mike Frysinger <vapier@chromium.org>
Reviewed-by: Alex Klein <saklein@chromium.org>
Don't call ensure_bootstrap from update_depot_tools.
ensure_bootstrap also updates gsutil and all versions of pylint
which is slow, particularly on MinGW.
Original change's description:
> depot_tools: Bootstrap Python 3 on Linux/Mac
>
> This will make it possible for developers to execute depot_tools
> scripts using Python 3 in a known environment.
>
> Bug: 1002153
> Change-Id: I5ff492a49d227c1b5876f49adba020f51a575bdd
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1762664
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Bug: 1002153
Change-Id: Ia7579e440438897ba4a7c65a8b228dcfe7f28c86
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1810040
Auto-Submit: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
This should be faster and more general than running
`git index --refresh` over all git repositories we can find.
See "Proposal: Setting core.trustctime=false on Chromium's bots"
on infra-dev (linked to from bug 330461 comment 45).
Bug: 330461,642711,746763,740301.
Change-Id: I7c35cffa107d46d9cf98f0e3e813ef2ab98df5fc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1809419
Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
Commit-Queue: Nico Weber <thakis@chromium.org>
This reverts commits 0e85f633c7 and 9d25ad4192.
Reason for revert:
Causes problems for MinGW users.
Original change's description:
> Reland "depot_tools: Bootstrap Python 3 on Linux/Mac"
>
> This is a reland of dbca865e55
>
> Original change's description:
> > depot_tools: Bootstrap Python 3 on Linux/Mac
> >
> > This will make it possible for developers to execute depot_tools
> > scripts using Python 3 in a known environment.
> >
> > Bug: 1002153
> > Change-Id: I5ff492a49d227c1b5876f49adba020f51a575bdd
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1762664
> > Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
> > Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> > Reviewed-by: Andrii Shyshkalov <tandrii@google.com>
>
> Bug: 1002153
> Change-Id: I6938a76ffa05366c21f7c56aed3d07ecb2b84443
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1798845
> Reviewed-by: Dirk Pranke <dpranke@chromium.org>
> Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
TBR=iannucci@chromium.org,dpranke@chromium.org,tandrii@google.com,ehmaldonado@chromium.org
# Not skipping CQ checks because original CL landed > 1 day ago.
Bug: 1002153
Change-Id: I3ee87b95b2532ec694d7f4f87cce8cde8ccf86eb
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1808137
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Robert Iannucci
Edward Lemur
Mirko Bonadei
recipe-roller
Quinten Yearsley
Victor Costan
Robert Liao
Anthony Polito
Andrii Shyshkalov
Ryan Harrison
Andrii Shyshkalov
Raul Tambre
Nico Weber
Mike Frysinger