This way the function is much more similar to subprocess.call(). Further changes
will be done, to be able to convert all the function, to be able to parallelize
output without corrupting it.
Used pylint to verify call graph correctness, causing some other unrelated
changes.
TEST=unit tests
BUG=none
Review URL: http://codereview.chromium.org/3117039
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@57369 0039d316-1c4b-4281-b951-d872f2087c98
It will eventually replace 'print_messages' and be used to annotate printed
lines when using --jobs.
Review URL: http://codereview.chromium.org/3174020
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@57092 0039d316-1c4b-4281-b951-d872f2087c98
Original Description:
Remove code duplication and improve style.
Create a Popen function to reduce code duplication.
Use RemoveDirectory where relevant.
Make drover slightly more posix friendly.
Review URL: http://codereview.chromium.org/3199011
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@57083 0039d316-1c4b-4281-b951-d872f2087c98
Create a Popen function to reduce code duplication.
Use RemoveDirectory where relevant.
Make drover slightly more posix friendly.
Review URL: http://codereview.chromium.org/3126020
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@56893 0039d316-1c4b-4281-b951-d872f2087c98
Speed up flushing of output on platforms that may not
flush as often as we expect (like Windows). Wait at
least 10s between flushes to avoid overloading busy
readers.
BUG=none
TEST=gclient flushes on Windows more often, regular
gclient operations continue to work as expected
Review URL: http://codereview.chromium.org/3140013
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@56217 0039d316-1c4b-4281-b951-d872f2087c98
Needed to use the special 3 args raise notation.
Review URL: http://codereview.chromium.org/3164014
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@56209 0039d316-1c4b-4281-b951-d872f2087c98
Huge thanks to piman@ for working on a patch. I chose a different design but he
gave me motivation and ideas. Sorry for not accepting his patch earlier, this
was mostly due to broken gclient implementation itself.
gclient can now run an unlimited number of parallel checkouts and always keep
the checkout coherency correct.
--jobs=1 is single threaded as before, albeit with a different code path.
Issues:
- Using --jobs with a value other than 1 will result in a mangled output.
- Exceptions thrown in a thread will be have the wrong stack trace.
TEST=gclient sync -j 99 in a ssh:// chromiumos checkout is dramatically faster.
---
Here's the perf on linux on i7-860 for a chromium checkout with warm cache. Cold
cache will result is significantly reduced improvements so this is best case
improvements. The sync was no-op all the time except where noted. All execution
where with "time gclient sync " + args. Didn't include 'sys' column since it was
statistically insignifiant and highly correlated with 'user'.
runs with -f runs with -m without -f nor -m
args real user real user real user
-j 12 20.59s 18.00s 5.64s 7.95s 5.86s 8.10s
#1 1m05.26s 20.02s 5.20s 7.94s 5.10s 8.09s
22.79s 18.17s
-j 1 #2 1m47.00s 16.72s 9.69s 5.72s 12.35s 5.96s
1m31.28s 17.06s 9.54s 5.85s 10.51s 6.20s
1m31.79s 16.39s
before #3 1m30.94s 16.74s 9.77s 5.83s 10.45s 5.77s
1m30.17s 17.30s 10.36s 5.68s 10.16s 5.88s
hook #4 8.52s 7.93s
8.73s 8.13s
#1 This particular run synched to r56023, a webkit roll updating layout tests.
It's still faster than a no-op sync without parallel checkout.
#2 Maybe there was a sync or computer hickup, I didn't realize.
#3 This is depot_tools@56020
#4 Since -f implies runhooks, I ran the hook 'python src/build/gyp_chromium'
manually to compare. Hooks are still run in a single thread. I didn't rest
'gclient runhooks'.
I tried to go a ssh:// checkout of chromium os tree but it timed out everytime I
tried to sync so I couldn't get data points. I expect an order of magnitude of
improvement or more.
Review URL: http://codereview.chromium.org/3135014
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@56079 0039d316-1c4b-4281-b951-d872f2087c98
Bring some OOP and sanity to gclient.py.
- Changed the algorithm from breadth-first to depth-first.
- Added infinite recursion support.
- Fixed From() of From() dependency.
- Fixed cross solution custom deps aliasing.
- Removed support code for old .gclient_entries format.
- Removed IsGitCheckout() in favor for a direct check for .git presence.
TEST=all test pass
The main difference with r51760 is that new smoke tests have been added since which replicates how webkit checkouts with gclient.
Review URL: http://codereview.chromium.org/2917009
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@53205 0039d316-1c4b-4281-b951-d872f2087c98
Add the fix for FileImpl() plus unit tests.
TEST=more tests
Review URL: http://codereview.chromium.org/2808048
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@52005 0039d316-1c4b-4281-b951-d872f2087c98
- Changed the algorithm from breadth-first to depth-first.
- Added infinite recursion support.
- Fixed From() of From() dependency.
- Fixed cross solution custom deps aliasing.
- Removed support code for old .gclient_entries format.
- Removed IsGitCheckout() in favor for a direct check for .git presence.
TEST=all test pass
Review URL: http://codereview.chromium.org/2867047
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@51760 0039d316-1c4b-4281-b951-d872f2087c98
TEST=none, no big deal. I'll get a breakpad report otherwise anyway. This is solely to *reduce* the number of breakpad stack trace that this is done.
Review URL: http://codereview.chromium.org/2885020
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@51739 0039d316-1c4b-4281-b951-d872f2087c98
Fix gclient diff & pack to not throw uncatched exceptions when a directory is missing.
Reorder things in gclient.py for easier diff later
Update revinfo help
Add GetScmName(), it will be needed later.
TEST=unit tests
BUG=none
Review URL: http://codereview.chromium.org/2786013
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@49565 0039d316-1c4b-4281-b951-d872f2087c98
Switch "" usage to '' to please Brad. Didn't change gclient.py yet because it
will conflict with the refactor.
TEST=unit tests still pass.
Review URL: http://codereview.chromium.org/2769011
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@49545 0039d316-1c4b-4281-b951-d872f2087c98
On a chromeos checkout, -j 10 brings down null sync time from 2 minutes to 16 seconds
Currently -j may break some assumptions about ordering of commands (which may be fine for some clients, but not all), so it's not on by default.
Review URL: http://codereview.chromium.org/1640001
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@44869 0039d316-1c4b-4281-b951-d872f2087c98
This tool displays all checkins by one (or several) authors in the past week (or configurable time), in all dependencies. Only works on git dependencies currently.
Review URL: http://codereview.chromium.org/1602020
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@44612 0039d316-1c4b-4281-b951-d872f2087c98
We now return the stderr half of the tuple.
This required a clean up of any usage of CheckCall and GIT.Capture.
Patch contributed by Nasser Grainawi <nasser@codeaurora.org>
Review URL: http://codereview.chromium.org/551215
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@37650 0039d316-1c4b-4281-b951-d872f2087c98
Add real logging support.
Fix the patch path mungling that was broken earlier.
Add and proper refactor automatic gclient and gcl settings detection.
Factored so it is possible to add other autodetection algorithms.
Now works standalone in hybrid svn&git checkouts within a gclient meta-checkout.
TEST=bah
BUG=none
TBR=bradnelson
Review URL: http://codereview.chromium.org/517005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@35218 0039d316-1c4b-4281-b951-d872f2087c98
Make lack of email address an error.
TEST=none
BUG=none
Review URL: http://codereview.chromium.org/507072
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@35167 0039d316-1c4b-4281-b951-d872f2087c98
Remove upload dependency.
This is towards making trychange.py work mostly standalone for the webkit try server.
TEST=unit tests and tested both gcl try and git-try.
Review URL: http://codereview.chromium.org/501143
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@35071 0039d316-1c4b-4281-b951-d872f2087c98
Doing something like basename when concatenating base_url with
relative url.
Added use_relative_urls2 to enable switching between old
implementation and new implementation. Once the old implementation
is deprecated we can remove the use_relative_urls2.
Review URL: http://codereview.chromium.org/500015
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@34563 0039d316-1c4b-4281-b951-d872f2087c98
TEST=none
BUG=none
Move scm functions into a class to make it simpler to manage.
Review URL: http://codereview.chromium.org/391052
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@32057 0039d316-1c4b-4281-b951-d872f2087c98
ssh://test@example.com wasn't being parsed correctly for revinfo.
Fixed by factoring out all code which splits url into a base_url
and revision.
Review URL: http://codereview.chromium.org/391048
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@31910 0039d316-1c4b-4281-b951-d872f2087c98
If you're not running gclient sync with --verbose and you get a conflict on
a file, the conflict message prints *before* the message that you're running
update in the appropriate directory. This is happening because we intercept
stdout but not stderr, and the conflict prompt goes to stderr. redirecting
stderr in the popen() to the pipe fixes this.
R=maruel@chromium.org
BUG=none
TEST=none
Review URL: http://codereview.chromium.org/373003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@31276 0039d316-1c4b-4281-b951-d872f2087c98
print messages unless the underlying commands actually generate output.
This is a more conservative version of r28271.
BUG=none
TEST=none
R=maruel
Review URL: http://codereview.chromium.org/344004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@30296 0039d316-1c4b-4281-b951-d872f2087c98
This reverts commit 28058.
BUG=24081
TEST=gclient sync without --force is not working anymore.
TBR=dpranke
Review URL: http://codereview.chromium.org/269006
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@28271 0039d316-1c4b-4281-b951-d872f2087c98
Its side-effect caused tree breakages, at least on the try slaves.
TBR=dpranke
BUG=none
TEST=none
Review URL: http://codereview.chromium.org/251095
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@28058 0039d316-1c4b-4281-b951-d872f2087c98
messages when --verbose is specified or when something actually happens.
Also, remove --manually_grab_svn_rev, because we need this to happen all
the time to get gclient update to shut up if we're comparing against HEAD.
It doesn't look like there are any real side effects of removing this.
An alternative change would be to wrap this behind a --quiet flag.
Note that the change to tests/trychange_unittest.py is an unrelated fix to get
all the tests to pass.
R=maruel@chromium.org
TEST=none
BUG=none
Review URL: http://codereview.chromium.org/257023
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@28039 0039d316-1c4b-4281-b951-d872f2087c98
maruel@chromium.org
chase@chromium.org
tony@chromium.org
piman@chromium.org
msb@chromium.org
dpranke@google.com