Commit Graph

13 Commits (f78dd18ee16651fb34e4cfccfa266e151ba6adb8)

Author SHA1 Message Date
Takuto Ikuta 8a2e6a7ae0 ninjalog_uploader_wrapper: do not use binary read/write for json
This is for python3 compatibility.

Bug: b/177465438
Change-Id: I4770136ea702bcaf7a8b644ef842db2f1a7d1200
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2717272
Reviewed-by: Dirk Pranke <dpranke@google.com>
Reviewed-by: Chris Hall <chrishall@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
4 years ago
Takuto Ikuta 84e43fa2c4 ninjalog uploader: migrate to python3
This removes dependency to httplib2 by using urllib.
https://docs.python.org/3/library/http.client.html#httpresponse-objects

Use six for both py2/py3 unittest.

Bug: b/177465438
Change-Id: I48383559842615f97abea45b55ca0acf4d7c8bd3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2629087
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
4 years ago
Takuto Ikuta c603339365 autoninja: use pure python for ninjalog_uploader_wrapper.py
I'll update to python3 separately.

Bug: b/177465438
Change-Id: Id204789f2d0854a796e5408597700d90dc8cd191
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2626863
Reviewed-by: Fumitoshi Ukai <ukai@google.com>
Reviewed-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
4 years ago
Takuto Ikuta 59622a5b31 use CREATE_NEW_PROCESS_GROUP to invoke ninjalog_uploader
This is necessary to make ninjalog_uploader_wrapper.py exit without
waitinng finish of ninjalog_uploader.py in windows.

Bug: 992282
Change-Id: I69d11074bcbe5d0958468a9c8ec8391b0741a839
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2237269
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Yoshisato Yanagisawa <yyanagisawa@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Takuto Ikuta a2e91dbb61 apply yapf for ninjalog*.py
This is preparation to fix the issue.

Bug: 992282
Change-Id: I64b237670bf460ff765df3eaf069b0e0bdcfb31d
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/2237268
Commit-Queue: Ye Kuang <yekuang@google.com>
Reviewed-by: Ye Kuang <yekuang@google.com>
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
5 years ago
Edward Lemur 59a3b2fd5d Run autoninja scripts using vpython.
Bug: 1002153
Change-Id: I769a5370d726f4ce478035f59bfd1df50fef452e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1992315
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
5 years ago
Raul Tambre 80ee78e7fa Convert print statements to Python 3 style
Ran "2to3 -w -n -f print ./" and manually added imports.
Ran "^\s*print " and "\s+print " to find batch/shell scripts, comments and the like with embedded code, and updated them manually.
Also manually added imports to files, which used print as a function, but were missing the import.

The scripts still work with Python 2.
There are no intended behaviour changes.

Bug: 942522
Change-Id: Id777e4d4df4adcdfdab1b18bde89f235ef491b9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1595684
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
6 years ago
Takuto Ikuta 65ace12763 do not decrements countdown when that is 0
Change-Id: If3eee28eae24bb9fd45e65b426cd7bd7c3b26c2f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1534210
Auto-Submit: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Simon Que <sque@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
6 years ago
Bruce Dawson cf9613f166 Avoiding writing ninjalog.cfg unnecessarily
Writing to ninjalog.cfg comes with the risk of failure (Ctrl+C at the
wrong time) which can lead to an empty or corrupted file. This will
cause errors on subsequent builds when loading the config file, and
will lose the users opt-in/out settings.

This change ensures that the config file is only written when something
has changed, thus essentially avoiding the risk.

Bug: 941810
Change-Id: If266f07536569c9ac91ee37c28797c8b2ccc6fdc
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1522291
Reviewed-by: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
6 years ago
Takuto Ikuta 5c00a22f22 [ninjalog_uploader] add path for httplib2
This is to let httplib2 import its sock.py in __init__.py correctly.
This is pointed out in
https://groups.google.com/a/chromium.org/d/msg/chromium-dev/8BF_cywS9JY/8g-HHmuqEQAJ

Also removed an unnecessary httplib2 import.

Change-Id: Iad0377ad282acd9e1e222890f522e4e62086dcaf
Reviewed-on: https://chromium-review.googlesource.com/c/1475255
Reviewed-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
6 years ago
Takuto Ikuta c8069af594 [ninjalog] fully anonymize build stats before sending
We cannot send PII data.

This CL tried to remove following data,
* build config that can contain build directory path
* build directory path
* hostname
* cwd

I use per build uuid instead of per user uuid.


Bug: 900161
Change-Id: Id533762749806ad8616b7dc07f21b27dfe632c9a
Reviewed-on: https://chromium-review.googlesource.com/c/1369473
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
Reviewed-by: Fumitoshi Ukai <ukai@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
6 years ago
Takuto Ikuta 540ba9dde0 [ninjalog] add ninjalog.README.md
I added document explaining ninjalog collection.
This document explains what type of data is collected and how the data will be used.

Bug: 900161, b/120645342
Change-Id: If47517aaa7a5510e4a634b5ccc0e4127d014dcb9
Reviewed-on: https://chromium-review.googlesource.com/c/1381912
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
6 years ago
Takuto Ikuta 9af233a508 introduce ninjalog_uploader to autoninja
This CL introduces ninjalog_uploader to autoninja for posix environment.
This is to collect build statistics from googler to find user side build performance bottleneck.

ninjalog_uploader_wrapper is used to notify user and manage small config of upload script.

Bug: 900161
Change-Id: I48ac8cd1d52f64e8fdafaec43636d2d79ef9040d
Reviewed-on: https://chromium-review.googlesource.com/c/1345255
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Shinya Kawanaka <shinyak@chromium.org>
Commit-Queue: Takuto Ikuta <tikuta@chromium.org>
6 years ago