Commit Graph

16 Commits (d03e4f3019f82d34d92dbd8fa4c295ec830b65dd)

Author SHA1 Message Date
Struan Shrimpton dc4e911623 Revert "Telemetry: disable for non-linux"
This reverts commit dcbf502f02.

Reason for revert: Fixes have landed

Bug: 326277821
Original change's description:
> Telemetry: disable for non-linux
>
> Something about collecting traces before explicit opt-in is causing
> the output on win to get consumed. Disable for non-linux until I can
> more thoroughly test and figure out what's going on
>
> Bug: 326277821
> Change-Id: I8536cb61078ef98e37c5ab3e1c8abe2141a50eef
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6648247
> Auto-Submit: Struan Shrimpton <sshrimp@google.com>
> Commit-Queue: Struan Shrimpton <sshrimp@google.com>
> Reviewed-by: Ben Pastene <bpastene@chromium.org>

Bug: 326277821
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: I0d122895a832130fef5eebaf4cd4b5896dba4806
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6648626
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
3 days ago
Struan Shrimpton 712066553a Telemetry: use print instead of logging
Using the logging library before UTR can set it up puts it in a weird
state that consumes all output. Instead of using logging, just print
anything from initializing telemetry.

Also make some small updated to avoid features not supported by
non-linux os's right now (these have been generating unnecessary
warnings)

Bug: 425298118
Change-Id: I1ccc749fb3c3410b105116ff25160fc23f0336df
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6648785
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
1 week ago
Struan Shrimpton 2ac2988f7b Telemetry: add bot opt-in support
This adds a new option to bypass the googler check for bots which
might have unexpected hostnames

Bug: 326277821
Change-Id: I56aa2e60d9a7a91ba0b8c8202e659d5b35076c52
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6622310
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
2 weeks ago
Struan Shrimpton dcbf502f02 Telemetry: disable for non-linux
Something about collecting traces before explicit opt-in is causing
the output on win to get consumed. Disable for non-linux until I can
more thoroughly test and figure out what's going on

Bug: 326277821
Change-Id: I8536cb61078ef98e37c5ab3e1c8abe2141a50eef
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6648247
Auto-Submit: Struan Shrimpton <sshrimp@google.com>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
2 weeks ago
Struan Shrimpton 46ad34abca Reland "telemetry: Collect until opt-out"
This is a reland of commit 9c11bcef1f

Original change's description:
> telemetry: Collect until opt-out
>
> The pdd says we will collect by default:
> https://eldar.corp.google.com/assessments/570486509/revisions/1/sections/550004#questions/550404/revisions/2
>
> Also reduce the notice count to 9 since it notices on "0 remaining"
>
> Bug: 326277821
> Change-Id: I4de584c36868b45cefaa5cea42f88d245485dce7
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6616983
> Reviewed-by: Ben Pastene <bpastene@chromium.org>
> Commit-Queue: Struan Shrimpton <sshrimp@google.com>

Bug: 326277821
Change-Id: Ieee21e20ede1dfa71e6c7944ba0e92e74b5a5ded
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6624994
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
3 weeks ago
Struan Shrimpton 4e7e235dc8 Revert "telemetry: Collect until opt-out"
This reverts commit 9c11bcef1f.

Reason for revert: Last patch introduced a bug:
https://ci.chromium.org/ui/p/chromium/builders/try.shadow/mac-utr-tester/6/infra

Bug: 326277821
Original change's description:
> telemetry: Collect until opt-out
>
> The pdd says we will collect by default:
> https://eldar.corp.google.com/assessments/570486509/revisions/1/sections/550004#questions/550404/revisions/2
>
> Also reduce the notice count to 9 since it notices on "0 remaining"
>
> Bug: 326277821
> Change-Id: I4de584c36868b45cefaa5cea42f88d245485dce7
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6616983
> Reviewed-by: Ben Pastene <bpastene@chromium.org>
> Commit-Queue: Struan Shrimpton <sshrimp@google.com>

Bug: 326277821
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Change-Id: Iffaf2a8318cd93c3dd1014080c57de92cfdc4eff
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6622519
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Auto-Submit: Struan Shrimpton <sshrimp@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
3 weeks ago
Struan Shrimpton 929a231882 Telemetry: add win support
The .vpython3 change is only necessary for running the opt in/out on
windows. Otherwise the only oddity was temporary files being weird
in windows.

Bug: 326277821
Change-Id: I88326410526d6fe7f6feb14a0bc27ab22eb12b58
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6615021
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
3 weeks ago
Struan Shrimpton 9c11bcef1f telemetry: Collect until opt-out
The pdd says we will collect by default:
https://eldar.corp.google.com/assessments/570486509/revisions/1/sections/550004#questions/550404/revisions/2

Also reduce the notice count to 9 since it notices on "0 remaining"

Bug: 326277821
Change-Id: I4de584c36868b45cefaa5cea42f88d245485dce7
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6616983
Reviewed-by: Ben Pastene <bpastene@chromium.org>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
3 weeks ago
Struan Shrimpton 3be09c6d74 telemetry: Fix imports
Logging was missing from __init__.py which broke under mac. This adds
that back in and remove unused imports from other files. The os check
is also moved up to be the first thing we look for since is_google_host
does not work with mac at the moment.

Bug:326277821
Change-Id: I3f28838d367eb9583332fd33ad35d71c0851fe4b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/6604498
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
Reviewed-by: Ben Pastene <bpastene@chromium.org>
4 weeks ago
Struan Shrimpton d495580f42 Add telemetry initialization and opt out utility
This includes initializing the module which will handle printing the
notice and enabling/disabling telemetry based on config. The __main__
file also allows for the banner to have a consistent entry point for
disabling telemetry so instrumenting programs don't need to worry
about it if they are using the same default config file.

Bug: 326277821
Change-Id: I861d6b9311ed48c2e1effcac22b314b73e2641e3
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5901465
Reviewed-by: Terrence Reilly <treilly@google.com>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
9 months ago
Struan Shrimpton b5210fdb74 Add Tracer to the telemetry lib
This wraps the tracer and allows us to capture keyboard interrupts.
This is based on cros builds chromite_tracer:
https://crrev.com/ff59bf4edd7614a50f27e1cc067e0b6f0af62f1e/lib/telemetry/trace/chromite_tracer.py
except the failed_packages removed (and a custom span is subsequently
not needed)

Bug: 326277821
Change-Id: I0ac60797563c777ee00abac1e5924c45fb59df29
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5896032
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
Reviewed-by: Terrence Reilly <treilly@google.com>
9 months ago
Struan Shrimpton 5bd66c3869 Add exporter to the telemetry lib
This translates the spans into protos and does the actual publishing to
the Clearcut endpoint using cros build's _LOG_SOURCE so it will be
routed to their Sawmill. The service_name on the trace is what will be
used to store our traces in an F1 table accessible by us.

Bug: 326277821
Change-Id: Id1af3b7d5cffa6d9bfb86c34185e4247e85f9619
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5888844
Reviewed-by: Terrence Reilly <treilly@google.com>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
9 months ago
Struan Shrimpton 9903e62e20 Add config to the telemetry lib
This provides the consistent config between runs. This only supports
linux at the moment, a windows support CL will follow up. Note the
current count down is stored to ensure we provide the banner a minimum
of 10 times and a uuid is stored which is collected in lieu of the
actual username

Bug: 326277821
Change-Id: I3817436ea213c755f490914023508a10cf3cef1b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5855459
Reviewed-by: Terrence Reilly <treilly@google.com>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
10 months ago
Struan Shrimpton 55d065cc0c Add anonymization and the protos to the telemetry lib
Add the protos for the traces used to publish metrics to Clearcut and
the anonymization code/test that ensure we don't collect de-anonymized
paths.

Bug: 326277821
Change-Id: Ifae4d51f59db2219995a0a8d21785729f5eeb137
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5850298
Reviewed-by: Terrence Reilly <treilly@google.com>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
10 months ago
Struan Shrimpton c36eb432d9 Add the detector to telemetry library
The detector can be a pretty direct port from chromite. A lot of these
metrics aren't likely neccesary but will be good to have complete
trace protos rather than missings parts. This also creates the
presubmit and .vpython3 file necessary to run the unit tests

Bug: 326277821
Change-Id: I4dbeabbbced4715527201eca888948b07b6004ca
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5840599
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
Reviewed-by: Terrence Reilly <treilly@google.com>
10 months ago
Struan Shrimpton e416566a1e Establish a directory for chrome infra telemetry
See go/chrome-infra-telemetry-dd for more details. This CL is to
establish a folder where we can create infra libraries shared by
depot_tools and chromium/src. Note that the folder cannot be named
lib due to https://crbug.com/40259280. "libs" should avoid that
problem

Bug: 326277821
Change-Id: I71641ee74d15474353ef0210579a3a24d2726270
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5797754
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Commit-Queue: Struan Shrimpton <sshrimp@google.com>
10 months ago