aiden
/
depot_tools
mirror of https://chromium.googlesource.com/chromium/tools/depot_tools
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
10,933 Commits
6 Branches
0 Tags
185 MiB
main
chrome/4147
chrome/3987
chrome/3865
chrome/3904
infra/config
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'd43a3eb748'
${ noResults }
Commit Graph

12 Commits (d43a3eb748e40838c41e7a3ca53a3c6cbee0ea7a)

Author SHA1 Message Date
Stephanie Kim 188b08d7bf Make the downloaded tarfile a dotfile 
This is so that the third_party directories can gitignore the
downloaded tarfile. Currently theyre ignoring on *tar.gz

Bug: b/324418194
Change-Id: Id4aeeec366de3a0cec410de6ddd63369a3a4680a
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5454165
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
 1 year ago
Stephanie Kim 3a30482282 Update *_hash and *_is_first_class_gcs to be dotfiles 
Bug: b/324418194
Change-Id: Iae22d6895c8a15d7be351ce0023bd2ada83292a5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5441097
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
 1 year ago
Stephanie Kim 2c9e581fad Allow multiple objects per GCS dep 
Allows multiple objects to use the same directory path.
This requires that each object has its own unique hash file and
unique migration file name. All object names must be unique.

Also update download_from_google_storage to check for the unique
migration file name.

Remove ConvertToGcs smoke tests since we're not converting
any git <-> gcs deps.

Example:
```
'src/third_party/llvm-build/Release+Asserts': {
  'dep_type': 'gcs',
  'condition': 'not llvm_force_head_revision',
  'bucket': 'chromium-browser-clang',
  'objects': [
      {
          'object_name': 'Linux_x64/clang-llvmorg-19-init-2941-ga0b3dbaf-22.tar.xz',
          'sha256sum': '7b33138d8592199f97d132242d7b3e10f460c5c9655d49a3ad3767218fba7a77',
          'size_bytes': 50212876,
      },
      {
         'object_name': 'Linux_x64/llvmobjdump-llvmorg-19-init-2941-ga0b3dbaf-22.tar.xz',
         'sha256sum': '14d669650cd212eb0ccb8c34a9e655338dfdee66fe2ecdaa517f6bd607c09a97',
         'size_bytes': 5302312,
       },
    ]
 },
```

TODO: update .gitignore to search for *_is_first_class and *_hash

Bug: b/324418194
Change-Id: I89d34b06ee24f4c1aa316cd51530ad078e823143
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5420793
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
 1 year ago
Stephanie Kim 16e7252be4 Add is_first_class_gcs migration file 
If is_first_class_gcs file exists with contents '1'
- GCS hooks will download content and then overwrite the file with '0'
- First class GCS deps will download as necessary by checking the hash
file.

If there is no is_first_class_gcs file or its content is '0' for a GCS hook:
- Hook will run normally and, if content is downloaded, an is_first_class_gcs
file will be written with '0'

If there is no is_first_class_gcs file or its content is '0' for a first
class gcs dep:
- Content will be downloaded and is_first_class_gcs will be set to '1'

Verified locally with migrating and unmigrating a GCS hook.
TODO: update other GCS hook scripts in chromium/src to check migration
file.

Bug: b/324418194
Change-Id: Ida8541cb70839b86e99115bcabfc38bc87e40136
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5368250
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
 1 year ago
Stephanie Kim 53870d4056 Optional output_file field in GCS dep 
When an 'output_file' arg is included, the downloaded object from GCS
will be downloaded in that path.

Example:
```
'src/buildtools/linux': {
  'bucket': 'bucket123',
  'object_name': 'clang-format-version123',
  'dep_type': 'gcs',
  'sha256sum': 'abcd123',
  'output_file': 'clang-format',
}
```
The GCS object will be downloaded at src/buildtools/linux/clang-format

Bug: b/324418194
Change-Id: I1049abeb09a1027c5477d955e50611d43015d0a4
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5353387
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
 1 year ago
Stephanie Kim 63ae5028c1 Reland "Reland "Add support for GCS deps"" 
This is a reland of commit 46d5382f69

Whats fixed: Removed the call_google_storage refactor so
that all `import download_from_google_storage` callsites
(in depot_tools and chromium/src) won't be affected.
Instead, gclient.py will import Gsutil from download_from_google_storage.

Original change's description:
> Reland "Add support for GCS deps"
>
> This is a reland of commit 3eedee7b55
>
> Fixed more imports in download_google_storage.py
>
> Original change's description:
> > Add support for GCS deps
> >
> > Also take out GCS calling logic from download_google_storage and
> > into call_google_storage.
> >
> > GCS deps look like:
> >    'src/third_party/node/linux': {
> >        'dep_type': 'gcs',
> >        'condition': 'checkout_linux',
> >        'bucket': 'chromium-nodejs/20.11.0',
> >        'object_name': '46795170ff5df9831955f163f6966abde581c8af',
> >        'sha256sum': '887504c37404898ca41b896f448ee6d7fc24179d8fb6a4b79d028ab7e1b7153d',
> >    },
> >
> >    'src/third_party/llvm-build/Release+Asserts': {
> >        'dep_type': 'gcs',
> >        'condition': 'checkout_linux',
> >        'bucket': 'chromium-browser-clang',
> >        'object_name': 'Linux_x64/clang-llvmorg-18-init-17730-gf670112a-2.tar.xz',
> >        'sha256sum': '1e46df9b4e63c074064d75646310cb76be2f19815997a8486987189d80f991e8',
> >    },
> >
> > Example directory for src/third_party/node/linux after gclient sync:
> > - tar_file.gz is the downloaded file from GCS.
> > - node_linux_x64/ is extracted in its path.
> > - `hash` contains the sha of GCS filename.
> > ```
> > chromium/src/ ->
> >    third_party/node/linux/ ->
> >        hash, tar_file.gz, node_linux_x64/
> > ```
> >
> > Bug: b/324418194
> > Change-Id: Ibcbbff27e211f194ddb8a08494af56570a84a12b
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5299722
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Joanna Wang <jojwang@chromium.org>
>
> Bug: b/324418194
> Change-Id: Ie64265a86abcec0135408715a45c32a8bb7c7408
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5346338
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

Bug: b/324418194
Change-Id: I8b58dadbaa740fd9da1fbaf29b3b6ff5ef67fd12
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5352896
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
 1 year ago
Michael Ershov 3c79cfc485 Revert "Reland "Add support for GCS deps"" 
This reverts commit 46d5382f69.

Reason for revert: win-presubmit builder probably started failing after this CL.
https://ci.chromium.org/ui/p/chromium/builders/ci/win-presubmit

Original change's description:
> Reland "Add support for GCS deps"
>
> This is a reland of commit 3eedee7b55
>
> Fixed more imports in download_google_storage.py
>
> Original change's description:
> > Add support for GCS deps
> >
> > Also take out GCS calling logic from download_google_storage and
> > into call_google_storage.
> >
> > GCS deps look like:
> >    'src/third_party/node/linux': {
> >        'dep_type': 'gcs',
> >        'condition': 'checkout_linux',
> >        'bucket': 'chromium-nodejs/20.11.0',
> >        'object_name': '46795170ff5df9831955f163f6966abde581c8af',
> >        'sha256sum': '887504c37404898ca41b896f448ee6d7fc24179d8fb6a4b79d028ab7e1b7153d',
> >    },
> >
> >    'src/third_party/llvm-build/Release+Asserts': {
> >        'dep_type': 'gcs',
> >        'condition': 'checkout_linux',
> >        'bucket': 'chromium-browser-clang',
> >        'object_name': 'Linux_x64/clang-llvmorg-18-init-17730-gf670112a-2.tar.xz',
> >        'sha256sum': '1e46df9b4e63c074064d75646310cb76be2f19815997a8486987189d80f991e8',
> >    },
> >
> > Example directory for src/third_party/node/linux after gclient sync:
> > - tar_file.gz is the downloaded file from GCS.
> > - node_linux_x64/ is extracted in its path.
> > - `hash` contains the sha of GCS filename.
> > ```
> > chromium/src/ ->
> >    third_party/node/linux/ ->
> >        hash, tar_file.gz, node_linux_x64/
> > ```
> >
> > Bug: b/324418194
> > Change-Id: Ibcbbff27e211f194ddb8a08494af56570a84a12b
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5299722
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Joanna Wang <jojwang@chromium.org>
>
> Bug: b/324418194
> Change-Id: Ie64265a86abcec0135408715a45c32a8bb7c7408
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5346338
> Reviewed-by: Joanna Wang <jojwang@chromium.org>
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>

Bug: b/324418194
Change-Id: Ic4517f6c9e05aea3f3f052d2a44865733236998b
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5352167
Auto-Submit: Michael Ershov <miersh@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Commit-Queue: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Michael Ershov <miersh@google.com>
 1 year ago
Stephanie Kim 46d5382f69 Reland "Add support for GCS deps" 
This is a reland of commit 3eedee7b55

Fixed more imports in download_google_storage.py

Original change's description:
> Add support for GCS deps
>
> Also take out GCS calling logic from download_google_storage and
> into call_google_storage.
>
> GCS deps look like:
>    'src/third_party/node/linux': {
>        'dep_type': 'gcs',
>        'condition': 'checkout_linux',
>        'bucket': 'chromium-nodejs/20.11.0',
>        'object_name': '46795170ff5df9831955f163f6966abde581c8af',
>        'sha256sum': '887504c37404898ca41b896f448ee6d7fc24179d8fb6a4b79d028ab7e1b7153d',
>    },
>
>    'src/third_party/llvm-build/Release+Asserts': {
>        'dep_type': 'gcs',
>        'condition': 'checkout_linux',
>        'bucket': 'chromium-browser-clang',
>        'object_name': 'Linux_x64/clang-llvmorg-18-init-17730-gf670112a-2.tar.xz',
>        'sha256sum': '1e46df9b4e63c074064d75646310cb76be2f19815997a8486987189d80f991e8',
>    },
>
> Example directory for src/third_party/node/linux after gclient sync:
> - tar_file.gz is the downloaded file from GCS.
> - node_linux_x64/ is extracted in its path.
> - `hash` contains the sha of GCS filename.
> ```
> chromium/src/ ->
>    third_party/node/linux/ ->
>        hash, tar_file.gz, node_linux_x64/
> ```
>
> Bug: b/324418194
> Change-Id: Ibcbbff27e211f194ddb8a08494af56570a84a12b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5299722
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Joanna Wang <jojwang@chromium.org>

Bug: b/324418194
Change-Id: Ie64265a86abcec0135408715a45c32a8bb7c7408
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5346338
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
 1 year ago
Stephanie Kim 93d5662c1d Revert "Reland "Add support for GCS deps"" 
This reverts commit d94a03f21e.

Reason for revert: PLATFORM_MAPPING import issue

Original change's description:
> Reland "Add support for GCS deps"
>
> This is a reland of commit 3eedee7b55
>
> Original change's description:
> > Add support for GCS deps
> >
> > Also take out GCS calling logic from download_google_storage and
> > into call_google_storage.
> >
> > GCS deps look like:
> >    'src/third_party/node/linux': {
> >        'dep_type': 'gcs',
> >        'condition': 'checkout_linux',
> >        'bucket': 'chromium-nodejs/20.11.0',
> >        'object_name': '46795170ff5df9831955f163f6966abde581c8af',
> >        'sha256sum': '887504c37404898ca41b896f448ee6d7fc24179d8fb6a4b79d028ab7e1b7153d',
> >    },
> >
> >    'src/third_party/llvm-build/Release+Asserts': {
> >        'dep_type': 'gcs',
> >        'condition': 'checkout_linux',
> >        'bucket': 'chromium-browser-clang',
> >        'object_name': 'Linux_x64/clang-llvmorg-18-init-17730-gf670112a-2.tar.xz',
> >        'sha256sum': '1e46df9b4e63c074064d75646310cb76be2f19815997a8486987189d80f991e8',
> >    },
> >
> > Example directory for src/third_party/node/linux after gclient sync:
> > - tar_file.gz is the downloaded file from GCS.
> > - node_linux_x64/ is extracted in its path.
> > - `hash` contains the sha of GCS filename.
> > ```
> > chromium/src/ ->
> >    third_party/node/linux/ ->
> >        hash, tar_file.gz, node_linux_x64/
> > ```
> >
> > Bug: b/324418194
> > Change-Id: Ibcbbff27e211f194ddb8a08494af56570a84a12b
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5299722
> > Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> > Reviewed-by: Joanna Wang <jojwang@chromium.org>
>
> Bug: b/324418194
> Change-Id: I8b960f6dd3534658a9ee0b7688e68eaa15df1444
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5334791
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
> Reviewed-by: Joanna Wang <jojwang@chromium.org>

Bug: b/324418194
Change-Id: I28bc0ba838a1135ff882cd88b77dbb2cb4b85b82
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5345022
Reviewed-by: Joanna Wang <jojwang@chromium.org>
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
 1 year ago
Stephanie Kim d94a03f21e Reland "Add support for GCS deps" 
This is a reland of commit 3eedee7b55

Original change's description:
> Add support for GCS deps
>
> Also take out GCS calling logic from download_google_storage and
> into call_google_storage.
>
> GCS deps look like:
>    'src/third_party/node/linux': {
>        'dep_type': 'gcs',
>        'condition': 'checkout_linux',
>        'bucket': 'chromium-nodejs/20.11.0',
>        'object_name': '46795170ff5df9831955f163f6966abde581c8af',
>        'sha256sum': '887504c37404898ca41b896f448ee6d7fc24179d8fb6a4b79d028ab7e1b7153d',
>    },
>
>    'src/third_party/llvm-build/Release+Asserts': {
>        'dep_type': 'gcs',
>        'condition': 'checkout_linux',
>        'bucket': 'chromium-browser-clang',
>        'object_name': 'Linux_x64/clang-llvmorg-18-init-17730-gf670112a-2.tar.xz',
>        'sha256sum': '1e46df9b4e63c074064d75646310cb76be2f19815997a8486987189d80f991e8',
>    },
>
> Example directory for src/third_party/node/linux after gclient sync:
> - tar_file.gz is the downloaded file from GCS.
> - node_linux_x64/ is extracted in its path.
> - `hash` contains the sha of GCS filename.
> ```
> chromium/src/ ->
>    third_party/node/linux/ ->
>        hash, tar_file.gz, node_linux_x64/
> ```
>
> Bug: b/324418194
> Change-Id: Ibcbbff27e211f194ddb8a08494af56570a84a12b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5299722
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Joanna Wang <jojwang@chromium.org>

Bug: b/324418194
Change-Id: I8b960f6dd3534658a9ee0b7688e68eaa15df1444
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5334791
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
 1 year ago
Stephanie Kim 064e03aa1c Revert "Add support for GCS deps" 
This reverts commit 3eedee7b55.

Reason for revert: https://buganizer.corp.google.com/issues/324418194#comment7

Original change's description:
> Add support for GCS deps
>
> Also take out GCS calling logic from download_google_storage and
> into call_google_storage.
>
> GCS deps look like:
>    'src/third_party/node/linux': {
>        'dep_type': 'gcs',
>        'condition': 'checkout_linux',
>        'bucket': 'chromium-nodejs/20.11.0',
>        'object_name': '46795170ff5df9831955f163f6966abde581c8af',
>        'sha256sum': '887504c37404898ca41b896f448ee6d7fc24179d8fb6a4b79d028ab7e1b7153d',
>    },
>
>    'src/third_party/llvm-build/Release+Asserts': {
>        'dep_type': 'gcs',
>        'condition': 'checkout_linux',
>        'bucket': 'chromium-browser-clang',
>        'object_name': 'Linux_x64/clang-llvmorg-18-init-17730-gf670112a-2.tar.xz',
>        'sha256sum': '1e46df9b4e63c074064d75646310cb76be2f19815997a8486987189d80f991e8',
>    },
>
> Example directory for src/third_party/node/linux after gclient sync:
> - tar_file.gz is the downloaded file from GCS.
> - node_linux_x64/ is extracted in its path.
> - `hash` contains the sha of GCS filename.
> ```
> chromium/src/ ->
>    third_party/node/linux/ ->
>        hash, tar_file.gz, node_linux_x64/
> ```
>
> Bug: b/324418194
> Change-Id: Ibcbbff27e211f194ddb8a08494af56570a84a12b
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5299722
> Commit-Queue: Stephanie Kim <kimstephanie@google.com>
> Reviewed-by: Joanna Wang <jojwang@chromium.org>

Bug: b/324418194
Change-Id: Ie74df90065c1fea087b240d98005b9d9b4f44411
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5336079
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Josip Sokcevic <sokcevic@chromium.org>
 1 year ago
Stephanie Kim 3eedee7b55 Add support for GCS deps 
Also take out GCS calling logic from download_google_storage and
into call_google_storage.

GCS deps look like:
   'src/third_party/node/linux': {
       'dep_type': 'gcs',
       'condition': 'checkout_linux',
       'bucket': 'chromium-nodejs/20.11.0',
       'object_name': '46795170ff5df9831955f163f6966abde581c8af',
       'sha256sum': '887504c37404898ca41b896f448ee6d7fc24179d8fb6a4b79d028ab7e1b7153d',
   },

   'src/third_party/llvm-build/Release+Asserts': {
       'dep_type': 'gcs',
       'condition': 'checkout_linux',
       'bucket': 'chromium-browser-clang',
       'object_name': 'Linux_x64/clang-llvmorg-18-init-17730-gf670112a-2.tar.xz',
       'sha256sum': '1e46df9b4e63c074064d75646310cb76be2f19815997a8486987189d80f991e8',
   },

Example directory for src/third_party/node/linux after gclient sync:
- tar_file.gz is the downloaded file from GCS.
- node_linux_x64/ is extracted in its path.
- `hash` contains the sha of GCS filename.
```
chromium/src/ ->
   third_party/node/linux/ ->
       hash, tar_file.gz, node_linux_x64/
```

Bug: b/324418194
Change-Id: Ibcbbff27e211f194ddb8a08494af56570a84a12b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/5299722
Commit-Queue: Stephanie Kim <kimstephanie@google.com>
Reviewed-by: Joanna Wang <jojwang@chromium.org>
 1 year ago