Ran:
vi $(git grep --name-only iteritems | grep -v third_party)
vi $(git grep --name-only itervalues | grep -v third_party)
vi $(git grep --name-only 'print ' | grep -v third_party)
and edited the files quickly with adhoc macros. Then ran in recipes/:
./recipes.py test train
There was only a small subset of files that had been updated to use
six.iteritems() and six.itervalues(). Since the dataset size that is
being used in gclient is small (pretty much always below 200 items),
it's better to just switch to .items() right away and take the temporary
performance hit, so that we don't need to come back to rewrite the code.
Recipe-Nontrivial-Roll: build
Bug: 984182
Change-Id: I5faf11486b66b0d73c9098ab0f2ce1b15a45c53e
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1854900
Commit-Queue: Edward Lesmes <ehmaldonado@chromium.org>
Reviewed-by: Edward Lesmes <ehmaldonado@chromium.org>
Auto-Submit: Marc-Antoine Ruel <maruel@chromium.org>
This is easy to miss and a toolchain package without win_sdk\Debuggers
will not work as expected so this CL adds a check to stop the
packaging/repacking process as soon as possible.
Bug: 1006238
Change-Id: I272fbaf3e1cbde48a3e260e38e34ffc10d6c86df
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1829272
Commit-Queue: Mirko Bonadei <mbonadei@chromium.org>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Not everyone builds Chromium for arm so then it's practical to be
able to use a smaller SDK package without arm support included.
This also removes some samples that should not be needed.
Recipe-Nontrivial-Roll: chromiumos
Change-Id: I10f64d8907600fe79d3051083d100a7562d3a797
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1752202
Commit-Queue: Daniel Bratell <bratell@opera.com>
Reviewed-by: Bruce Dawson <brucedawson@chromium.org>
Ran "2to3 -w -n -f print ./" and manually added imports.
Ran "^\s*print " and "\s+print " to find batch/shell scripts, comments and the like with embedded code, and updated them manually.
Also manually added imports to files, which used print as a function, but were missing the import.
The scripts still work with Python 2.
There are no intended behaviour changes.
Bug: 942522
Change-Id: Id777e4d4df4adcdfdab1b18bde89f235ef491b9f
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/tools/depot_tools/+/1595684
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Auto-Submit: Raul Tambre <raul@tambre.ee>
Recent versions of the 10.0.17763 SDK have rearranged the redist
directory. This change adjusts the packaging so that we can package the
SDK with and without this change. The script looks first in the
versioned directory and if that doesn't exist then it falls back to the
unversioned directory.
See also http://crrev.com/c/1371017 for the build\vs_toolchain.py
version of this change.
Change-Id: I835bf45f613a0aebb56eb1e8e63e6ffa5252edf0
Reviewed-on: https://chromium-review.googlesource.com/c/1370609
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
These changes ensure that the packaged Windows SDK bits support targeting
ARM64 Windows. This change must not land until crrev.com/c/1339101 lands,
since it is needed to the varying number of vs_runtime_dll_dirs.
Bug: 893460
Change-Id: Ie32563067c6fb6078acfaccd6d3d572d1dd44888
Reviewed-on: https://chromium-review.googlesource.com/c/1330185
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
The script that packages the Visual Studio toolchain had special code to
handle grabbing dbghelp.dll from the VS install and copying it to the
Debuggers directories. This was necessary around VS 2017 RTM to handle
/debug:fastlink but it now causes failures due to mismatched DLLs
leading to a missing ordinal. The hack shouldn't be needed anymore
because we no longer depend on /debug:fastlink (lld!) and because the
Debuggers package is assumed to have been updated to handle VS 2017 by
now.
Bug: 846313
Change-Id: I2b5fd87aaa785ce24a0905d70e7e586ff4838b1c
Reviewed-on: https://chromium-review.googlesource.com/1086990
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
It appears that newer VS 2017 installers install CppUnitFramework in different
way. My version VS 15.6.6. The problem is that normal users do not seem to be
able to download the VS 15.3.2 that Chromium suggests and this script was
probably originally tested with.
I installed:
vs_professional__428566190.xxx.exe
--add Microsoft.VisualStudio.Workload.NativeDesktop
--add Microsoft.VisualStudio.Component.Debugger.JustInTime
--add Microsoft.VisualStudio.Component.Graphics.Tools
--add Microsoft.VisualStudio.Component.Graphics.Win81
--add Microsoft.VisualStudio.Component.NuGet
--add Microsoft.VisualStudio.Component.Static.Analysis.Tools
--add Microsoft.VisualStudio.Component.VC.ATL
--add Microsoft.VisualStudio.Component.VC.DiagnosticTools
--add Microsoft.VisualStudio.Component.VC.TestAdapterForGoogleTest
--add Microsoft.VisualStudio.Component.VC.Tools.x86.x64
--add Microsoft.VisualStudio.Component.Windows10SDK.15063.Desktop
--add Microsoft.VisualStudio.Component.Windows10SDK.15063.UWP
--add Microsoft.VisualStudio.Component.Windows10SDK.15063.UWP.Native
Go to apps -> Windows sdk 15063
Modify -> Change
Select Debugging tools for Windows
Ended up with VS 15.6.6 and Windows SDK 10.0.15063.673
Packaged with:
python %HOME%\depot_tools\win_toolchain\package_from_installed.py -w 10.0.15063.0 2017
Bug: 834656
Change-Id: I5e6bf6a5f94a28d5ba81a3c8cd79e37028b9429b
Reviewed-on: https://chromium-review.googlesource.com/1018470
Reviewed-by: Scott Graham <scottmg@chromium.org>
Commit-Queue: Scott Graham <scottmg@chromium.org>
Sometimes it is necessary to recreate a VS toolchain package with a tiny
change such as editing or removing a single file. Recreating the VS
environment can be time consuming or even impossible (the VS installer
does not allow going to arbitrary versions) so this switch allows
repackaging an existing unpacked directory. The typical usage would be
to make the modifications to one of the toolchain directories in
depot_tools\win_toolchain\vs_files and then repackage it using:
python package_from_installed.py --repackage third_party\depot_tools\win_toolchain\vs_files\9bc7ccbf9f4bd50d4a3bd185e8ca94ff1618de0b
Bug: 772123
Change-Id: I77b928f695e5f07e33f68dd37711c8761a3c7a22
Reviewed-on: https://chromium-review.googlesource.com/713562
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
clang-cl relies on this env var being set for 2017 to find the linker.
Bug: 772123
Change-Id: Id03896504d38dc706b2bd96a2c3834c6cb9db9fe
TBR=brucedawson
Reviewed-on: https://chromium-review.googlesource.com/709697
Commit-Queue: Nico Weber <thakis@chromium.org>
Reviewed-by: Nico Weber <thakis@chromium.org>
With VS 2017 Update 3 Preview 4 the behavior of vswhere was changed so
that it only reports on non-prerelease versions by default. This can be
overridden by passing -prerelease. This broke our packaging setup. A
temporary fix was used to package Preview 4 and this is the permanent.
When -prerelease is passed then vswhere will report on all installed
versions of VS, whether prerelease or not. The script will package the
first one that it encounters. For best results you should only install
one copy of VS when packaging it.
Trivia: the original RTW version of VS 2017 was incorrectly tagged as
isPrerelease: 1 which means that without the -prerelease flag it
doesn't show up either!
BUG=683729
Change-Id: I98c1acb671dccef7ede4443fbbf498796946c52b
Reviewed-on: https://chromium-review.googlesource.com/578767
Reviewed-by: Scott Graham <scottmg@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Some build machines end up running vctip.exe for some reason and then
it doesn't shut down in a timely manner, leaving locks on directories.
This changes the packaging script so that vctip.exe is not packaged, and
therefore won't be run.
vctip.exe is the "Microsoft VC compiler and tools experience
improvement data uploader" and it presumably runs automatically as part
of running the compiler. It's not clear what triggers it to run,
however omitting it should be safe.
BUG=735226
Change-Id: Ie6af562def6214a5bb130ccc09c732efc1769bcd
Reviewed-on: https://chromium-review.googlesource.com/544395
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
The second preview of VS 2017 Update 3 removed the MFC DLLs from the
redist directory. Luckily it appears that we don't actually need them.
The directories were also renamed from VC150 to VC141. This change
switches to VC* to make it more robust, enabled by the use of glob added
a few changes ago.
crrev.com/2938453003, which tests a package that was created with this
updated script, passed.
BUG=683729
R=scottmg@chromium.org
Change-Id: I4ae5dec949d16a83e1751e3d2ff7bd10a78b680f
Reviewed-on: https://chromium-review.googlesource.com/534353
Reviewed-by: Scott Graham <scottmg@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
On recent SDKs the size of the toolchain package grew. This change
filters out some unneeded directories in order to minimize this growth.
This change also fixes the bin paths for the 10.0.15063.0 SDK. Starting
with this SDK version the SDK version is part of the path. The script
does *not* handle packaging older SDKs anymore - older versions of the
script can be used for that purpose.
The 10.0.15063.0 SDK will be needed eventually in order to support
Windows 10 Creators Update.
Test builds are running on crrev.com/2913873003 (VS 2017) and
crrev.com/2914643003 (VS 2015).
Bug: 683729,682416
Change-Id: Ia89e3253869a45dd10c923a2edee53aaf086e12c
Reviewed-on: https://chromium-review.googlesource.com/519982
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
VS 2017 Preview installs to a different directory from VS 2017 RTM, so
the packaging script needs to be updated to handle this. It does this by
using vswhere.exe so that any VS 2017 install can be found
automatically. If there are multiple installs then it is not well
defined which one will be found, so make sure you don't have multiple
installs.
The other change is to more robustly find the many directory paths which
contain embedded version numbers. Previously these were hard-coded for
VS 2017 RTM but this quickly gets tedious so now glob.glob is used to
find the matching directories. This will intentionally fail if there is
any ambiguity.
R=scottmg@chromium.org
BUG=683729
Change-Id: I02f7ae7589e271d6d9897f899e0730d7163f76ef
Reviewed-on: https://chromium-review.googlesource.com/516442
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
The versions of dbghelp.dll that ship with the latest 10.0.14393.0 SDK
(yes, there are multiple versions) as of the VS 2017 launch cannot
handle /debug:fastlink binaries created by VS 2017. This leads to hangs
during symbol lookup, as reported here:
https://developercommunity.visualstudio.com/content/problem/36255/chromes-base-unittests-fails-with-vs-2017-due-to-s.html
The recommended fix is to copy dbghelp.dll from the VS install instead,
from Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions\CppUnitFramework
Without this fix base_unittests will hang and windbg will not work on
/debug:fastlink binaries. With this fix base_unittests completes
promptly.
BUG=683729
Change-Id: Ie58b9d898a1feb04f11e99891035d2e40a2a9c0f
Reviewed-on: https://chromium-review.googlesource.com/461385
Reviewed-by: Scott Graham <scottmg@chromium.org>
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
VS 2017 has been released and needs to be packaged so that we can
experiment with building with it. This is an initial pass at updating
the packaging script. The file layout has changed significantly in VS
2017. Compatibility with VS 2013 and VS 2015 has, I believe, been
maintained but it is not important enough to merit significant testing.
BUG=683729
Change-Id: I68e5a8d9fd389132b641743dbc070108497f54cb
Reviewed-on: https://chromium-review.googlesource.com/457153
Commit-Queue: Bruce Dawson <brucedawson@chromium.org>
Reviewed-by: Scott Graham <scottmg@chromium.org>
This change adds a --override option to the VC++ packaging script so
that an arbitrary bin/include/lib containing directory can be used to
package up those components of VC++. See this post for details:
https://blogs.msdn.microsoft.com/vcblog/2016/02/16/try-out-the-latest-c-compiler-toolset-without-waiting-for-the-next-update-of-visual-studio/
Typical usage is like this:
package_from_installed.py 2015 --override <projdir>\packages\VisualCppTools.14.0.24109-Pre\lib\native
In order to make this work the path tuple support was fixed so that the
local relative path was propagated to the destination path, instead of
just the file name.
At the same time, the code to package up the UCRT installers was deleted
because it isn't needed, and windows.winmd is filtered out because it
makes consistent package creation impossible (it is different on every
machine).
While removing the UCRT installers I discovered nested if VS_VERSION
checks. Oops. Fixed.
BUG=440500
Review-Url: https://codereview.chromium.org/1967653002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@300527 0039d316-1c4b-4281-b951-d872f2087c98
Every time I write a json file, I end up wishing I put my toplevel items
into non-toplevel items down the line when I want to add more stuff to the
json file. Address this now, while no toolchain with this json file has
been built yet.
Follow-up to https://codereview.chromium.org/1706423002/
BUG=495204
Review URL: https://codereview.chromium.org/1718083003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298908 0039d316-1c4b-4281-b951-d872f2087c98
No intended behavior change. This makes it possible to dump this state into
SetEnv.x32.json and SetEnv.x64.json in an easy follow-up.
BUG=495204
Review URL: https://codereview.chromium.org/1708223002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298856 0039d316-1c4b-4281-b951-d872f2087c98
This change packages the api-ms-* DLLs and the VS 2015 CRT DLLs in all
of the VS package directories that we add to the path, so that they can
run without having the UCRT installed.
The Common7\IDE path was removed because it isn't actually packaged, in
VS 2013 or VS 2015, so adding it to the path is purely confusing.
In addition to changing the packaging script the installation script has
to change in order to continue if the UCRT cannot be installed. It
still makes sense to try to install it, and print a message saying where
the installer is, for the convenience of Google developers who may want
more flexibility in running VS 2015 binaries.
A 'calculating hash' message was added to make the mysterious hashing
hangs (which can be several minutes long) less mysterious.
BUG=440500
Review URL: https://codereview.chromium.org/1660723002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298541 0039d316-1c4b-4281-b951-d872f2087c98
In change crrev.com/1504983002 the include\ucrt path from
the Windows 10 SDK was added to the include search path,
but this is not a legal thing to do on VS 2013. This change
makes the ucrt path VS 2015 specific.
Testing shows that this makes no difference to the VS 2015
package.
BUG=440500
Review URL: https://codereview.chromium.org/1609933004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298330 0039d316-1c4b-4281-b951-d872f2087c98
We use depot_tools's toolchain to build LLVM on the clang/win bots.
llvm-symbolizer relies on VSINSTALLDIR to be set to find the DIA SDK,
so set it. While here, also set VCINSTALLDIR.
BUG=82385
Review URL: https://codereview.chromium.org/1604423002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298326 0039d316-1c4b-4281-b951-d872f2087c98
The VS 2015 tools require the Windows 10 Universal C Runtime, either
installed in C:\Windows or in every directory containing tools, or
dynamically linked executables created with VS 2015. Installing to
C:\Windows seems less error prone.
This is only applicable for Google developers and build machines that
don't have VS 2015 installed.
This updates the packaging script so that it packages the three
installers, and no longer packages the installed files (which vary
between operating systems anyway).
The installer is updated to check for the existence of one of the
Universal C Runtime files. If it isn't found then it detects the
version of Windows in order to select and run the correct installer.
I manually confirmed that, for instance, the installers for Windows 7
and Windows 2008 R2, were identical despite coming from different
download URLs.
If the installation fails because gclient runhooks is run non-elevated
then the developer will have to do a one-time manual install of the
update. A message will be printed indicating this.
BUG=440500
Review URL: https://codereview.chromium.org/1588673004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298286 0039d316-1c4b-4281-b951-d872f2087c98
I coded these incorrectly in a previous change and hit them when
building a VS package.
Review URL: https://codereview.chromium.org/1567863004
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@298156 0039d316-1c4b-4281-b951-d872f2087c98
This change updates the packaging script for VS 2015 Update 1. Changes
include:
- Filtering out Windows Performance Toolkit to save space
- Filtering out .msi files to save space
- Adding a 'dryrun' option to quickly print statistics
- Allowing specifying what OS sub-version is desired
- Filtering out unused versions from the include/lib/source directories
- Avoiding the double-include of the ucrt directory
- Adding ucrt directory to include and lib path
- Handling running from 64-bit or 32-bit python
R=scottmg@chromium.org
BUG=chromium:440500
Review URL: https://codereview.chromium.org/1504983002
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@297894 0039d316-1c4b-4281-b951-d872f2087c98
Based on some fiddling on a local VM (see bug), but possibly still insufficient.
TBR=sebmarchand@chromium.org
BUG=492774, 495944
Review URL: https://codereview.chromium.org/1160683005
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295521 0039d316-1c4b-4281-b951-d872f2087c98
This is the other side of https://codereview.chromium.org/1163723003/
The changes here are to remove the use of 'vs2013_files' and 'win8sdk'
(as those will be different numbers soon enough) but still maintain
behaviour for the old "style" while in transition.
Secondarily, to remove the dependence of these two scripts on
'toolchain2013.py' as most of the script is now unused.
R=dpranke@chromium.org
BUG=440500, 492774
Review URL: https://codereview.chromium.org/1165563003
git-svn-id: svn://svn.chromium.org/chrome/trunk/tools/depot_tools@295485 0039d316-1c4b-4281-b951-d872f2087c98
Marc-Antoine Ruel
Mirko Bonadei
Daniel Bratell
Raul Tambre
Bruce Dawson
Kimmo Kinnunen
Nico Weber
sebmarchand@chromium.org
scottmg@chromium.org
scottmg@chromium.org