diff --git a/.eslintrc.yaml b/.eslintrc.yaml new file mode 100644 index 0000000000..659dce8810 --- /dev/null +++ b/.eslintrc.yaml @@ -0,0 +1,5 @@ +--- + env: + node: true + rules: + no-unused-vars: error diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md new file mode 100644 index 0000000000..dbd053a90b --- /dev/null +++ b/.github/ISSUE_TEMPLATE.md @@ -0,0 +1,26 @@ + + +* **Node Version**: +* **Platform**: +* **Compiler**: +* **Module**: + +
Verbose output (from npm or node-gyp): + + + +``` + +``` + +
+ + + diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000000..10156d89af --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,17 @@ + + +##### Checklist + + +- [ ] `npm install && npm test` passes +- [ ] tests are included +- [ ] documentation is changed or added +- [ ] commit message follows [commit guidelines](https://github.com/nodejs/node/blob/master/doc/guides/contributing/pull-requests.md#commit-message-guidelines) + +##### Description of change + + diff --git a/.gitignore b/.gitignore index 6748492014..4d6b4d55b7 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,5 @@ gyp/test node_modules test/.node-gyp +.ncu +package-lock.json diff --git a/CHANGELOG.md b/CHANGELOG.md index 9f7e66f8d0..33bbfad5de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,54 @@ +v3.8.0 2018-08-09 +================= + +* [[`c5929cb4fe`](https://github.com/nodejs/node-gyp/commit/c5929cb4fe)] - **doc**: update Xcode preferences tab name. (Ivan Daniluk) [#1330](https://github.com/nodejs/node-gyp/pull/1330) +* [[`8b488da8b9`](https://github.com/nodejs/node-gyp/commit/8b488da8b9)] - **doc**: update link to commit guidelines (Jonas Hermsmeier) [#1456](https://github.com/nodejs/node-gyp/pull/1456) +* [[`b4fe8c16f9`](https://github.com/nodejs/node-gyp/commit/b4fe8c16f9)] - **doc**: fix visual studio links (Bartosz Sosnowski) [#1490](https://github.com/nodejs/node-gyp/pull/1490) +* [[`536759c7e9`](https://github.com/nodejs/node-gyp/commit/536759c7e9)] - **configure**: use sys.version\_info to get python version (Yang Guo) [#1504](https://github.com/nodejs/node-gyp/pull/1504) +* [[`94c39c604e`](https://github.com/nodejs/node-gyp/commit/94c39c604e)] - **gyp**: fix ninja build failure (GYP patch) (Daniel Bevenius) [nodejs/node#12484](https://github.com/nodejs/node/pull/12484) +* [[`e8ea74e0fa`](https://github.com/nodejs/node-gyp/commit/e8ea74e0fa)] - **tools**: patch gyp to avoid xcrun errors (Ujjwal Sharma) [nodejs/node#21520](https://github.com/nodejs/node/pull/21520) +* [[`ea9aff44f2`](https://github.com/nodejs/node-gyp/commit/ea9aff44f2)] - **tools**: fix "the the" typos in comments (Masashi Hirano) [nodejs/node#20716](https://github.com/nodejs/node/pull/20716) +* [[`207e5aa4fd`](https://github.com/nodejs/node-gyp/commit/207e5aa4fd)] - **gyp**: implement LD/LDXX for ninja and FIPS (Sam Roberts) +* [[`b416c5f4b7`](https://github.com/nodejs/node-gyp/commit/b416c5f4b7)] - **gyp**: enable cctest to use objects (gyp part) (Daniel Bevenius) [nodejs/node#12450](https://github.com/nodejs/node/pull/12450) +* [[`40692d016b`](https://github.com/nodejs/node-gyp/commit/40692d016b)] - **gyp**: add compile\_commands.json gyp generator (Ben Noordhuis) [nodejs/node#12450](https://github.com/nodejs/node/pull/12450) +* [[`fc3c4e2b10`](https://github.com/nodejs/node-gyp/commit/fc3c4e2b10)] - **gyp**: float gyp patch for long filenames (Anna Henningsen) [nodejs/node#7963](https://github.com/nodejs/node/pull/7963) +* [[`8aedbfdef6`](https://github.com/nodejs/node-gyp/commit/8aedbfdef6)] - **gyp**: backport GYP fix to fix AIX shared suffix (Stewart Addison) +* [[`6cd84b84fc`](https://github.com/nodejs/node-gyp/commit/6cd84b84fc)] - **test**: formatting and minor fixes for execFileSync replacement (Rod Vagg) [#1521](https://github.com/nodejs/node-gyp/pull/1521) +* [[`60e421363f`](https://github.com/nodejs/node-gyp/commit/60e421363f)] - **test**: added test/processExecSync.js for when execFileSync is not available. (Rohit Hazra) [#1492](https://github.com/nodejs/node-gyp/pull/1492) +* [[`969447c5bd`](https://github.com/nodejs/node-gyp/commit/969447c5bd)] - **deps**: bump request to 2.8.7, fixes heok/hawk issues (Rohit Hazra) [#1492](https://github.com/nodejs/node-gyp/pull/1492) +* [[`340403ccfe`](https://github.com/nodejs/node-gyp/commit/340403ccfe)] - **win**: improve parsing of SDK version (Alessandro Vergani) [#1516](https://github.com/nodejs/node-gyp/pull/1516) + +v3.7.0 2018-06-08 +================= + +* [[`84cea7b30d`](https://github.com/nodejs/node-gyp/commit/84cea7b30d)] - Remove unused gyp test scripts. (Ben Noordhuis) [#1458](https://github.com/nodejs/node-gyp/pull/1458) +* [[`0540e4ec63`](https://github.com/nodejs/node-gyp/commit/0540e4ec63)] - **gyp**: escape spaces in filenames in make generator (Jeff Senn) [#1436](https://github.com/nodejs/node-gyp/pull/1436) +* [[`88fc6fa0ec`](https://github.com/nodejs/node-gyp/commit/88fc6fa0ec)] - Drop dependency on minimatch. (Brian Woodward) [#1158](https://github.com/nodejs/node-gyp/pull/1158) +* [[`1e203c5148`](https://github.com/nodejs/node-gyp/commit/1e203c5148)] - Fix include path when pointing to Node.js source (Richard Lau) [#1055](https://github.com/nodejs/node-gyp/pull/1055) +* [[`53d8cb967c`](https://github.com/nodejs/node-gyp/commit/53d8cb967c)] - Prefix build targets with /t: on Windows (Natalie Wolfe) [#1164](https://github.com/nodejs/node-gyp/pull/1164) +* [[`53a5f8ff38`](https://github.com/nodejs/node-gyp/commit/53a5f8ff38)] - **gyp**: add support for .mm files to msvs generator (Julien Racle) [#1167](https://github.com/nodejs/node-gyp/pull/1167) +* [[`dd8561e528`](https://github.com/nodejs/node-gyp/commit/dd8561e528)] - **zos**: don't use universal-new-lines mode (John Barboza) [#1451](https://github.com/nodejs/node-gyp/pull/1451) +* [[`e5a69010ed`](https://github.com/nodejs/node-gyp/commit/e5a69010ed)] - **zos**: add search locations for libnode.x (John Barboza) [#1451](https://github.com/nodejs/node-gyp/pull/1451) +* [[`79febace53`](https://github.com/nodejs/node-gyp/commit/79febace53)] - **doc**: update macOS information in README (Josh Parnham) [#1323](https://github.com/nodejs/node-gyp/pull/1323) +* [[`9425448945`](https://github.com/nodejs/node-gyp/commit/9425448945)] - **gyp**: don't print xcodebuild not found errors (Gibson Fahnestock) [#1370](https://github.com/nodejs/node-gyp/pull/1370) +* [[`6f1286f5b2`](https://github.com/nodejs/node-gyp/commit/6f1286f5b2)] - Fix infinite install loop. (Ben Noordhuis) [#1384](https://github.com/nodejs/node-gyp/pull/1384) +* [[`2580b9139e`](https://github.com/nodejs/node-gyp/commit/2580b9139e)] - Update `--nodedir` description in README. (Ben Noordhuis) [#1372](https://github.com/nodejs/node-gyp/pull/1372) +* [[`a61360391a`](https://github.com/nodejs/node-gyp/commit/a61360391a)] - Update README with another way to install on windows (JeffAtDeere) [#1352](https://github.com/nodejs/node-gyp/pull/1352) +* [[`47496bf6dc`](https://github.com/nodejs/node-gyp/commit/47496bf6dc)] - Fix IndexError when parsing GYP files. (Ben Noordhuis) [#1267](https://github.com/nodejs/node-gyp/pull/1267) +* [[`b2024dee7b`](https://github.com/nodejs/node-gyp/commit/b2024dee7b)] - **zos**: support platform (John Barboza) [#1276](https://github.com/nodejs/node-gyp/pull/1276) +* [[`90d86512f4`](https://github.com/nodejs/node-gyp/commit/90d86512f4)] - **win**: run PS with `-NoProfile` (Refael Ackermann) [#1292](https://github.com/nodejs/node-gyp/pull/1292) +* [[`2da5f86ef7`](https://github.com/nodejs/node-gyp/commit/2da5f86ef7)] - **doc**: add github PR and Issue templates (Gibson Fahnestock) [#1228](https://github.com/nodejs/node-gyp/pull/1228) +* [[`a46a770d68`](https://github.com/nodejs/node-gyp/commit/a46a770d68)] - **doc**: update proposed DCO and CoC (Mikeal Rogers) [#1229](https://github.com/nodejs/node-gyp/pull/1229) +* [[`7e803d58e0`](https://github.com/nodejs/node-gyp/commit/7e803d58e0)] - **doc**: headerify the Install instructions (Nick Schonning) [#1225](https://github.com/nodejs/node-gyp/pull/1225) +* [[`f27599193a`](https://github.com/nodejs/node-gyp/commit/f27599193a)] - **gyp**: update xml string encoding conversion (Liu Chao) [#1203](https://github.com/nodejs/node-gyp/pull/1203) +* [[`0a07e481f7`](https://github.com/nodejs/node-gyp/commit/0a07e481f7)] - **configure**: don't set ensure if tarball is set (Gibson Fahnestock) [#1220](https://github.com/nodejs/node-gyp/pull/1220) + +v3.6.3 2018-06-08 +================= + +* [[`90cd2e8da9`](https://github.com/nodejs/node-gyp/commit/90cd2e8da9)] - **gyp**: fix regex to match multi-digit versions (Jonas Hermsmeier) [#1455](https://github.com/nodejs/node-gyp/pull/1455) +* [[`7900122337`](https://github.com/nodejs/node-gyp/commit/7900122337)] - deps: pin `request` version range (Refael Ackerman) [#1300](https://github.com/nodejs/node-gyp/pull/1300) + v3.6.2 2017-06-01 ================= diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..f48786bd84 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,34 @@ +# Contributing to node-gyp + +## Code of Conduct + +Please read the +[Code of Conduct](https://github.com/nodejs/TSC/blob/master/CODE_OF_CONDUCT.md) +which explains the minimum behavior expectations for node-gyp contributors. + + +## Developer's Certificate of Origin 1.1 + +By making a contribution to this project, I certify that: + +* (a) The contribution was created in whole or in part by me and I + have the right to submit it under the open source license + indicated in the file; or + +* (b) The contribution is based upon previous work that, to the best + of my knowledge, is covered under an appropriate open source + license and I have the right under that license to submit that + work with modifications, whether created in whole or in part + by me, under the same open source license (unless I am + permitted to submit under a different license), as indicated + in the file; or + +* (c) The contribution was provided directly to me by some other + person who certified (a), (b) or (c) and I have not modified + it. + +* (d) I understand and agree that this project and the contribution + are public and that a record of the contribution (including all + personal information I submit with it, including my sign-off) is + maintained indefinitely and may be redistributed consistent with + this project or the open source license(s) involved. diff --git a/README.md b/README.md index 03db320900..e88bd0b5be 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,21 @@ -node-gyp -========= -### Node.js native addon build tool +# `node-gyp` - Node.js native addon build tool `node-gyp` is a cross-platform command-line tool written in Node.js for compiling -native addon modules for Node.js. It bundles the [gyp](https://gyp.gsrc.io) +native addon modules for Node.js. It bundles the [gyp](https://gyp.gsrc.io) project used by the Chromium team and takes away the pain of dealing with the -various differences in build platforms. It is the replacement to the `node-waf` -program which is removed for node `v0.8`. If you have a native addon for node that -still has a `wscript` file, then you should definitely add a `binding.gyp` file -to support the latest versions of node. +various differences in build platforms. -Multiple target versions of node are supported (i.e. `0.8`, ..., `4`, `5`, `6`, -etc.), regardless of what version of node is actually installed on your system +Multiple target versions of Node.js are supported (i.e. `0.8`, ..., `4`, `5`, `6`, +etc.), regardless of what version of Node.js is actually installed on your system (`node-gyp` downloads the necessary development files or headers for the target version). -#### Features: +## Features * Easy to use, consistent interface * Same commands to build your module on every platform - * Supports multiple target versions of Node + * Supports multiple target versions of Node.js - -Installation ------------- +## Installation You can install with `npm`: @@ -32,29 +25,35 @@ $ npm install -g node-gyp You will also need to install: - * On Unix: - * `python` (`v2.7` recommended, `v3.x.x` is __*not*__ supported) - * `make` - * A proper C/C++ compiler toolchain, like [GCC](https://gcc.gnu.org) - * On Mac OS X: - * `python` (`v2.7` recommended, `v3.x.x` is __*not*__ supported) (already installed on Mac OS X) - * [Xcode](https://developer.apple.com/xcode/download/) - * You also need to install the `Command Line Tools` via Xcode. You can find this under the menu `Xcode -> Preferences -> Downloads` - * This step will install `gcc` and the related toolchain containing `make` - * On Windows: - * Option 1: Install all the required tools and configurations using Microsoft's [windows-build-tools](https://github.com/felixrieseberg/windows-build-tools) using `npm install --global --production windows-build-tools` from an elevated PowerShell or CMD.exe (run as Administrator). - * Option 2: Install tools and configuration manually: - * Visual C++ Build Environment: - * Option 1: Install [Visual C++ Build Tools](http://landinghub.visualstudio.com/visual-cpp-build-tools) using the **Default Install** option. +### On Unix + + * `python` (`v2.7` recommended, `v3.x.x` is __*not*__ supported) + * `make` + * A proper C/C++ compiler toolchain, like [GCC](https://gcc.gnu.org) + +### On macOS + + * `python` (`v2.7` recommended, `v3.x.x` is __*not*__ supported) (already installed on macOS) + * [Xcode](https://developer.apple.com/xcode/download/) + * You also need to install the `Command Line Tools` via Xcode. You can find this under the menu `Xcode -> Preferences -> Locations` (or by running `xcode-select --install` in your Terminal) + * This step will install `gcc` and the related toolchain containing `make` - * Option 2: Install [Visual Studio 2015](https://www.visualstudio.com/products/visual-studio-community-vs) (or modify an existing installation) and select *Common Tools for Visual C++* during setup. This also works with the free Community and Express for Desktop editions. +### On Windows - > :bulb: [Windows Vista / 7 only] requires [.NET Framework 4.5.1](http://www.microsoft.com/en-us/download/details.aspx?id=40773) +#### Option 1 - * Install [Python 2.7](https://www.python.org/downloads/) (`v3.x.x` is not supported), and run `npm config set python python2.7` (or see below for further instructions on specifying the proper Python version and path.) - * Launch cmd, `npm config set msvs_version 2015` +Install all the required tools and configurations using Microsoft's [windows-build-tools](https://github.com/felixrieseberg/windows-build-tools) using `npm install --global --production windows-build-tools` from an elevated PowerShell or CMD.exe (run as Administrator). - If the above steps didn't work for you, please visit [Microsoft's Node.js Guidelines for Windows](https://github.com/Microsoft/nodejs-guidelines/blob/master/windows-environment.md#compiling-native-addon-modules) for additional tips. +#### Option 2 + +Install tools and configuration manually: + * Install Visual C++ Build Environment: [Visual Studio Build Tools](https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools) + (using "Visual C++ build tools" workload) or [Visual Studio 2017 Community](https://visualstudio.microsoft.com/pl/thank-you-downloading-visual-studio/?sku=Community) + (using the "Desktop development with C++" workload) + * Install [Python 2.7](https://www.python.org/downloads/) (`v3.x.x` is not supported), and run `npm config set python python2.7` (or see below for further instructions on specifying the proper Python version and path.) + * Launch cmd, `npm config set msvs_version 2017` + + If the above steps didn't work for you, please visit [Microsoft's Node.js Guidelines for Windows](https://github.com/Microsoft/nodejs-guidelines/blob/master/windows-environment.md#compiling-native-addon-modules) for additional tips. If you have multiple Python versions installed, you can identify which Python version `node-gyp` uses by setting the '--python' variable: @@ -63,7 +62,7 @@ version `node-gyp` uses by setting the '--python' variable: $ node-gyp --python /path/to/python2.7 ``` -If `node-gyp` is called by way of `npm` *and* you have multiple versions of +If `node-gyp` is called by way of `npm`, *and* you have multiple versions of Python installed, then you can set `npm`'s 'python' config key to the appropriate value: @@ -71,12 +70,7 @@ value: $ npm config set python /path/to/executable/python2.7 ``` -Note that OS X is just a flavour of Unix and so needs `python`, `make`, and C/C++. -An easy way to obtain these is to install XCode from Apple, -and then use it to install the command line tools (under Preferences -> Downloads). - -How to Use ----------- +## How to Use To compile your native addon, first go to its root directory: @@ -97,33 +91,30 @@ needs to be added (not needed when run by npm as configured above): $ node-gyp configure --msvs_version=2015 ``` -__Note__: The `configure` step looks for the `binding.gyp` file in the current -directory to process. See below for instructions on creating the `binding.gyp` file. +__Note__: The `configure` step looks for a `binding.gyp` file in the current +directory to process. See below for instructions on creating a `binding.gyp` file. Now you will have either a `Makefile` (on Unix platforms) or a `vcxproj` file -(on Windows) in the `build/` directory. Next invoke the `build` command: +(on Windows) in the `build/` directory. Next, invoke the `build` command: ``` bash $ node-gyp build ``` Now you have your compiled `.node` bindings file! The compiled bindings end up -in `build/Debug/` or `build/Release/`, depending on the build mode. At this point -you can require the `.node` file with Node and run your tests! +in `build/Debug/` or `build/Release/`, depending on the build mode. At this point, +you can require the `.node` file with Node.js and run your tests! __Note:__ To create a _Debug_ build of the bindings file, pass the `--debug` (or -`-d`) switch when running either the `configure`, `build` or `rebuild` command. +`-d`) switch when running either the `configure`, `build` or `rebuild` commands. +## The `binding.gyp` file -The "binding.gyp" file ----------------------- +A `binding.gyp` file describes the configuration to build your module, in a +JSON-like format. This file gets placed in the root of your package, alongside +`package.json`. -Previously when node had `node-waf` you had to write a `wscript` file. The -replacement for that is the `binding.gyp` file, which describes the configuration -to build your module in a JSON-like format. This file gets placed in the root of -your package, alongside the `package.json` file. - -A barebones `gyp` file appropriate for building a node addon looks like: +A barebones `gyp` file appropriate for building a Node.js addon could look like: ``` python { @@ -145,8 +136,7 @@ Some additional resources for addons and writing `gyp` files: * [*"binding.gyp" files out in the wild* wiki page](https://github.com/nodejs/node-gyp/wiki/%22binding.gyp%22-files-out-in-the-wild) -Commands --------- +## Commands `node-gyp` responds to the following commands: @@ -157,86 +147,74 @@ Commands | `clean` | Removes the `build` directory if it exists | `configure` | Generates project build files for the current platform | `rebuild` | Runs `clean`, `configure` and `build` all in a row -| `install` | Installs node header files for the given version -| `list` | Lists the currently installed node header versions -| `remove` | Removes the node header files for the given version +| `install` | Installs Node.js header files for the given version +| `list` | Lists the currently installed Node.js header versions +| `remove` | Removes the Node.js header files for the given version -Command Options --------- +## Command Options `node-gyp` accepts the following command options: | **Command** | **Description** |:----------------------------------|:------------------------------------------ -| `-j n`, `--jobs n` | Run make in parallel -| `--target=v6.2.1` | Node version to build for (default=process.version) +| `-j n`, `--jobs n` | Run `make` in parallel +| `--target=v6.2.1` | Node.js version to build for (default is `process.version`) | `--silly`, `--loglevel=silly` | Log all progress to console | `--verbose`, `--loglevel=verbose` | Log most progress to console | `--silent`, `--loglevel=silent` | Don't log anything to console -| `debug`, `--debug` | Make Debug build (default=Release) +| `debug`, `--debug` | Make Debug build (default is `Release`) | `--release`, `--no-debug` | Make Release build | `-C $dir`, `--directory=$dir` | Run command in different directory -| `--make=$make` | Override make command (e.g. gmake) +| `--make=$make` | Override `make` command (e.g. `gmake`) | `--thin=yes` | Enable thin static libraries | `--arch=$arch` | Set target architecture (e.g. ia32) | `--tarball=$path` | Get headers from a local tarball -| `--devdir=$path` | SDK download directory (default=~/.node-gyp) +| `--devdir=$path` | SDK download directory (default is `~/.node-gyp`) | `--ensure` | Don't reinstall headers if already present | `--dist-url=$url` | Download header tarball from custom URL | `--proxy=$url` | Set HTTP proxy for downloading header tarball | `--cafile=$cafile` | Override default CA chain (to download tarball) -| `--nodedir=$path` | Set the path to the node binary -| `--python=$path` | Set path to the python (2) binary -| `--msvs_version=$version` | Set Visual Studio version (win) -| `--solution=$solution` | Set Visual Studio Solution version (win) - - -Configuration --------- - -__`node-gyp` responds to environment variables or `npm` configuration__ -1. Environment variables take the form `npm_config_OPTION_NAME` for any of the - options listed above (dashes in option names should be replaced by underscores). - These work also when `node-gyp` is invoked directly: - `$ export npm_config_devdir=/tmp/.gyp` - or on Windows - `> set npm_config_devdir=c:\temp\.gyp` -2. As `npm` configuration, variables take the form `OPTION_NAME`. - This way only works when `node-gyp` is executed by `npm`: - `$ npm config set [--global] devdir /tmp/.gyp` - `$ npm i buffertools` - - - -License -------- - -(The MIT License) - -Copyright (c) 2012 Nathan Rajlich <nathan@tootallnate.net> - -Permission is hereby granted, free of charge, to any person obtaining -a copy of this software and associated documentation files (the -'Software'), to deal in the Software without restriction, including -without limitation the rights to use, copy, modify, merge, publish, -distribute, sublicense, and/or sell copies of the Software, and to -permit persons to whom the Software is furnished to do so, subject to -the following conditions: - -The above copyright notice and this permission notice shall be -included in all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, -EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. -IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY -CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, -TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE -SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - - -[python-v2.7.10]: https://www.python.org/downloads/release/python-2710/ -[msvc2013]: https://www.microsoft.com/en-gb/download/details.aspx?id=44914 -[win7sdk]: https://www.microsoft.com/en-us/download/details.aspx?id=8279 -[compiler update for the Windows SDK 7.1]: https://www.microsoft.com/en-us/download/details.aspx?id=4422 +| `--nodedir=$path` | Set the path to the node source code +| `--python=$path` | Set path to the Python 2 binary +| `--msvs_version=$version` | Set Visual Studio version (Windows only) +| `--solution=$solution` | Set Visual Studio Solution version (Windows only) + +## Configuration + +### Environment variables + +Use the form `npm_config_OPTION_NAME` for any of the command options listed +above (dashes in option names should be replaced by underscores). + +For example, to set `devdir` equal to `/tmp/.gyp`, you would: + +Run this on Unix: + +```bash +$ export npm_config_devdir=/tmp/.gyp +``` + +Or this on Windows: + +```console +> set npm_config_devdir=c:\temp\.gyp +``` + +### `npm` configuration + +Use the form `OPTION_NAME` for any of the command options listed above. + +For example, to set `devdir` equal to `/tmp/.gyp`, you would run: + +```bash +$ npm config set [--global] devdir /tmp/.gyp +``` + +**Note:** Configuration set via `npm` will only be used when `node-gyp` +is run via `npm`, not when `node-gyp` is run directly. + +## License + +`node-gyp` is available under the MIT license. See the [LICENSE +file](LICENSE) for details. diff --git a/addon.gypi b/addon.gypi index f2f6a7925e..55fb321118 100644 --- a/addon.gypi +++ b/addon.gypi @@ -18,7 +18,10 @@ 'include_dirs': [ '<(node_root_dir)/include/node', '<(node_root_dir)/src', + '<(node_root_dir)/deps/openssl/config', + '<(node_root_dir)/deps/openssl/openssl/include', '<(node_root_dir)/deps/uv/include', + '<(node_root_dir)/deps/zlib', '<(node_root_dir)/<(node_engine_include_dir)' ], 'defines!': [ @@ -89,6 +92,17 @@ '-Wl,-bimport:<(node_exp_file)' ], }], + [ 'OS=="zos"', { + 'cflags': [ + '-q64', + '-Wc,DLL', + '-qlonglong' + ], + 'ldflags': [ + '-q64', + '<(node_exp_file)' + ], + }], [ 'OS=="win"', { 'conditions': [ ['node_engine=="chakracore"', { diff --git a/bin/node-gyp.js b/bin/node-gyp.js index 70d7d50262..a8fd9bc529 100755 --- a/bin/node-gyp.js +++ b/bin/node-gyp.js @@ -1,15 +1,7 @@ #!/usr/bin/env node -/** - * Set the title. - */ - process.title = 'node-gyp' -/** - * Module dependencies. - */ - var gyp = require('../') var log = require('npmlog') var osenv = require('osenv') @@ -42,7 +34,7 @@ if (prog.todo.length === 0) { } else { console.log('%s', prog.usage()) } - return process.exit(0) + process.exit(0) } log.info('it worked if it ends with', 'ok') @@ -126,7 +118,7 @@ process.on('uncaughtException', function (err) { }) function errorMessage () { - // copied from npm's lib/util/error-handler.js + // copied from npm's lib/utils/error-handler.js var os = require('os') log.error('System', os.type() + ' ' + os.release()) log.error('command', process.argv diff --git a/gyp/AUTHORS b/gyp/AUTHORS index fecf84a1c4..d76d8cd768 100644 --- a/gyp/AUTHORS +++ b/gyp/AUTHORS @@ -1,9 +1,10 @@ # Names should be added to this file like so: # Name or Organization -Google Inc. -Bloomberg Finance L.P. -Yandex LLC +Google Inc. <*@google.com> +Bloomberg Finance L.P. <*@bloomberg.net> +IBM Inc. <*@*.ibm.com> +Yandex LLC <*@yandex-team.ru> Steven Knight Ryan Norton diff --git a/gyp/PRESUBMIT.py b/gyp/PRESUBMIT.py index dde025383c..e52f9d2d22 100644 --- a/gyp/PRESUBMIT.py +++ b/gyp/PRESUBMIT.py @@ -73,23 +73,14 @@ ] -def CheckChangeOnUpload(input_api, output_api): - report = [] - report.extend(input_api.canned_checks.PanProjectChecks( - input_api, output_api)) - return report - - -def CheckChangeOnCommit(input_api, output_api): - report = [] - +def _LicenseHeader(input_api): # Accept any year number from 2009 to the current year. current_year = int(input_api.time.strftime('%Y')) - allowed_years = (str(s) for s in reversed(xrange(2009, current_year + 1))) + allowed_years = (str(s) for s in reversed(range(2009, current_year + 1))) years_re = '(' + '|'.join(allowed_years) + ')' # The (c) is deprecated, but tolerate it until it's removed from all files. - license = ( + return ( r'.*? Copyright (\(c\) )?%(year)s Google Inc\. All rights reserved\.\n' r'.*? Use of this source code is governed by a BSD-style license that ' r'can be\n' @@ -98,8 +89,18 @@ def CheckChangeOnCommit(input_api, output_api): 'year': years_re, } +def CheckChangeOnUpload(input_api, output_api): + report = [] + report.extend(input_api.canned_checks.PanProjectChecks( + input_api, output_api, license_header=_LicenseHeader(input_api))) + return report + + +def CheckChangeOnCommit(input_api, output_api): + report = [] + report.extend(input_api.canned_checks.PanProjectChecks( - input_api, output_api, license_header=license)) + input_api, output_api, license_header=_LicenseHeader(input_api))) report.extend(input_api.canned_checks.CheckTreeIsOpen( input_api, output_api, 'http://gyp-status.appspot.com/status', diff --git a/gyp/buildbot/aosp_manifest.xml b/gyp/buildbot/aosp_manifest.xml deleted file mode 100644 index bd73b303c6..0000000000 --- a/gyp/buildbot/aosp_manifest.xml +++ /dev/null @@ -1,466 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/gyp/buildbot/buildbot_run.py b/gyp/buildbot/buildbot_run.py deleted file mode 100755 index 9a2b71f1b3..0000000000 --- a/gyp/buildbot/buildbot_run.py +++ /dev/null @@ -1,136 +0,0 @@ -#!/usr/bin/env python -# Copyright (c) 2012 Google Inc. All rights reserved. -# Use of this source code is governed by a BSD-style license that can be -# found in the LICENSE file. - -"""Argument-less script to select what to run on the buildbots.""" - -import os -import shutil -import subprocess -import sys - - -BUILDBOT_DIR = os.path.dirname(os.path.abspath(__file__)) -TRUNK_DIR = os.path.dirname(BUILDBOT_DIR) -ROOT_DIR = os.path.dirname(TRUNK_DIR) -CMAKE_DIR = os.path.join(ROOT_DIR, 'cmake') -CMAKE_BIN_DIR = os.path.join(CMAKE_DIR, 'bin') -OUT_DIR = os.path.join(TRUNK_DIR, 'out') - - -def CallSubProcess(*args, **kwargs): - """Wrapper around subprocess.call which treats errors as build exceptions.""" - with open(os.devnull) as devnull_fd: - retcode = subprocess.call(stdin=devnull_fd, *args, **kwargs) - if retcode != 0: - print '@@@STEP_EXCEPTION@@@' - sys.exit(1) - - -def PrepareCmake(): - """Build CMake 2.8.8 since the version in Precise is 2.8.7.""" - if os.environ['BUILDBOT_CLOBBER'] == '1': - print '@@@BUILD_STEP Clobber CMake checkout@@@' - shutil.rmtree(CMAKE_DIR) - - # We always build CMake 2.8.8, so no need to do anything - # if the directory already exists. - if os.path.isdir(CMAKE_DIR): - return - - print '@@@BUILD_STEP Initialize CMake checkout@@@' - os.mkdir(CMAKE_DIR) - - print '@@@BUILD_STEP Sync CMake@@@' - CallSubProcess( - ['git', 'clone', - '--depth', '1', - '--single-branch', - '--branch', 'v2.8.8', - '--', - 'git://cmake.org/cmake.git', - CMAKE_DIR], - cwd=CMAKE_DIR) - - print '@@@BUILD_STEP Build CMake@@@' - CallSubProcess( - ['/bin/bash', 'bootstrap', '--prefix=%s' % CMAKE_DIR], - cwd=CMAKE_DIR) - - CallSubProcess( ['make', 'cmake'], cwd=CMAKE_DIR) - - -def GypTestFormat(title, format=None, msvs_version=None, tests=[]): - """Run the gyp tests for a given format, emitting annotator tags. - - See annotator docs at: - https://sites.google.com/a/chromium.org/dev/developers/testing/chromium-build-infrastructure/buildbot-annotations - Args: - format: gyp format to test. - Returns: - 0 for sucesss, 1 for failure. - """ - if not format: - format = title - - print '@@@BUILD_STEP ' + title + '@@@' - sys.stdout.flush() - env = os.environ.copy() - if msvs_version: - env['GYP_MSVS_VERSION'] = msvs_version - command = ' '.join( - [sys.executable, 'gyp/gyptest.py', - '--all', - '--passed', - '--format', format, - '--path', CMAKE_BIN_DIR, - '--chdir', 'gyp'] + tests) - retcode = subprocess.call(command, cwd=ROOT_DIR, env=env, shell=True) - if retcode: - # Emit failure tag, and keep going. - print '@@@STEP_FAILURE@@@' - return 1 - return 0 - - -def GypBuild(): - # Dump out/ directory. - print '@@@BUILD_STEP cleanup@@@' - print 'Removing %s...' % OUT_DIR - shutil.rmtree(OUT_DIR, ignore_errors=True) - print 'Done.' - - retcode = 0 - if sys.platform.startswith('linux'): - retcode += GypTestFormat('ninja') - retcode += GypTestFormat('make') - PrepareCmake() - retcode += GypTestFormat('cmake') - elif sys.platform == 'darwin': - retcode += GypTestFormat('ninja') - retcode += GypTestFormat('xcode') - retcode += GypTestFormat('make') - elif sys.platform == 'win32': - retcode += GypTestFormat('ninja') - if os.environ['BUILDBOT_BUILDERNAME'] == 'gyp-win64': - retcode += GypTestFormat('msvs-ninja-2013', format='msvs-ninja', - msvs_version='2013', - tests=[ - r'test\generator-output\gyptest-actions.py', - r'test\generator-output\gyptest-relocate.py', - r'test\generator-output\gyptest-rules.py']) - retcode += GypTestFormat('msvs-2013', format='msvs', msvs_version='2013') - else: - raise Exception('Unknown platform') - if retcode: - # TODO(bradnelson): once the annotator supports a postscript (section for - # after the build proper that could be used for cumulative failures), - # use that instead of this. This isolates the final return value so - # that it isn't misattributed to the last stage. - print '@@@BUILD_STEP failures@@@' - sys.exit(retcode) - - -if __name__ == '__main__': - GypBuild() diff --git a/gyp/buildbot/commit_queue/OWNERS b/gyp/buildbot/commit_queue/OWNERS deleted file mode 100644 index b269c198b4..0000000000 --- a/gyp/buildbot/commit_queue/OWNERS +++ /dev/null @@ -1,6 +0,0 @@ -set noparent -bradnelson@chromium.org -bradnelson@google.com -iannucci@chromium.org -scottmg@chromium.org -thakis@chromium.org diff --git a/gyp/buildbot/commit_queue/README b/gyp/buildbot/commit_queue/README deleted file mode 100644 index 9428497883..0000000000 --- a/gyp/buildbot/commit_queue/README +++ /dev/null @@ -1,3 +0,0 @@ -cq_config.json describes the trybots that must pass in order -to land a change through the commit queue. -Comments are here as the file is strictly JSON. diff --git a/gyp/buildbot/commit_queue/cq_config.json b/gyp/buildbot/commit_queue/cq_config.json deleted file mode 100644 index 656c21e54f..0000000000 --- a/gyp/buildbot/commit_queue/cq_config.json +++ /dev/null @@ -1,15 +0,0 @@ -{ - "trybots": { - "launched": { - "tryserver.nacl": { - "gyp-presubmit": ["defaulttests"], - "gyp-linux": ["defaulttests"], - "gyp-mac": ["defaulttests"], - "gyp-win32": ["defaulttests"], - "gyp-win64": ["defaulttests"] - } - }, - "triggered": { - } - } -} diff --git a/gyp/gyptest.py b/gyp/gyptest.py deleted file mode 100755 index 8e4fc47d5c..0000000000 --- a/gyp/gyptest.py +++ /dev/null @@ -1,274 +0,0 @@ -#!/usr/bin/env python - -# Copyright (c) 2012 Google Inc. All rights reserved. -# Use of this source code is governed by a BSD-style license that can be -# found in the LICENSE file. - -__doc__ = """ -gyptest.py -- test runner for GYP tests. -""" - -import os -import optparse -import subprocess -import sys - -class CommandRunner(object): - """ - Executor class for commands, including "commands" implemented by - Python functions. - """ - verbose = True - active = True - - def __init__(self, dictionary={}): - self.subst_dictionary(dictionary) - - def subst_dictionary(self, dictionary): - self._subst_dictionary = dictionary - - def subst(self, string, dictionary=None): - """ - Substitutes (via the format operator) the values in the specified - dictionary into the specified command. - - The command can be an (action, string) tuple. In all cases, we - perform substitution on strings and don't worry if something isn't - a string. (It's probably a Python function to be executed.) - """ - if dictionary is None: - dictionary = self._subst_dictionary - if dictionary: - try: - string = string % dictionary - except TypeError: - pass - return string - - def display(self, command, stdout=None, stderr=None): - if not self.verbose: - return - if type(command) == type(()): - func = command[0] - args = command[1:] - s = '%s(%s)' % (func.__name__, ', '.join(map(repr, args))) - if type(command) == type([]): - # TODO: quote arguments containing spaces - # TODO: handle meta characters? - s = ' '.join(command) - else: - s = self.subst(command) - if not s.endswith('\n'): - s += '\n' - sys.stdout.write(s) - sys.stdout.flush() - - def execute(self, command, stdout=None, stderr=None): - """ - Executes a single command. - """ - if not self.active: - return 0 - if type(command) == type(''): - command = self.subst(command) - cmdargs = shlex.split(command) - if cmdargs[0] == 'cd': - command = (os.chdir,) + tuple(cmdargs[1:]) - if type(command) == type(()): - func = command[0] - args = command[1:] - return func(*args) - else: - if stdout is sys.stdout: - # Same as passing sys.stdout, except python2.4 doesn't fail on it. - subout = None - else: - # Open pipe for anything else so Popen works on python2.4. - subout = subprocess.PIPE - if stderr is sys.stderr: - # Same as passing sys.stderr, except python2.4 doesn't fail on it. - suberr = None - elif stderr is None: - # Merge with stdout if stderr isn't specified. - suberr = subprocess.STDOUT - else: - # Open pipe for anything else so Popen works on python2.4. - suberr = subprocess.PIPE - p = subprocess.Popen(command, - shell=(sys.platform == 'win32'), - stdout=subout, - stderr=suberr) - p.wait() - if stdout is None: - self.stdout = p.stdout.read() - elif stdout is not sys.stdout: - stdout.write(p.stdout.read()) - if stderr not in (None, sys.stderr): - stderr.write(p.stderr.read()) - return p.returncode - - def run(self, command, display=None, stdout=None, stderr=None): - """ - Runs a single command, displaying it first. - """ - if display is None: - display = command - self.display(display) - return self.execute(command, stdout, stderr) - - -class Unbuffered(object): - def __init__(self, fp): - self.fp = fp - def write(self, arg): - self.fp.write(arg) - self.fp.flush() - def __getattr__(self, attr): - return getattr(self.fp, attr) - -sys.stdout = Unbuffered(sys.stdout) -sys.stderr = Unbuffered(sys.stderr) - - -def is_test_name(f): - return f.startswith('gyptest') and f.endswith('.py') - - -def find_all_gyptest_files(directory): - result = [] - for root, dirs, files in os.walk(directory): - if '.svn' in dirs: - dirs.remove('.svn') - result.extend([ os.path.join(root, f) for f in files if is_test_name(f) ]) - result.sort() - return result - - -def main(argv=None): - if argv is None: - argv = sys.argv - - usage = "gyptest.py [-ahlnq] [-f formats] [test ...]" - parser = optparse.OptionParser(usage=usage) - parser.add_option("-a", "--all", action="store_true", - help="run all tests") - parser.add_option("-C", "--chdir", action="store", default=None, - help="chdir to the specified directory") - parser.add_option("-f", "--format", action="store", default='', - help="run tests with the specified formats") - parser.add_option("-G", '--gyp_option', action="append", default=[], - help="Add -G options to the gyp command line") - parser.add_option("-l", "--list", action="store_true", - help="list available tests and exit") - parser.add_option("-n", "--no-exec", action="store_true", - help="no execute, just print the command line") - parser.add_option("--passed", action="store_true", - help="report passed tests") - parser.add_option("--path", action="append", default=[], - help="additional $PATH directory") - parser.add_option("-q", "--quiet", action="store_true", - help="quiet, don't print test command lines") - opts, args = parser.parse_args(argv[1:]) - - if opts.chdir: - os.chdir(opts.chdir) - - if opts.path: - extra_path = [os.path.abspath(p) for p in opts.path] - extra_path = os.pathsep.join(extra_path) - os.environ['PATH'] = extra_path + os.pathsep + os.environ['PATH'] - - if not args: - if not opts.all: - sys.stderr.write('Specify -a to get all tests.\n') - return 1 - args = ['test'] - - tests = [] - for arg in args: - if os.path.isdir(arg): - tests.extend(find_all_gyptest_files(os.path.normpath(arg))) - else: - if not is_test_name(os.path.basename(arg)): - print >>sys.stderr, arg, 'is not a valid gyp test name.' - sys.exit(1) - tests.append(arg) - - if opts.list: - for test in tests: - print test - sys.exit(0) - - CommandRunner.verbose = not opts.quiet - CommandRunner.active = not opts.no_exec - cr = CommandRunner() - - os.environ['PYTHONPATH'] = os.path.abspath('test/lib') - if not opts.quiet: - sys.stdout.write('PYTHONPATH=%s\n' % os.environ['PYTHONPATH']) - - passed = [] - failed = [] - no_result = [] - - if opts.format: - format_list = opts.format.split(',') - else: - # TODO: not duplicate this mapping from pylib/gyp/__init__.py - format_list = { - 'aix5': ['make'], - 'freebsd7': ['make'], - 'freebsd8': ['make'], - 'openbsd5': ['make'], - 'cygwin': ['msvs'], - 'win32': ['msvs', 'ninja'], - 'linux2': ['make', 'ninja'], - 'linux3': ['make', 'ninja'], - 'darwin': ['make', 'ninja', 'xcode', 'xcode-ninja'], - }[sys.platform] - - for format in format_list: - os.environ['TESTGYP_FORMAT'] = format - if not opts.quiet: - sys.stdout.write('TESTGYP_FORMAT=%s\n' % format) - - gyp_options = [] - for option in opts.gyp_option: - gyp_options += ['-G', option] - if gyp_options and not opts.quiet: - sys.stdout.write('Extra Gyp options: %s\n' % gyp_options) - - for test in tests: - status = cr.run([sys.executable, test] + gyp_options, - stdout=sys.stdout, - stderr=sys.stderr) - if status == 2: - no_result.append(test) - elif status: - failed.append(test) - else: - passed.append(test) - - if not opts.quiet: - def report(description, tests): - if tests: - if len(tests) == 1: - sys.stdout.write("\n%s the following test:\n" % description) - else: - fmt = "\n%s the following %d tests:\n" - sys.stdout.write(fmt % (description, len(tests))) - sys.stdout.write("\t" + "\n\t".join(tests) + "\n") - - if opts.passed: - report("Passed", passed) - report("Failed", failed) - report("No result from", no_result) - - if failed: - return 1 - else: - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/gyp/pylib/gyp/MSVSSettings.py b/gyp/pylib/gyp/MSVSSettings.py index 4985756bdd..8073f92b8d 100644 --- a/gyp/pylib/gyp/MSVSSettings.py +++ b/gyp/pylib/gyp/MSVSSettings.py @@ -14,6 +14,10 @@ MSBuild install directory, e.g. c:\Program Files (x86)\MSBuild """ +from __future__ import print_function + +from gyp import string_types + import sys import re @@ -106,11 +110,11 @@ class _String(_Type): """A setting that's just a string.""" def ValidateMSVS(self, value): - if not isinstance(value, basestring): + if not isinstance(value, string_types): raise ValueError('expected string; got %r' % value) def ValidateMSBuild(self, value): - if not isinstance(value, basestring): + if not isinstance(value, string_types): raise ValueError('expected string; got %r' % value) def ConvertToMSBuild(self, value): @@ -122,11 +126,11 @@ class _StringList(_Type): """A settings that's a list of strings.""" def ValidateMSVS(self, value): - if not isinstance(value, basestring) and not isinstance(value, list): + if not isinstance(value, string_types) and not isinstance(value, list): raise ValueError('expected string list; got %r' % value) def ValidateMSBuild(self, value): - if not isinstance(value, basestring) and not isinstance(value, list): + if not isinstance(value, string_types) and not isinstance(value, list): raise ValueError('expected string list; got %r' % value) def ConvertToMSBuild(self, value): @@ -400,7 +404,7 @@ def _ValidateExclusionSetting(setting, settings, error_msg, stderr=sys.stderr): if unrecognized: # We don't know this setting. Give a warning. - print >> stderr, error_msg + print(error_msg, file=stderr) def FixVCMacroSlashes(s): @@ -417,7 +421,7 @@ def FixVCMacroSlashes(s): def ConvertVCMacrosToMSBuild(s): - """Convert the the MSVS macros found in the string to the MSBuild equivalent. + """Convert the MSVS macros found in the string to the MSBuild equivalent. This list is probably not exhaustive. Add as needed. """ @@ -433,7 +437,7 @@ def ConvertVCMacrosToMSBuild(s): '$(PlatformName)': '$(Platform)', '$(SafeInputName)': '%(Filename)', } - for old, new in replace_map.iteritems(): + for old, new in replace_map.items(): s = s.replace(old, new) s = FixVCMacroSlashes(s) return s @@ -453,17 +457,17 @@ def ConvertToMSBuildSettings(msvs_settings, stderr=sys.stderr): dictionaries of settings and their values. """ msbuild_settings = {} - for msvs_tool_name, msvs_tool_settings in msvs_settings.iteritems(): + for msvs_tool_name, msvs_tool_settings in msvs_settings.items(): if msvs_tool_name in _msvs_to_msbuild_converters: msvs_tool = _msvs_to_msbuild_converters[msvs_tool_name] - for msvs_setting, msvs_value in msvs_tool_settings.iteritems(): + for msvs_setting, msvs_value in msvs_tool_settings.items(): if msvs_setting in msvs_tool: # Invoke the translation function. try: msvs_tool[msvs_setting](msvs_value, msbuild_settings) - except ValueError, e: - print >> stderr, ('Warning: while converting %s/%s to MSBuild, ' - '%s' % (msvs_tool_name, msvs_setting, e)) + except ValueError as e: + print('Warning: while converting %s/%s to MSBuild, ' + '%s' % (msvs_tool_name, msvs_setting, e), file=stderr) else: _ValidateExclusionSetting(msvs_setting, msvs_tool, @@ -472,8 +476,8 @@ def ConvertToMSBuildSettings(msvs_settings, stderr=sys.stderr): (msvs_tool_name, msvs_setting)), stderr) else: - print >> stderr, ('Warning: unrecognized tool %s while converting to ' - 'MSBuild.' % msvs_tool_name) + print('Warning: unrecognized tool %s while converting to ' + 'MSBuild.' % msvs_tool_name, file=stderr) return msbuild_settings @@ -513,13 +517,13 @@ def _ValidateSettings(validators, settings, stderr): for tool_name in settings: if tool_name in validators: tool_validators = validators[tool_name] - for setting, value in settings[tool_name].iteritems(): + for setting, value in settings[tool_name].items(): if setting in tool_validators: try: tool_validators[setting](value) - except ValueError, e: - print >> stderr, ('Warning: for %s/%s, %s' % - (tool_name, setting, e)) + except ValueError as e: + print('Warning: for %s/%s, %s' % + (tool_name, setting, e), file=stderr) else: _ValidateExclusionSetting(setting, tool_validators, @@ -528,7 +532,7 @@ def _ValidateSettings(validators, settings, stderr): stderr) else: - print >> stderr, ('Warning: unrecognized tool %s' % tool_name) + print('Warning: unrecognized tool %s' % (tool_name), file=stderr) # MSVS and MBuild names of the tools. diff --git a/gyp/pylib/gyp/MSVSUserFile.py b/gyp/pylib/gyp/MSVSUserFile.py index 6c07e9a893..2264d64015 100644 --- a/gyp/pylib/gyp/MSVSUserFile.py +++ b/gyp/pylib/gyp/MSVSUserFile.py @@ -91,7 +91,7 @@ def AddDebugSettings(self, config_name, command, environment = {}, if environment and isinstance(environment, dict): env_list = ['%s="%s"' % (key, val) - for (key,val) in environment.iteritems()] + for (key,val) in environment.items()] environment = ' '.join(env_list) else: environment = '' @@ -135,7 +135,7 @@ def AddDebugSettings(self, config_name, command, environment = {}, def WriteIfChanged(self): """Writes the user file.""" configs = ['Configurations'] - for config, spec in sorted(self.configurations.iteritems()): + for config, spec in sorted(self.configurations.items()): configs.append(spec) content = ['VisualStudioUserFile', diff --git a/gyp/pylib/gyp/MSVSUtil.py b/gyp/pylib/gyp/MSVSUtil.py index 0b32e91180..c8187eb331 100644 --- a/gyp/pylib/gyp/MSVSUtil.py +++ b/gyp/pylib/gyp/MSVSUtil.py @@ -235,7 +235,7 @@ def InsertLargePdbShims(target_list, target_dicts, vars): # Set up the shim to output its PDB to the same location as the final linker # target. - for config_name, config in shim_dict.get('configurations').iteritems(): + for config_name, config in shim_dict.get('configurations').items(): pdb_path = _GetPdbPath(target_dict, config_name, vars) # A few keys that we don't want to propagate. diff --git a/gyp/pylib/gyp/MSVSVersion.py b/gyp/pylib/gyp/MSVSVersion.py index d9bfa684fa..293b4145c1 100644 --- a/gyp/pylib/gyp/MSVSVersion.py +++ b/gyp/pylib/gyp/MSVSVersion.py @@ -158,7 +158,7 @@ def _RegistryQuery(key, value=None): text = None try: text = _RegistryQueryBase('Sysnative', key, value) - except OSError, e: + except OSError as e: if e.errno == errno.ENOENT: text = _RegistryQueryBase('System32', key, value) else: @@ -176,12 +176,18 @@ def _RegistryGetValueUsingWinReg(key, value): contents of the registry key's value, or None on failure. Throws ImportError if _winreg is unavailable. """ - import _winreg + try: + # Python 2 + from _winreg import OpenKey, QueryValueEx + except ImportError: + # Python 3 + from winreg import OpenKey, QueryValueEx + try: root, subkey = key.split('\\', 1) assert root == 'HKLM' # Only need HKLM for now. - with _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, subkey) as hkey: - return _winreg.QueryValueEx(hkey, value)[0] + with OpenKey(_winreg.HKEY_LOCAL_MACHINE, subkey) as hkey: + return QueryValueEx(hkey, value)[0] except WindowsError: return None diff --git a/gyp/pylib/gyp/__init__.py b/gyp/pylib/gyp/__init__.py index 668f38b60d..d5fa9ecf50 100755 --- a/gyp/pylib/gyp/__init__.py +++ b/gyp/pylib/gyp/__init__.py @@ -4,6 +4,8 @@ # Use of this source code is governed by a BSD-style license that can be # found in the LICENSE file. +from __future__ import print_function + import copy import gyp.input import optparse @@ -14,6 +16,13 @@ import traceback from gyp.common import GypError +try: + # Python 2 + string_types = basestring +except NameError: + # Python 3 + string_types = str + # Default debug modes for GYP debug = {} @@ -34,8 +43,8 @@ def DebugOutput(mode, message, *args): pass if args: message %= args - print '%s:%s:%d:%s %s' % (mode.upper(), os.path.basename(ctx[0]), - ctx[1], ctx[2], message) + print('%s:%s:%d:%s %s' % (mode.upper(), os.path.basename(ctx[0]), + ctx[1], ctx[2], message)) def FindBuildFiles(): extension = '.gyp' @@ -207,7 +216,7 @@ def Noop(value): # We always want to ignore the environment when regenerating, to avoid # duplicate or changed flags in the environment at the time of regeneration. flags = ['--ignore-environment'] - for name, metadata in options._regeneration_metadata.iteritems(): + for name, metadata in options._regeneration_metadata.items(): opt = metadata['opt'] value = getattr(options, name) value_predicate = metadata['type'] == 'path' and FixPath or Noop @@ -226,12 +235,12 @@ def Noop(value): (action == 'store_false' and not value)): flags.append(opt) elif options.use_environment and env_name: - print >>sys.stderr, ('Warning: environment regeneration unimplemented ' + print('Warning: environment regeneration unimplemented ' 'for %s flag %r env_name %r' % (action, opt, - env_name)) + env_name), file=sys.stderr) else: - print >>sys.stderr, ('Warning: regeneration unimplemented for action %r ' - 'flag %r' % (action, opt)) + print('Warning: regeneration unimplemented for action %r ' + 'flag %r' % (action, opt), file=sys.stderr) return flags @@ -410,7 +419,7 @@ def gyp_main(args): for option, value in sorted(options.__dict__.items()): if option[0] == '_': continue - if isinstance(value, basestring): + if isinstance(value, string_types): DebugOutput(DEBUG_GENERAL, " %s: '%s'", option, value) else: DebugOutput(DEBUG_GENERAL, " %s: %s", option, value) @@ -432,7 +441,7 @@ def gyp_main(args): build_file_dir = os.path.abspath(os.path.dirname(build_file)) build_file_dir_components = build_file_dir.split(os.path.sep) components_len = len(build_file_dir_components) - for index in xrange(components_len - 1, -1, -1): + for index in range(components_len - 1, -1, -1): if build_file_dir_components[index] == 'src': options.depth = os.path.sep.join(build_file_dir_components) break @@ -475,7 +484,7 @@ def gyp_main(args): if home_dot_gyp != None: default_include = os.path.join(home_dot_gyp, 'include.gypi') if os.path.exists(default_include): - print 'Using overrides found in ' + default_include + print('Using overrides found in ' + default_include) includes.append(default_include) # Command-line --include files come after the default include. @@ -536,7 +545,7 @@ def gyp_main(args): def main(args): try: return gyp_main(args) - except GypError, e: + except GypError as e: sys.stderr.write("gyp: %s\n" % e) return 1 diff --git a/gyp/pylib/gyp/common.py b/gyp/pylib/gyp/common.py index 256e3f3a6b..b7bae925f4 100644 --- a/gyp/pylib/gyp/common.py +++ b/gyp/pylib/gyp/common.py @@ -363,7 +363,7 @@ def close(self): same = False try: same = filecmp.cmp(self.tmp_path, filename, False) - except OSError, e: + except OSError as e: if e.errno != errno.ENOENT: raise @@ -382,9 +382,9 @@ def close(self): # # No way to get the umask without setting a new one? Set a safe one # and then set it back to the old value. - umask = os.umask(077) + umask = os.umask(0o77) os.umask(umask) - os.chmod(self.tmp_path, 0666 & ~umask) + os.chmod(self.tmp_path, 0o666 & ~umask) if sys.platform == 'win32' and os.path.exists(filename): # NOTE: on windows (but not cygwin) rename will not replace an # existing file, so it must be preceded with a remove. Sadly there @@ -429,6 +429,10 @@ def GetFlavor(params): return 'netbsd' if sys.platform.startswith('aix'): return 'aix' + if sys.platform.startswith('zos'): + return 'zos' + if sys.platform.startswith('os390'): + return 'zos' return 'linux' @@ -460,7 +464,7 @@ def CopyTool(flavor, out_path): ''.join([source[0], '# Generated by gyp. Do not edit.\n'] + source[1:])) # Make file executable. - os.chmod(tool_path, 0755) + os.chmod(tool_path, 0o755) # From Alex Martelli, diff --git a/gyp/pylib/gyp/easy_xml.py b/gyp/pylib/gyp/easy_xml.py index 2b0bb60cb4..7c3f621f1f 100644 --- a/gyp/pylib/gyp/easy_xml.py +++ b/gyp/pylib/gyp/easy_xml.py @@ -4,6 +4,8 @@ import re import os +import locale +from functools import reduce def XmlToString(content, encoding='utf-8', pretty=False): @@ -79,7 +81,7 @@ def _ConstructContentList(xml_parts, specification, pretty, level=0): # Optionally in second position is a dictionary of the attributes. rest = specification[1:] if rest and isinstance(rest[0], dict): - for at, val in sorted(rest[0].iteritems()): + for at, val in sorted(rest[0].items()): xml_parts.append(' %s="%s"' % (at, _XmlEscape(val, attr=True))) rest = rest[1:] if rest: @@ -115,11 +117,10 @@ def WriteXmlIfChanged(content, path, encoding='utf-8', pretty=False, xml_string = XmlToString(content, encoding, pretty) if win32 and os.linesep != '\r\n': xml_string = xml_string.replace('\n', '\r\n') - - try: - xml_string = xml_string.encode(encoding) - except Exception: - xml_string = unicode(xml_string, 'latin-1').encode(encoding) + + default_encoding = locale.getdefaultlocale()[1] + if default_encoding.upper() != encoding.upper(): + xml_string = xml_string.decode(default_encoding).encode(encoding) # Get the old content try: diff --git a/gyp/pylib/gyp/flock_tool.py b/gyp/pylib/gyp/flock_tool.py index b38d8660f7..81fb79d136 100755 --- a/gyp/pylib/gyp/flock_tool.py +++ b/gyp/pylib/gyp/flock_tool.py @@ -39,7 +39,7 @@ def ExecFlock(self, lockfile, *cmd_list): # where fcntl.flock(fd, LOCK_EX) always fails # with EBADF, that's why we use this F_SETLK # hack instead. - fd = os.open(lockfile, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT, 0666) + fd = os.open(lockfile, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT, 0o666) if sys.platform.startswith('aix'): # Python on AIX is compiled with LARGEFILE support, which changes the # struct size. diff --git a/gyp/pylib/gyp/generator/analyzer.py b/gyp/pylib/gyp/generator/analyzer.py index 921c1a6b71..dc17c96524 100644 --- a/gyp/pylib/gyp/generator/analyzer.py +++ b/gyp/pylib/gyp/generator/analyzer.py @@ -62,6 +62,8 @@ then the "all" target includes "b1" and "b2". """ +from __future__ import print_function + import gyp.common import gyp.ninja_syntax as ninja_syntax import json @@ -155,7 +157,7 @@ def _AddSources(sources, base_path, base_path_components, result): continue result.append(base_path + source) if debug: - print 'AddSource', org_source, result[len(result) - 1] + print('AddSource', org_source, result[len(result) - 1]) def _ExtractSourcesFromAction(action, base_path, base_path_components, @@ -185,7 +187,7 @@ def _ExtractSources(target, target_dict, toplevel_dir): base_path += '/' if debug: - print 'ExtractSources', target, base_path + print('ExtractSources', target, base_path) results = [] if 'sources' in target_dict: @@ -278,7 +280,7 @@ def _WasBuildFileModified(build_file, data, files, toplevel_dir): the root of the source tree.""" if _ToLocalPath(toplevel_dir, _ToGypPath(build_file)) in files: if debug: - print 'gyp file modified', build_file + print('gyp file modified', build_file) return True # First element of included_files is the file itself. @@ -291,8 +293,8 @@ def _WasBuildFileModified(build_file, data, files, toplevel_dir): _ToGypPath(gyp.common.UnrelativePath(include_file, build_file)) if _ToLocalPath(toplevel_dir, rel_include_file) in files: if debug: - print 'included gyp file modified, gyp_file=', build_file, \ - 'included file=', rel_include_file + print('included gyp file modified, gyp_file=', build_file, + 'included file=', rel_include_file) return True return False @@ -373,7 +375,7 @@ def _GenerateTargets(data, target_list, target_dicts, toplevel_dir, files, # If a build file (or any of its included files) is modified we assume all # targets in the file are modified. if build_file_in_files[build_file]: - print 'matching target from modified build file', target_name + print('matching target from modified build file', target_name) target.match_status = MATCH_STATUS_MATCHES matching_targets.append(target) else: @@ -381,7 +383,7 @@ def _GenerateTargets(data, target_list, target_dicts, toplevel_dir, files, toplevel_dir) for source in sources: if _ToGypPath(os.path.normpath(source)) in files: - print 'target', target_name, 'matches', source + print('target', target_name, 'matches', source) target.match_status = MATCH_STATUS_MATCHES matching_targets.append(target) break @@ -433,7 +435,7 @@ def _DoesTargetDependOnMatchingTargets(target): for dep in target.deps: if _DoesTargetDependOnMatchingTargets(dep): target.match_status = MATCH_STATUS_MATCHES_BY_DEPENDENCY - print '\t', target.name, 'matches by dep', dep.name + print('\t', target.name, 'matches by dep', dep.name) return True target.match_status = MATCH_STATUS_DOESNT_MATCH return False @@ -445,7 +447,7 @@ def _GetTargetsDependingOnMatchingTargets(possible_targets): supplied as input to analyzer. possible_targets: targets to search from.""" found = [] - print 'Targets that matched by dependency:' + print('Targets that matched by dependency:') for target in possible_targets: if _DoesTargetDependOnMatchingTargets(target): found.append(target) @@ -484,12 +486,12 @@ def _AddCompileTargets(target, roots, add_if_no_ancestor, result): (add_if_no_ancestor or target.requires_build)) or (target.is_static_library and add_if_no_ancestor and not target.is_or_has_linked_ancestor)): - print '\t\tadding to compile targets', target.name, 'executable', \ - target.is_executable, 'added_to_compile_targets', \ - target.added_to_compile_targets, 'add_if_no_ancestor', \ - add_if_no_ancestor, 'requires_build', target.requires_build, \ - 'is_static_library', target.is_static_library, \ - 'is_or_has_linked_ancestor', target.is_or_has_linked_ancestor + print('\t\tadding to compile targets', target.name, 'executable', + target.is_executable, 'added_to_compile_targets', + target.added_to_compile_targets, 'add_if_no_ancestor', + add_if_no_ancestor, 'requires_build', target.requires_build, + 'is_static_library', target.is_static_library, + 'is_or_has_linked_ancestor', target.is_or_has_linked_ancestor) result.add(target) target.added_to_compile_targets = True @@ -500,7 +502,7 @@ def _GetCompileTargets(matching_targets, supplied_targets): supplied_targets: set of targets supplied to analyzer to search from.""" result = set() for target in matching_targets: - print 'finding compile targets for match', target.name + print('finding compile targets for match', target.name) _AddCompileTargets(target, supplied_targets, True, result) return result @@ -508,46 +510,46 @@ def _GetCompileTargets(matching_targets, supplied_targets): def _WriteOutput(params, **values): """Writes the output, either to stdout or a file is specified.""" if 'error' in values: - print 'Error:', values['error'] + print('Error:', values['error']) if 'status' in values: - print values['status'] + print(values['status']) if 'targets' in values: values['targets'].sort() - print 'Supplied targets that depend on changed files:' + print('Supplied targets that depend on changed files:') for target in values['targets']: - print '\t', target + print('\t', target) if 'invalid_targets' in values: values['invalid_targets'].sort() - print 'The following targets were not found:' + print('The following targets were not found:') for target in values['invalid_targets']: - print '\t', target + print('\t', target) if 'build_targets' in values: values['build_targets'].sort() - print 'Targets that require a build:' + print('Targets that require a build:') for target in values['build_targets']: - print '\t', target + print('\t', target) if 'compile_targets' in values: values['compile_targets'].sort() - print 'Targets that need to be built:' + print('Targets that need to be built:') for target in values['compile_targets']: - print '\t', target + print('\t', target) if 'test_targets' in values: values['test_targets'].sort() - print 'Test targets:' + print('Test targets:') for target in values['test_targets']: - print '\t', target + print('\t', target) output_path = params.get('generator_flags', {}).get( 'analyzer_output_path', None) if not output_path: - print json.dumps(values) + print(json.dumps(values)) return try: f = open(output_path, 'w') f.write(json.dumps(values) + '\n') f.close() except IOError as e: - print 'Error writing to output file', output_path, str(e) + print('Error writing to output file', output_path, str(e)) def _WasGypIncludeFileModified(params, files): @@ -556,7 +558,7 @@ def _WasGypIncludeFileModified(params, files): if params['options'].includes: for include in params['options'].includes: if _ToGypPath(os.path.normpath(include)) in files: - print 'Include file modified, assuming all changed', include + print('Include file modified, assuming all changed', include) return True return False @@ -638,13 +640,13 @@ def find_matching_test_target_names(self): set(self._root_targets))] else: test_targets = [x for x in test_targets_no_all] - print 'supplied test_targets' + print('supplied test_targets') for target_name in self._test_target_names: - print '\t', target_name - print 'found test_targets' + print('\t', target_name) + print('found test_targets') for target in test_targets: - print '\t', target.name - print 'searching for matching test targets' + print('\t', target.name) + print('searching for matching test targets') matching_test_targets = _GetTargetsDependingOnMatchingTargets(test_targets) matching_test_targets_contains_all = (test_target_names_contains_all and set(matching_test_targets) & @@ -654,14 +656,14 @@ def find_matching_test_target_names(self): # 'all' is subsequentely added to the matching names below. matching_test_targets = [x for x in (set(matching_test_targets) & set(test_targets_no_all))] - print 'matched test_targets' + print('matched test_targets') for target in matching_test_targets: - print '\t', target.name + print('\t', target.name) matching_target_names = [gyp.common.ParseQualifiedTarget(target.name)[1] for target in matching_test_targets] if matching_test_targets_contains_all: matching_target_names.append('all') - print '\tall' + print('\tall') return matching_target_names def find_matching_compile_target_names(self): @@ -677,10 +679,10 @@ def find_matching_compile_target_names(self): if 'all' in self._supplied_target_names(): supplied_targets = [x for x in (set(supplied_targets) | set(self._root_targets))] - print 'Supplied test_targets & compile_targets' + print('Supplied test_targets & compile_targets') for target in supplied_targets: - print '\t', target.name - print 'Finding compile targets' + print('\t', target.name) + print('Finding compile targets') compile_targets = _GetCompileTargets(self._changed_targets, supplied_targets) return [gyp.common.ParseQualifiedTarget(target.name)[1] @@ -699,7 +701,7 @@ def GenerateOutput(target_list, target_dicts, data, params): toplevel_dir = _ToGypPath(os.path.abspath(params['options'].toplevel_dir)) if debug: - print 'toplevel_dir', toplevel_dir + print('toplevel_dir', toplevel_dir) if _WasGypIncludeFileModified(params, config.files): result_dict = { 'status': all_changed_string, diff --git a/gyp/pylib/gyp/generator/android.py b/gyp/pylib/gyp/generator/android.py index 5b26cc785a..b7f9842888 100644 --- a/gyp/pylib/gyp/generator/android.py +++ b/gyp/pylib/gyp/generator/android.py @@ -14,6 +14,8 @@ # variables set potentially clash with other Android build system variables. # Try to avoid setting global variables where possible. +from __future__ import print_function + import gyp import gyp.common import gyp.generator.make as make # Reuse global functions from make backend. @@ -250,7 +252,7 @@ def WriteActions(self, actions, extra_sources, extra_outputs): dirs = set() for out in outputs: if not out.startswith('$'): - print ('WARNING: Action for target "%s" writes output to local path ' + print('WARNING: Action for target "%s" writes output to local path ' '"%s".' % (self.target, out)) dir = os.path.split(out)[0] if dir: @@ -355,7 +357,7 @@ def WriteRules(self, rules, extra_sources, extra_outputs): dirs = set() for out in outputs: if not out.startswith('$'): - print ('WARNING: Rule for target %s writes output to local path %s' + print('WARNING: Rule for target %s writes output to local path %s' % (self.target, out)) dir = os.path.dirname(out) if dir: @@ -429,7 +431,7 @@ def WriteCopies(self, copies, extra_outputs): # $(gyp_shared_intermediate_dir). Note that we can't use an assertion # because some of the gyp tests depend on this. if not copy['destination'].startswith('$'): - print ('WARNING: Copy rule for target %s writes output to ' + print('WARNING: Copy rule for target %s writes output to ' 'local path %s' % (self.target, copy['destination'])) # LocalPathify() calls normpath, stripping trailing slashes. @@ -458,7 +460,7 @@ def WriteSourceFlags(self, spec, configs): Args: spec, configs: input from gyp. """ - for configname, config in sorted(configs.iteritems()): + for configname, config in sorted(configs.items()): extracted_includes = [] self.WriteLn('\n# Flags passed to both C and C++ files.') @@ -636,7 +638,7 @@ def ComputeOutputParts(self, spec): elif self.type == 'none': target_ext = '.stamp' elif self.type != 'executable': - print ("ERROR: What output file should be generated?", + print("ERROR: What output file should be generated?", "type", self.type, "target", target) if self.type != 'static_library' and self.type != 'shared_library': @@ -788,7 +790,7 @@ def WriteTargetFlags(self, spec, configs, link_deps): static_libs, dynamic_libs, ldflags_libs = self.FilterLibraries(libraries) if self.type != 'static_library': - for configname, config in sorted(configs.iteritems()): + for configname, config in sorted(configs.items()): ldflags = list(config.get('ldflags', [])) self.WriteLn('') self.WriteList(ldflags, 'LOCAL_LDFLAGS_%s' % configname) @@ -837,7 +839,7 @@ def WriteTarget(self, spec, configs, deps, link_deps, part_of_all, settings = spec.get('aosp_build_settings', {}) if settings: self.WriteLn('### Set directly by aosp_build_settings.') - for k, v in settings.iteritems(): + for k, v in settings.items(): if isinstance(v, list): self.WriteList(v, k) else: @@ -956,7 +958,7 @@ def PerformBuild(data, configurations, params): env = dict(os.environ) env['ONE_SHOT_MAKEFILE'] = makefile arguments = ['make', '-C', os.environ['ANDROID_BUILD_TOP'], 'gyp_all_modules'] - print 'Building: %s' % arguments + print('Building: %s' % arguments) subprocess.check_call(arguments, env=env) @@ -1065,7 +1067,7 @@ def CalculateMakefilePath(build_file, base_name): write_alias_target=write_alias_targets, sdk_version=sdk_version) if android_module in android_modules: - print ('ERROR: Android module names must be unique. The following ' + print('ERROR: Android module names must be unique. The following ' 'targets both generate Android module name %s.\n %s\n %s' % (android_module, android_modules[android_module], qualified_target)) diff --git a/gyp/pylib/gyp/generator/cmake.py b/gyp/pylib/gyp/generator/cmake.py index 17f5e6396c..7aabddb633 100644 --- a/gyp/pylib/gyp/generator/cmake.py +++ b/gyp/pylib/gyp/generator/cmake.py @@ -28,6 +28,8 @@ CMakeLists.txt file. """ +from __future__ import print_function + import multiprocessing import os import signal @@ -639,8 +641,8 @@ def WriteTarget(namer, qualified_target, target_dicts, build_dir, config_to_use, cmake_target_type = cmake_target_type_from_gyp_target_type.get(target_type) if cmake_target_type is None: - print ('Target %s has unknown target type %s, skipping.' % - ( target_name, target_type ) ) + print('Target %s has unknown target type %s, skipping.' % + ( target_name, target_type )) return SetVariable(output, 'TARGET', target_name) @@ -863,7 +865,7 @@ def WriteTarget(namer, qualified_target, target_dicts, build_dir, config_to_use, default_product_ext = generator_default_variables['SHARED_LIB_SUFFIX'] elif target_type != 'executable': - print ('ERROR: What output file should be generated?', + print('ERROR: What output file should be generated?', 'type', target_type, 'target', target_name) product_prefix = spec.get('product_prefix', default_product_prefix) @@ -1180,11 +1182,11 @@ def PerformBuild(data, configurations, params): output_dir, config_name)) arguments = ['cmake', '-G', 'Ninja'] - print 'Generating [%s]: %s' % (config_name, arguments) + print('Generating [%s]: %s' % (config_name, arguments)) subprocess.check_call(arguments, cwd=build_dir) arguments = ['ninja', '-C', build_dir] - print 'Building [%s]: %s' % (config_name, arguments) + print('Building [%s]: %s' % (config_name, arguments)) subprocess.check_call(arguments) @@ -1212,7 +1214,7 @@ def GenerateOutput(target_list, target_dicts, data, params): arglists.append((target_list, target_dicts, data, params, config_name)) pool.map(CallGenerateOutputForConfig, arglists) - except KeyboardInterrupt, e: + except KeyboardInterrupt as e: pool.terminate() raise e else: diff --git a/gyp/pylib/gyp/generator/dump_dependency_json.py b/gyp/pylib/gyp/generator/dump_dependency_json.py index 160eafe2ef..8e4f3168f3 100644 --- a/gyp/pylib/gyp/generator/dump_dependency_json.py +++ b/gyp/pylib/gyp/generator/dump_dependency_json.py @@ -1,3 +1,4 @@ +from __future__ import print_function # Copyright (c) 2012 Google Inc. All rights reserved. # Use of this source code is governed by a BSD-style license that can be # found in the LICENSE file. @@ -96,4 +97,4 @@ def GenerateOutput(target_list, target_dicts, data, params): f = open(filename, 'w') json.dump(edges, f) f.close() - print 'Wrote json to %s.' % filename + print('Wrote json to %s.' % filename) diff --git a/gyp/pylib/gyp/generator/eclipse.py b/gyp/pylib/gyp/generator/eclipse.py index 3544347b3b..b7c6aa951f 100644 --- a/gyp/pylib/gyp/generator/eclipse.py +++ b/gyp/pylib/gyp/generator/eclipse.py @@ -141,7 +141,7 @@ def GetAllIncludeDirectories(target_list, target_dicts, compiler_includes_list.append(include_dir) # Find standard gyp include dirs. - if config.has_key('include_dirs'): + if 'include_dirs' in config: include_dirs = config['include_dirs'] for shared_intermediate_dir in shared_intermediate_dirs: for include_dir in include_dirs: diff --git a/gyp/pylib/gyp/generator/gypd.py b/gyp/pylib/gyp/generator/gypd.py index 3efdb9966a..78eeaa61b2 100644 --- a/gyp/pylib/gyp/generator/gypd.py +++ b/gyp/pylib/gyp/generator/gypd.py @@ -88,7 +88,7 @@ def GenerateOutput(target_list, target_dicts, data, params): if not output_file in output_files: output_files[output_file] = input_file - for output_file, input_file in output_files.iteritems(): + for output_file, input_file in output_files.items(): output = open(output_file, 'w') pprint.pprint(data[input_file], output) output.close() diff --git a/gyp/pylib/gyp/generator/make.py b/gyp/pylib/gyp/generator/make.py index 64b9dd267b..d549e899d8 100644 --- a/gyp/pylib/gyp/generator/make.py +++ b/gyp/pylib/gyp/generator/make.py @@ -19,7 +19,9 @@ # # Global settings and utility functions are currently stuffed in the # toplevel Makefile. It may make sense to generate some .mk files on -# the side to keep the the files readable. +# the side to keep the files readable. + +from __future__ import print_function import os import re @@ -31,6 +33,8 @@ from gyp.common import GetEnvironFallback from gyp.common import GypError +import hashlib + generator_default_variables = { 'EXECUTABLE_PREFIX': '', 'EXECUTABLE_SUFFIX': '', @@ -90,7 +94,10 @@ def CalculateVariables(default_variables, params): if flavor == 'android': operating_system = 'linux' # Keep this legacy behavior for now. default_variables.setdefault('OS', operating_system) - default_variables.setdefault('SHARED_LIB_SUFFIX', '.so') + if flavor == 'aix': + default_variables.setdefault('SHARED_LIB_SUFFIX', '.a') + else: + default_variables.setdefault('SHARED_LIB_SUFFIX', '.so') default_variables.setdefault('SHARED_LIB_DIR','$(builddir)/lib.$(TOOLSET)') default_variables.setdefault('LIB_DIR', '$(obj).$(TOOLSET)') @@ -142,7 +149,7 @@ def CalculateGeneratorInputInfo(params): # special "figure out circular dependencies" flags around the entire # input list during linking. quiet_cmd_link = LINK($(TOOLSET)) $@ -cmd_link = $(LINK.$(TOOLSET)) $(GYP_LDFLAGS) $(LDFLAGS.$(TOOLSET)) -o $@ -Wl,--start-group $(LD_INPUTS) -Wl,--end-group $(LIBS) +cmd_link = $(LINK.$(TOOLSET)) $(GYP_LDFLAGS) $(LDFLAGS.$(TOOLSET)) -o $@ -Wl,--start-group $(LD_INPUTS) $(LIBS) -Wl,--end-group # We support two kinds of shared objects (.so): # 1) shared_library, which is just bundling together many dependent libraries @@ -227,6 +234,25 @@ def CalculateGeneratorInputInfo(params): """ +LINK_COMMANDS_OS390 = """\ +quiet_cmd_alink = AR($(TOOLSET)) $@ +cmd_alink = rm -f $@ && $(AR.$(TOOLSET)) crs $@ $(filter %.o,$^) + +quiet_cmd_alink_thin = AR($(TOOLSET)) $@ +cmd_alink_thin = rm -f $@ && $(AR.$(TOOLSET)) crsT $@ $(filter %.o,$^) + +quiet_cmd_link = LINK($(TOOLSET)) $@ +cmd_link = $(LINK.$(TOOLSET)) $(GYP_LDFLAGS) $(LDFLAGS.$(TOOLSET)) -o $@ $(LD_INPUTS) $(LIBS) + +quiet_cmd_solink = SOLINK($(TOOLSET)) $@ +cmd_solink = $(LINK.$(TOOLSET)) $(GYP_LDFLAGS) $(LDFLAGS.$(TOOLSET)) -o $@ $(LD_INPUTS) $(LIBS) -Wl,DLL + +quiet_cmd_solink_module = SOLINK_MODULE($(TOOLSET)) $@ +cmd_solink_module = $(LINK.$(TOOLSET)) $(GYP_LDFLAGS) $(LDFLAGS.$(TOOLSET)) -o $@ $(filter-out FORCE_DO_CMD, $^) $(LIBS) -Wl,DLL + +""" + + # Header of toplevel Makefile. # This should go into the build tree, but it's easier to keep it here for now. SHARED_HEADER = ("""\ @@ -310,7 +336,7 @@ def CalculateGeneratorInputInfo(params): # We write to a dep file on the side first and then rename at the end # so we can't end up with a broken dep file. depfile = $(depsdir)/$(call replace_spaces,$@).d -DEPFLAGS = -MMD -MF $(depfile).raw +DEPFLAGS = %(makedep_args)s -MF $(depfile).raw # We have to fixup the deps output in a few ways. # (1) the file output should mention the proper .o file. @@ -623,6 +649,9 @@ def Sourceify(path): def QuoteSpaces(s, quote=r'\ '): return s.replace(' ', quote) +def SourceifyAndQuoteSpaces(path): + """Convert a path to its source directory form and quote spaces.""" + return QuoteSpaces(Sourceify(path)) # TODO: Avoid code duplication with _ValidateSourcesForMSVSProject in msvs.py. def _ValidateSourcesForOSX(spec, all_sources): @@ -645,14 +674,13 @@ def _ValidateSourcesForOSX(spec, all_sources): basenames.setdefault(basename, []).append(source) error = '' - for basename, files in basenames.iteritems(): + for basename, files in basenames.items(): if len(files) > 1: error += ' %s: %s\n' % (basename, ' '.join(files)) if error: - print('static library %s has several files with the same basename:\n' % - spec['target_name'] + error + 'libtool on OS X will generate' + - ' warnings for them.') + print(('static library %s has several files with the same basename:\n' % spec['target_name']) + + error + 'libtool on OS X will generate' + ' warnings for them.') raise GypError('Duplicate basenames in sources section, see list above') @@ -1347,11 +1375,14 @@ def ComputeOutputBasename(self, spec): if target[:3] == 'lib': target = target[3:] target_prefix = 'lib' - target_ext = '.so' + if self.flavor == 'aix': + target_ext = '.a' + else: + target_ext = '.so' elif self.type == 'none': target = '%s.stamp' % target elif self.type != 'executable': - print ("ERROR: What output file should be generated?", + print("ERROR: What output file should be generated?", "type", self.type, "target", target) target_prefix = spec.get('product_prefix', target_prefix) @@ -1516,7 +1547,7 @@ def WriteTarget(self, spec, configs, deps, link_deps, bundle_deps, # Postbuilds expect to be run in the gyp file's directory, so insert an # implicit postbuild to cd to there. postbuilds.insert(0, gyp.common.EncodePOSIXShellList(['cd', self.path])) - for i in xrange(len(postbuilds)): + for i in range(len(postbuilds)): if not postbuilds[i].startswith('$'): postbuilds[i] = EscapeShellArgument(postbuilds[i]) self.WriteLn('%s: builddir := $(abs_builddir)' % QuoteSpaces(self.output)) @@ -1608,7 +1639,7 @@ def WriteTarget(self, spec, configs, deps, link_deps, bundle_deps, self.WriteDoCmd([self.output_binary], deps, 'touch', part_of_all, postbuilds=postbuilds) else: - print "WARNING: no output for", self.type, target + print("WARNING: no output for", self.type, self.target) # Add an alias for each target (if there are any outputs). # Installable target aliases are created below. @@ -1743,7 +1774,10 @@ def WriteMakeRule(self, outputs, inputs, actions=None, comment=None, # actual command. # - The intermediate recipe will 'touch' the intermediate file. # - The multi-output rule will have an do-nothing recipe. - intermediate = "%s.intermediate" % (command if command else self.target) + + # Hash the target name to avoid generating overlong filenames. + cmddigest = hashlib.sha1(command if command else self.target).hexdigest() + intermediate = "%s.intermediate" % cmddigest self.WriteLn('%s: %s' % (' '.join(outputs), intermediate)) self.WriteLn('\t%s' % '@:'); self.WriteLn('%s: %s' % ('.INTERMEDIATE', intermediate)) @@ -1945,7 +1979,7 @@ def WriteAutoRegenerationRule(params, root_makefile, makefile_name, "%(makefile_name)s: %(deps)s\n" "\t$(call do_cmd,regen_makefile)\n\n" % { 'makefile_name': makefile_name, - 'deps': ' '.join(map(Sourceify, build_files)), + 'deps': ' '.join(map(SourceifyAndQuoteSpaces, build_files)), 'cmd': gyp.common.EncodePOSIXShellList( [gyp_binary, '-fmake'] + gyp.RegenerateFlags(options) + @@ -1959,7 +1993,7 @@ def PerformBuild(data, configurations, params): if options.toplevel_dir and options.toplevel_dir != '.': arguments += '-C', options.toplevel_dir arguments.append('BUILDTYPE=' + config) - print 'Building [%s]: %s' % (config, arguments) + print('Building [%s]: %s' % (config, arguments)) subprocess.check_call(arguments) @@ -2013,6 +2047,7 @@ def CalculateMakefilePath(build_file, base_name): flock_command= 'flock' copy_archive_arguments = '-af' + makedep_arguments = '-MMD' header_params = { 'default_target': default_target, 'builddir': builddir_name, @@ -2023,6 +2058,7 @@ def CalculateMakefilePath(build_file, base_name): 'extra_commands': '', 'srcdir': srcdir, 'copy_archive_args': copy_archive_arguments, + 'makedep_args': makedep_arguments, } if flavor == 'mac': flock_command = './gyp-mac-tool flock' @@ -2036,6 +2072,14 @@ def CalculateMakefilePath(build_file, base_name): header_params.update({ 'link_commands': LINK_COMMANDS_ANDROID, }) + elif flavor == 'zos': + copy_archive_arguments = '-fPR' + makedep_arguments = '-qmakedep=gcc' + header_params.update({ + 'copy_archive_args': copy_archive_arguments, + 'makedep_args': makedep_arguments, + 'link_commands': LINK_COMMANDS_OS390, + }) elif flavor == 'solaris': header_params.update({ 'flock': './gyp-flock-tool flock', diff --git a/gyp/pylib/gyp/generator/msvs.py b/gyp/pylib/gyp/generator/msvs.py index 6bfad0f3bd..fb2549d025 100644 --- a/gyp/pylib/gyp/generator/msvs.py +++ b/gyp/pylib/gyp/generator/msvs.py @@ -2,6 +2,8 @@ # Use of this source code is governed by a BSD-style license that can be # found in the LICENSE file. +from __future__ import print_function + import copy import ntpath import os @@ -449,7 +451,7 @@ def _AddCustomBuildToolForMSVS(p, spec, primary_input, 'CommandLine': cmd, }) # Add to the properties of primary input for each config. - for config_name, c_data in spec['configurations'].iteritems(): + for config_name, c_data in spec['configurations'].items(): p.AddFileConfig(_FixPath(primary_input), _ConfigFullName(config_name, c_data), tools=[tool]) @@ -755,8 +757,8 @@ def _Replace(match): # the VCProj but cause the same problem on the final command-line. Moving # the item to the end of the list does works, but that's only possible if # there's only one such item. Let's just warn the user. - print >> sys.stderr, ('Warning: MSVS may misinterpret the odd number of ' + - 'quotes in ' + s) + print('Warning: MSVS may misinterpret the odd number of ' + + 'quotes in ' + s, file=sys.stderr) return s @@ -969,13 +971,13 @@ def _ValidateSourcesForMSVSProject(spec, version): basenames.setdefault(basename, []).append(source) error = '' - for basename, files in basenames.iteritems(): + for basename, files in basenames.items(): if len(files) > 1: error += ' %s: %s\n' % (basename, ' '.join(files)) if error: - print('static library %s has several files with the same basename:\n' % - spec['target_name'] + error + 'MSVC08 cannot handle that.') + print('static library %s has several files with the same basename:\n' % spec['target_name'] + + error + 'MSVC08 cannot handle that.') raise GypError('Duplicate basenames in sources section, see list above') @@ -1001,7 +1003,7 @@ def _GenerateMSVSProject(project, options, version, generator_flags): relative_path_of_gyp_file = gyp.common.RelativePath(gyp_path, project_dir) config_type = _GetMSVSConfigurationType(spec, project.build_file) - for config_name, config in spec['configurations'].iteritems(): + for config_name, config in spec['configurations'].items(): _AddConfigurationToMSVSProject(p, spec, config_type, config_name, config) # MSVC08 and prior version cannot handle duplicate basenames in the same @@ -1367,10 +1369,10 @@ def _ConvertToolsToExpectedForm(tools): A list of Tool objects. """ tool_list = [] - for tool, settings in tools.iteritems(): + for tool, settings in tools.items(): # Collapse settings with lists. settings_fixed = {} - for setting, value in settings.iteritems(): + for setting, value in settings.items(): if type(value) == list: if ((tool == 'VCLinkerTool' and setting == 'AdditionalDependencies') or @@ -1545,7 +1547,7 @@ def _IdlFilesHandledNonNatively(spec, sources): def _GetPrecompileRelatedFiles(spec): # Gather a list of precompiled header related sources. precompiled_related = [] - for _, config in spec['configurations'].iteritems(): + for _, config in spec['configurations'].items(): for k in precomp_keys: f = config.get(k) if f: @@ -1556,7 +1558,7 @@ def _GetPrecompileRelatedFiles(spec): def _ExcludeFilesFromBeingBuilt(p, spec, excluded_sources, excluded_idl, list_excluded): exclusions = _GetExcludedFilesFromBuild(spec, excluded_sources, excluded_idl) - for file_name, excluded_configs in exclusions.iteritems(): + for file_name, excluded_configs in exclusions.items(): if (not list_excluded and len(excluded_configs) == len(spec['configurations'])): # If we're not listing excluded files, then they won't appear in the @@ -1573,7 +1575,7 @@ def _GetExcludedFilesFromBuild(spec, excluded_sources, excluded_idl): # Exclude excluded sources from being built. for f in excluded_sources: excluded_configs = [] - for config_name, config in spec['configurations'].iteritems(): + for config_name, config in spec['configurations'].items(): precomped = [_FixPath(config.get(i, '')) for i in precomp_keys] # Don't do this for ones that are precompiled header related. if f not in precomped: @@ -1583,7 +1585,7 @@ def _GetExcludedFilesFromBuild(spec, excluded_sources, excluded_idl): # Exclude them now. for f in excluded_idl: excluded_configs = [] - for config_name, config in spec['configurations'].iteritems(): + for config_name, config in spec['configurations'].items(): excluded_configs.append((config_name, config)) exclusions[f] = excluded_configs return exclusions @@ -1592,7 +1594,7 @@ def _GetExcludedFilesFromBuild(spec, excluded_sources, excluded_idl): def _AddToolFilesToMSVS(p, spec): # Add in tool files (rules). tool_files = OrderedSet() - for _, config in spec['configurations'].iteritems(): + for _, config in spec['configurations'].items(): for f in config.get('msvs_tool_files', []): tool_files.add(f) for f in tool_files: @@ -1605,7 +1607,7 @@ def _HandlePreCompiledHeaders(p, sources, spec): # kind (i.e. C vs. C++) as the precompiled header source stub needs # to have use of precompiled headers disabled. extensions_excluded_from_precompile = [] - for config_name, config in spec['configurations'].iteritems(): + for config_name, config in spec['configurations'].items(): source = config.get('msvs_precompiled_source') if source: source = _FixPath(source) @@ -1626,7 +1628,7 @@ def DisableForSourceTree(source_tree): else: basename, extension = os.path.splitext(source) if extension in extensions_excluded_from_precompile: - for config_name, config in spec['configurations'].iteritems(): + for config_name, config in spec['configurations'].items(): tool = MSVSProject.Tool('VCCLCompilerTool', {'UsePrecompiledHeader': '0', 'ForcedIncludeFiles': '$(NOINHERIT)'}) @@ -1677,7 +1679,7 @@ def _WriteMSVSUserFile(project_path, version, spec): return # Nothing to add # Write out the user file. user_file = _CreateMSVSUserFile(project_path, version, spec) - for config_name, c_data in spec['configurations'].iteritems(): + for config_name, c_data in spec['configurations'].items(): user_file.AddDebugSettings(_ConfigFullName(config_name, c_data), action, environment, working_directory) user_file.WriteIfChanged() @@ -1728,7 +1730,7 @@ def _GetPathDict(root, path): def _DictsToFolders(base_path, bucket, flat): # Convert to folders recursively. children = [] - for folder, contents in bucket.iteritems(): + for folder, contents in bucket.items(): if type(contents) == dict: folder_children = _DictsToFolders(os.path.join(base_path, folder), contents, flat) @@ -1800,7 +1802,7 @@ def _GetPlatformOverridesOfProject(spec): # Prepare a dict indicating which project configurations are used for which # solution configurations for this target. config_platform_overrides = {} - for config_name, c in spec['configurations'].iteritems(): + for config_name, c in spec['configurations'].items(): config_fullname = _ConfigFullName(config_name, c) platform = c.get('msvs_target_platform', _ConfigPlatform(c)) fixed_config_fullname = '%s|%s' % ( @@ -1939,7 +1941,7 @@ def PerformBuild(data, configurations, params): msvs_version = params['msvs_version'] devenv = os.path.join(msvs_version.path, 'Common7', 'IDE', 'devenv.com') - for build_file, build_file_dict in data.iteritems(): + for build_file, build_file_dict in data.items(): (build_file_root, build_file_ext) = os.path.splitext(build_file) if build_file_ext != '.gyp': continue @@ -1949,7 +1951,7 @@ def PerformBuild(data, configurations, params): for config in configurations: arguments = [devenv, sln_path, '/Build', config] - print 'Building [%s]: %s' % (config, arguments) + print('Building [%s]: %s' % (config, arguments)) rtn = subprocess.check_call(arguments) @@ -1988,7 +1990,7 @@ def GenerateOutput(target_list, target_dicts, data, params): configs = set() for qualified_target in target_list: spec = target_dicts[qualified_target] - for config_name, config in spec['configurations'].iteritems(): + for config_name, config in spec['configurations'].items(): configs.add(_ConfigFullName(config_name, config)) configs = list(configs) @@ -2031,7 +2033,7 @@ def GenerateOutput(target_list, target_dicts, data, params): if generator_flags.get('msvs_error_on_missing_sources', False): raise GypError(error_message) else: - print >> sys.stdout, "Warning: " + error_message + print("Warning: " + error_message, file=sys.stdout) def _GenerateMSBuildFiltersFile(filters_path, source_files, @@ -2118,7 +2120,7 @@ def _MapFileToMsBuildSourceType(source, rule_dependencies, if ext in extension_to_rule_name: group = 'rule' element = extension_to_rule_name[ext] - elif ext in ['.cc', '.cpp', '.c', '.cxx']: + elif ext in ['.cc', '.cpp', '.c', '.cxx', '.mm']: group = 'compile' element = 'ClCompile' elif ext in ['.h', '.hxx']: @@ -2628,7 +2630,7 @@ def _GetConfigurationCondition(name, settings): def _GetMSBuildProjectConfigurations(configurations): group = ['ItemGroup', {'Label': 'ProjectConfigurations'}] - for (name, settings) in sorted(configurations.iteritems()): + for (name, settings) in sorted(configurations.items()): configuration, platform = _GetConfigurationAndPlatform(name, settings) designation = '%s|%s' % (configuration, platform) group.append( @@ -2698,7 +2700,7 @@ def _GetMSBuildGlobalProperties(spec, guid, gyp_file_name): def _GetMSBuildConfigurationDetails(spec, build_file): properties = {} - for name, settings in spec['configurations'].iteritems(): + for name, settings in spec['configurations'].items(): msbuild_attributes = _GetMSBuildAttributes(spec, settings, build_file) condition = _GetConfigurationCondition(name, settings) character_set = msbuild_attributes.get('CharacterSet') @@ -2727,9 +2729,9 @@ def _GetMSBuildPropertySheets(configurations): user_props = r'$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props' additional_props = {} props_specified = False - for name, settings in sorted(configurations.iteritems()): + for name, settings in sorted(configurations.items()): configuration = _GetConfigurationCondition(name, settings) - if settings.has_key('msbuild_props'): + if 'msbuild_props' in settings: additional_props[configuration] = _FixPaths(settings['msbuild_props']) props_specified = True else: @@ -2749,7 +2751,7 @@ def _GetMSBuildPropertySheets(configurations): ] else: sheets = [] - for condition, props in additional_props.iteritems(): + for condition, props in additional_props.items(): import_group = [ 'ImportGroup', {'Label': 'PropertySheets', @@ -2782,7 +2784,7 @@ def _ConvertMSVSBuildAttributes(spec, config, build_file): elif a == 'ConfigurationType': msbuild_attributes[a] = _ConvertMSVSConfigurationType(msvs_attributes[a]) else: - print 'Warning: Do not know how to convert MSVS attribute ' + a + print('Warning: Do not know how to convert MSVS attribute ' + a) return msbuild_attributes @@ -2876,7 +2878,7 @@ def _GetMSBuildConfigurationGlobalProperties(spec, configurations, build_file): new_paths = '$(ExecutablePath);' + ';'.join(new_paths) properties = {} - for (name, configuration) in sorted(configurations.iteritems()): + for (name, configuration) in sorted(configurations.items()): condition = _GetConfigurationCondition(name, configuration) attributes = _GetMSBuildAttributes(spec, configuration, build_file) msbuild_settings = configuration['finalized_msbuild_settings'] @@ -2901,7 +2903,7 @@ def _GetMSBuildConfigurationGlobalProperties(spec, configurations, build_file): _AddConditionalProperty(properties, condition, 'ExecutablePath', new_paths) tool_settings = msbuild_settings.get('', {}) - for name, value in sorted(tool_settings.iteritems()): + for name, value in sorted(tool_settings.items()): formatted_value = _GetValueFormattedForMSBuild('', name, value) _AddConditionalProperty(properties, condition, name, formatted_value) return _GetMSBuildPropertyGroup(spec, None, properties) @@ -2970,7 +2972,7 @@ def GetEdges(node): # NOTE: reverse(topsort(DAG)) = topsort(reverse_edges(DAG)) for name in reversed(properties_ordered): values = properties[name] - for value, conditions in sorted(values.iteritems()): + for value, conditions in sorted(values.items()): if len(conditions) == num_configurations: # If the value is the same all configurations, # just add one unconditional entry. @@ -2983,18 +2985,18 @@ def GetEdges(node): def _GetMSBuildToolSettingsSections(spec, configurations): groups = [] - for (name, configuration) in sorted(configurations.iteritems()): + for (name, configuration) in sorted(configurations.items()): msbuild_settings = configuration['finalized_msbuild_settings'] group = ['ItemDefinitionGroup', {'Condition': _GetConfigurationCondition(name, configuration)} ] - for tool_name, tool_settings in sorted(msbuild_settings.iteritems()): + for tool_name, tool_settings in sorted(msbuild_settings.items()): # Skip the tool named '' which is a holder of global settings handled # by _GetMSBuildConfigurationGlobalProperties. if tool_name: if tool_settings: tool = [tool_name] - for name, value in sorted(tool_settings.iteritems()): + for name, value in sorted(tool_settings.items()): formatted_value = _GetValueFormattedForMSBuild(tool_name, name, value) tool.append([name, formatted_value]) @@ -3027,7 +3029,7 @@ def _FinalizeMSBuildSettings(spec, configuration): for ignored_setting in ignored_settings: value = configuration.get(ignored_setting) if value: - print ('Warning: The automatic conversion to MSBuild does not handle ' + print('Warning: The automatic conversion to MSBuild does not handle ' '%s. Ignoring setting of %s' % (ignored_setting, str(value))) defines = [_EscapeCppDefineForMSBuild(d) for d in defines] @@ -3194,7 +3196,7 @@ def _AddSources2(spec, sources, exclusions, grouped_sources, {'Condition': condition}, 'true']) # Add precompile if needed - for config_name, configuration in spec['configurations'].iteritems(): + for config_name, configuration in spec['configurations'].items(): precompiled_source = configuration.get('msvs_precompiled_source', '') if precompiled_source != '': precompiled_source = _FixPath(precompiled_source) @@ -3434,7 +3436,7 @@ def _GenerateActionsForMSBuild(spec, actions_to_add): """ sources_handled_by_action = OrderedSet() actions_spec = [] - for primary_input, actions in actions_to_add.iteritems(): + for primary_input, actions in actions_to_add.items(): inputs = OrderedSet() outputs = OrderedSet() descriptions = [] diff --git a/gyp/pylib/gyp/generator/ninja.py b/gyp/pylib/gyp/generator/ninja.py index 841067ed34..aae5b71310 100644 --- a/gyp/pylib/gyp/generator/ninja.py +++ b/gyp/pylib/gyp/generator/ninja.py @@ -1,3 +1,4 @@ +from __future__ import print_function # Copyright (c) 2013 Google Inc. All rights reserved. # Use of this source code is governed by a BSD-style license that can be # found in the LICENSE file. @@ -140,7 +141,7 @@ def __init__(self, type): # On Windows, incremental linking requires linking against all the .objs # that compose a .lib (rather than the .lib itself). That list is stored # here. In this case, we also need to save the compile_deps for the target, - # so that the the target that directly depends on the .objs can also depend + # so that the target that directly depends on the .objs can also depend # on those. self.component_objs = None self.compile_deps = None @@ -472,8 +473,8 @@ def WriteSpec(self, spec, config_name, generator_flags): if self.flavor != 'mac' or len(self.archs) == 1: link_deps += [self.GypPathToNinja(o) for o in obj_outputs] else: - print "Warning: Actions/rules writing object files don't work with " \ - "multiarch targets, dropping. (target %s)" % spec['target_name'] + print("Warning: Actions/rules writing object files don't work with " \ + "multiarch targets, dropping. (target %s)" % spec['target_name']) elif self.flavor == 'mac' and len(self.archs) > 1: link_deps = collections.defaultdict(list) @@ -796,7 +797,7 @@ def WriteMacXCassets(self, xcassets, bundle_depends): 'XCASSETS_LAUNCH_IMAGE': 'launch-image', } settings = self.xcode_settings.xcode_settings[self.config_name] - for settings_key, arg_name in settings_to_arg.iteritems(): + for settings_key, arg_name in settings_to_arg.items(): value = settings.get(settings_key) if value: extra_arguments[arg_name] = value @@ -1873,6 +1874,10 @@ def GenerateOutputForConfig(target_list, target_dicts, data, params, ld = os.path.join(build_to_root, value) if key == 'LD.host': ld_host = os.path.join(build_to_root, value) + if key == 'LDXX': + ldxx = os.path.join(build_to_root, value) + if key == 'LDXX.host': + ldxx_host = os.path.join(build_to_root, value) if key == 'NM': nm = os.path.join(build_to_root, value) if key == 'NM.host': @@ -1885,7 +1890,7 @@ def GenerateOutputForConfig(target_list, target_dicts, data, params, wrappers[key[:-len('_wrapper')]] = os.path.join(build_to_root, value) # Support wrappers from environment variables too. - for key, value in os.environ.iteritems(): + for key, value in os.environ.items(): if key.lower().endswith('_wrapper'): key_prefix = key[:-len('_wrapper')] key_prefix = re.sub(r'\.HOST$', '.host', key_prefix) @@ -1901,7 +1906,7 @@ def GenerateOutputForConfig(target_list, target_dicts, data, params, configs, generator_flags) cl_paths = gyp.msvs_emulation.GenerateEnvironmentFiles( toplevel_build, generator_flags, shared_system_includes, OpenOutput) - for arch, path in cl_paths.iteritems(): + for arch, path in cl_paths.items(): if clang_cl: # If we have selected clang-cl, use that instead. path = clang_cl @@ -1962,6 +1967,7 @@ def GenerateOutputForConfig(target_list, target_dicts, data, params, CommandWithWrapper('CXX.host', wrappers, cxx_host)) if flavor == 'win': master_ninja.variable('ld_host', ld_host) + master_ninja.variable('ldxx_host', ldxx_host) else: master_ninja.variable('ld_host', CommandWithWrapper( 'LINK', wrappers, ld_host)) @@ -2086,13 +2092,13 @@ def GenerateOutputForConfig(target_list, target_dicts, data, params, restat=True, command=mtime_preserving_solink_base % {'suffix': '@$link_file_list'}, rspfile='$link_file_list', - rspfile_content='-Wl,--start-group $in -Wl,--end-group $solibs $libs', + rspfile_content='-Wl,--start-group $in $solibs $libs -Wl,--end-group', pool='link_pool') master_ninja.rule( 'link', description='LINK $out', command=('$ld $ldflags -o $out ' - '-Wl,--start-group $in -Wl,--end-group $solibs $libs'), + '-Wl,--start-group $in $solibs $libs -Wl,--end-group'), pool='link_pool') elif flavor == 'win': master_ninja.rule( @@ -2238,15 +2244,22 @@ def GenerateOutputForConfig(target_list, target_dicts, data, params, 'stamp', description='STAMP $out', command='%s gyp-win-tool stamp $out' % sys.executable) - master_ninja.rule( - 'copy', - description='COPY $in $out', - command='%s gyp-win-tool recursive-mirror $in $out' % sys.executable) else: master_ninja.rule( 'stamp', description='STAMP $out', command='${postbuilds}touch $out') + if flavor == 'win': + master_ninja.rule( + 'copy', + description='COPY $in $out', + command='%s gyp-win-tool recursive-mirror $in $out' % sys.executable) + elif flavor == 'zos': + master_ninja.rule( + 'copy', + description='COPY $in $out', + command='rm -rf $out && cp -fRP $in $out') + else: master_ninja.rule( 'copy', description='COPY $in $out', @@ -2364,7 +2377,7 @@ def PerformBuild(data, configurations, params): for config in configurations: builddir = os.path.join(options.toplevel_dir, 'out', config) arguments = ['ninja', '-C', builddir] - print 'Building [%s]: %s' % (config, arguments) + print('Building [%s]: %s' % (config, arguments)) subprocess.check_call(arguments) @@ -2401,7 +2414,7 @@ def GenerateOutput(target_list, target_dicts, data, params): arglists.append( (target_list, target_dicts, data, params, config_name)) pool.map(CallGenerateOutputForConfig, arglists) - except KeyboardInterrupt, e: + except KeyboardInterrupt as e: pool.terminate() raise e else: diff --git a/gyp/pylib/gyp/generator/xcode.py b/gyp/pylib/gyp/generator/xcode.py index 0e3fb9301e..694a28afb1 100644 --- a/gyp/pylib/gyp/generator/xcode.py +++ b/gyp/pylib/gyp/generator/xcode.py @@ -1,3 +1,4 @@ +from __future__ import print_function # Copyright (c) 2012 Google Inc. All rights reserved. # Use of this source code is governed by a BSD-style license that can be # found in the LICENSE file. @@ -128,7 +129,7 @@ def __init__(self, gyp_path, path, build_file_dict): try: os.makedirs(self.path) self.created_dir = True - except OSError, e: + except OSError as e: if e.errno != errno.EEXIST: raise @@ -182,7 +183,7 @@ def Finalize1(self, xcode_targets, serialize_all_tests): # the tree tree view for UI display. # Any values set globally are applied to all configurations, then any # per-configuration values are applied. - for xck, xcv in self.build_file_dict.get('xcode_settings', {}).iteritems(): + for xck, xcv in self.build_file_dict.get('xcode_settings', {}).items(): xccl.SetBuildSetting(xck, xcv) if 'xcode_config_file' in self.build_file_dict: config_ref = self.project.AddOrGetFileInRootGroup( @@ -196,7 +197,7 @@ def Finalize1(self, xcode_targets, serialize_all_tests): if build_file_configuration_named: xcc = xccl.ConfigurationNamed(config_name) for xck, xcv in build_file_configuration_named.get('xcode_settings', - {}).iteritems(): + {}).items(): xcc.SetBuildSetting(xck, xcv) if 'xcode_config_file' in build_file_configuration_named: config_ref = self.project.AddOrGetFileInRootGroup( @@ -272,7 +273,7 @@ def Finalize1(self, xcode_targets, serialize_all_tests): script = script + "\n".join( ['export %s="%s"' % (key, gyp.xcodeproj_file.ConvertVariablesToShellSyntax(val)) - for (key, val) in command.get('environment').iteritems()]) + "\n" + for (key, val) in command.get('environment').items()]) + "\n" # Some test end up using sockets, files on disk, etc. and can get # confused if more then one test runs at a time. The generator @@ -453,7 +454,7 @@ def Write(self): same = False try: same = filecmp.cmp(pbxproj_path, new_pbxproj_path, False) - except OSError, e: + except OSError as e: if e.errno != errno.ENOENT: raise @@ -472,10 +473,10 @@ def Write(self): # # No way to get the umask without setting a new one? Set a safe one # and then set it back to the old value. - umask = os.umask(077) + umask = os.umask(0o77) os.umask(umask) - os.chmod(new_pbxproj_path, 0666 & ~umask) + os.chmod(new_pbxproj_path, 0o666 & ~umask) os.rename(new_pbxproj_path, pbxproj_path) except Exception: @@ -565,7 +566,7 @@ def EscapeXcodeDefine(s): def PerformBuild(data, configurations, params): options = params['options'] - for build_file, build_file_dict in data.iteritems(): + for build_file, build_file_dict in data.items(): (build_file_root, build_file_ext) = os.path.splitext(build_file) if build_file_ext != '.gyp': continue @@ -576,7 +577,7 @@ def PerformBuild(data, configurations, params): for config in configurations: arguments = ['xcodebuild', '-project', xcodeproj_path] arguments += ['-configuration', config] - print "Building [%s]: %s" % (config, arguments) + print("Building [%s]: %s" % (config, arguments)) subprocess.check_call(arguments) @@ -624,7 +625,7 @@ def GenerateOutput(target_list, target_dicts, data, params): skip_excluded_files = \ not generator_flags.get('xcode_list_excluded_files', True) xcode_projects = {} - for build_file, build_file_dict in data.iteritems(): + for build_file, build_file_dict in data.items(): (build_file_root, build_file_ext) = os.path.splitext(build_file) if build_file_ext != '.gyp': continue @@ -736,7 +737,7 @@ def GenerateOutput(target_list, target_dicts, data, params): xctarget_type = gyp.xcodeproj_file.PBXNativeTarget try: target_properties['productType'] = _types[type_bundle_key] - except KeyError, e: + except KeyError as e: gyp.common.ExceptionAppend(e, "-- unknown product type while " "writing target %s" % target_name) raise @@ -1013,7 +1014,7 @@ def GenerateOutput(target_list, target_dicts, data, params): # target. makefile.write('all: \\\n') for concrete_output_index in \ - xrange(0, len(concrete_outputs_by_rule_source)): + range(0, len(concrete_outputs_by_rule_source)): # Only list the first (index [0]) concrete output of each input # in the "all" target. Otherwise, a parallel make (-j > 1) would # attempt to process each input multiple times simultaneously. @@ -1036,7 +1037,7 @@ def GenerateOutput(target_list, target_dicts, data, params): # rule source. Collect the names of the directories that are # required. concrete_output_dirs = [] - for concrete_output_index in xrange(0, len(concrete_outputs)): + for concrete_output_index in range(0, len(concrete_outputs)): concrete_output = concrete_outputs[concrete_output_index] if concrete_output_index == 0: bol = '' @@ -1055,7 +1056,7 @@ def GenerateOutput(target_list, target_dicts, data, params): # the set of additional rule inputs, if any. prerequisites = [rule_source] prerequisites.extend(rule.get('inputs', [])) - for prerequisite_index in xrange(0, len(prerequisites)): + for prerequisite_index in range(0, len(prerequisites)): prerequisite = prerequisites[prerequisite_index] if prerequisite_index == len(prerequisites) - 1: eol = '' @@ -1277,7 +1278,7 @@ def GenerateOutput(target_list, target_dicts, data, params): set_define = EscapeXcodeDefine(define) xcbc.AppendBuildSetting('GCC_PREPROCESSOR_DEFINITIONS', set_define) if 'xcode_settings' in configuration: - for xck, xcv in configuration['xcode_settings'].iteritems(): + for xck, xcv in configuration['xcode_settings'].items(): xcbc.SetBuildSetting(xck, xcv) if 'xcode_config_file' in configuration: config_ref = pbxp.AddOrGetFileInRootGroup( @@ -1285,7 +1286,7 @@ def GenerateOutput(target_list, target_dicts, data, params): xcbc.SetBaseConfiguration(config_ref) build_files = [] - for build_file, build_file_dict in data.iteritems(): + for build_file, build_file_dict in data.items(): if build_file.endswith('.gyp'): build_files.append(build_file) diff --git a/gyp/pylib/gyp/input.py b/gyp/pylib/gyp/input.py index 7567d0a05b..68e51131d8 100644 --- a/gyp/pylib/gyp/input.py +++ b/gyp/pylib/gyp/input.py @@ -2,6 +2,8 @@ # Use of this source code is governed by a BSD-style license that can be # found in the LICENSE file. +from __future__ import print_function + from compiler.ast import Const from compiler.ast import Dict from compiler.ast import Discard @@ -233,7 +235,12 @@ def LoadOneBuildFile(build_file_path, data, aux_data, includes, # Open the build file for read ('r') with universal-newlines mode ('U') # to make sure platform specific newlines ('\r\n' or '\r') are converted to '\n' # which otherwise will fail eval() - build_file_contents = open(build_file_path, 'rU').read() + if sys.platform == 'zos': + # On z/OS, universal-newlines mode treats the file as an ascii file. But since + # node-gyp produces ebcdic files, do not use that mode. + build_file_contents = open(build_file_path, 'r').read() + else: + build_file_contents = open(build_file_path, 'rU').read() else: raise GypError("%s not found (cwd: %s)" % (build_file_path, os.getcwd())) @@ -244,10 +251,10 @@ def LoadOneBuildFile(build_file_path, data, aux_data, includes, else: build_file_data = eval(build_file_contents, {'__builtins__': None}, None) - except SyntaxError, e: + except SyntaxError as e: e.filename = build_file_path raise - except Exception, e: + except Exception as e: gyp.common.ExceptionAppend(e, 'while reading ' + build_file_path) raise @@ -267,7 +274,7 @@ def LoadOneBuildFile(build_file_path, data, aux_data, includes, else: LoadBuildFileIncludesIntoDict(build_file_data, build_file_path, data, aux_data, None, check) - except Exception, e: + except Exception as e: gyp.common.ExceptionAppend(e, 'while reading includes of ' + build_file_path) raise @@ -304,7 +311,7 @@ def LoadBuildFileIncludesIntoDict(subdict, subdict_path, data, aux_data, subdict_path, include) # Recurse into subdictionaries. - for k, v in subdict.iteritems(): + for k, v in subdict.items(): if type(v) is dict: LoadBuildFileIncludesIntoDict(v, subdict_path, data, aux_data, None, check) @@ -469,7 +476,7 @@ def LoadTargetBuildFile(build_file_path, data, aux_data, variables, includes, try: LoadTargetBuildFile(dependency, data, aux_data, variables, includes, depth, check, load_dependencies) - except Exception, e: + except Exception as e: gyp.common.ExceptionAppend( e, 'while loading dependencies of %s' % build_file_path) raise @@ -490,7 +497,7 @@ def CallLoadTargetBuildFile(global_flags, signal.signal(signal.SIGINT, signal.SIG_IGN) # Apply globals so that the worker process behaves the same. - for key, value in global_flags.iteritems(): + for key, value in global_flags.items(): globals()[key] = value SetGeneratorGlobals(generator_input_info) @@ -512,12 +519,12 @@ def CallLoadTargetBuildFile(global_flags, return (build_file_path, build_file_data, dependencies) - except GypError, e: + except GypError as e: sys.stderr.write("gyp: %s\n" % e) return None - except Exception, e: - print >>sys.stderr, 'Exception:', e - print >>sys.stderr, traceback.format_exc() + except Exception as e: + print('Exception:', e, file=sys.stderr) + print(traceback.format_exc(), file=sys.stderr) return None @@ -607,7 +614,7 @@ def LoadTargetBuildFilesParallel(build_files, data, variables, includes, depth, args = (global_flags, dependency, variables, includes, depth, check, generator_input_info), callback = parallel_state.LoadTargetBuildFileCallback) - except KeyboardInterrupt, e: + except KeyboardInterrupt as e: parallel_state.pool.terminate() raise e @@ -907,7 +914,7 @@ def ExpandVariables(input, phase, variables, build_file): stderr=subprocess.PIPE, stdin=subprocess.PIPE, cwd=build_file_dir) - except Exception, e: + except Exception as e: raise GypError("%s while executing command '%s' in %s" % (e, contents, build_file)) @@ -1021,7 +1028,7 @@ def ExpandVariables(input, phase, variables, build_file): # Convert all strings that are canonically-represented integers into integers. if type(output) is list: - for index in xrange(0, len(output)): + for index in range(0, len(output)): if IsStrCanonicalInt(output[index]): output[index] = int(output[index]) elif IsStrCanonicalInt(output): @@ -1092,13 +1099,13 @@ def EvalSingleCondition( if eval(ast_code, {'__builtins__': None}, variables): return true_dict return false_dict - except SyntaxError, e: + except SyntaxError as e: syntax_error = SyntaxError('%s while evaluating condition \'%s\' in %s ' 'at character %d.' % (str(e.args[0]), e.text, build_file, e.offset), e.filename, e.lineno, e.offset, e.text) raise syntax_error - except NameError, e: + except NameError as e: gyp.common.ExceptionAppend(e, 'while evaluating condition \'%s\' in %s' % (cond_expr_expanded, build_file)) raise GypError(e) @@ -1153,7 +1160,7 @@ def ProcessConditionsInDict(the_dict, phase, variables, build_file): def LoadAutomaticVariablesFromDict(variables, the_dict): # Any keys with plain string values in the_dict become automatic variables. # The variable name is the key name with a "_" character prepended. - for key, value in the_dict.iteritems(): + for key, value in the_dict.items(): if type(value) in (str, int, list): variables['_' + key] = value @@ -1166,7 +1173,7 @@ def LoadVariablesFromVariablesDict(variables, the_dict, the_dict_key): # the_dict in the_dict's parent dict. If the_dict's parent is not a dict # (it could be a list or it could be parentless because it is a root dict), # the_dict_key will be None. - for key, value in the_dict.get('variables', {}).iteritems(): + for key, value in the_dict.get('variables', {}).items(): if type(value) not in (str, int, list): continue @@ -1205,7 +1212,7 @@ def ProcessVariablesAndConditionsInDict(the_dict, phase, variables_in, # list before we process them so that you can reference one # variable from another. They will be fully expanded by recursion # in ExpandVariables. - for key, value in the_dict['variables'].iteritems(): + for key, value in the_dict['variables'].items(): variables[key] = value # Handle the associated variables dict first, so that any variable @@ -1218,7 +1225,7 @@ def ProcessVariablesAndConditionsInDict(the_dict, phase, variables_in, LoadVariablesFromVariablesDict(variables, the_dict, the_dict_key) - for key, value in the_dict.iteritems(): + for key, value in the_dict.items(): # Skip "variables", which was already processed if present. if key != 'variables' and type(value) is str: expanded = ExpandVariables(value, phase, variables, build_file) @@ -1276,7 +1283,7 @@ def ProcessVariablesAndConditionsInDict(the_dict, phase, variables_in, # Recurse into child dicts, or process child lists which may result in # further recursion into descendant dicts. - for key, value in the_dict.iteritems(): + for key, value in the_dict.items(): # Skip "variables" and string values, which were already processed if # present. if key == 'variables' or type(value) is str: @@ -1373,12 +1380,12 @@ def QualifyDependencies(targets): for dep in dependency_sections for op in ('', '!', '/')] - for target, target_dict in targets.iteritems(): + for target, target_dict in targets.items(): target_build_file = gyp.common.BuildFile(target) toolset = target_dict['toolset'] for dependency_key in all_dependency_sections: dependencies = target_dict.get(dependency_key, []) - for index in xrange(0, len(dependencies)): + for index in range(0, len(dependencies)): dep_file, dep_target, dep_toolset = gyp.common.ResolveTarget( target_build_file, dependencies[index], toolset) if not multiple_toolsets: @@ -1413,13 +1420,13 @@ def ExpandWildcardDependencies(targets, data): dependency list, must be qualified when this function is called. """ - for target, target_dict in targets.iteritems(): + for target, target_dict in targets.items(): toolset = target_dict['toolset'] target_build_file = gyp.common.BuildFile(target) for dependency_key in dependency_sections: dependencies = target_dict.get(dependency_key, []) - # Loop this way instead of "for dependency in" or "for index in xrange" + # Loop this way instead of "for dependency in" or "for index in range" # because the dependencies list will be modified within the loop body. index = 0 while index < len(dependencies): @@ -1475,7 +1482,7 @@ def Unify(l): def RemoveDuplicateDependencies(targets): """Makes sure every dependency appears only once in all targets's dependency lists.""" - for target_name, target_dict in targets.iteritems(): + for target_name, target_dict in targets.items(): for dependency_key in dependency_sections: dependencies = target_dict.get(dependency_key, []) if dependencies: @@ -1491,7 +1498,7 @@ def Filter(l, item): def RemoveSelfDependencies(targets): """Remove self dependencies from targets that have the prune_self_dependency variable set.""" - for target_name, target_dict in targets.iteritems(): + for target_name, target_dict in targets.items(): for dependency_key in dependency_sections: dependencies = target_dict.get(dependency_key, []) if dependencies: @@ -1504,7 +1511,7 @@ def RemoveSelfDependencies(targets): def RemoveLinkDependenciesFromNoneTargets(targets): """Remove dependencies having the 'link_dependency' attribute from the 'none' targets.""" - for target_name, target_dict in targets.iteritems(): + for target_name, target_dict in targets.items(): for dependency_key in dependency_sections: dependencies = target_dict.get(dependency_key, []) if dependencies: @@ -1790,14 +1797,14 @@ def BuildDependencyList(targets): # Create a DependencyGraphNode for each target. Put it into a dict for easy # access. dependency_nodes = {} - for target, spec in targets.iteritems(): + for target, spec in targets.items(): if target not in dependency_nodes: dependency_nodes[target] = DependencyGraphNode(target) # Set up the dependency links. Targets that have no dependencies are treated # as dependent on root_node. root_node = DependencyGraphNode(None) - for target, spec in targets.iteritems(): + for target, spec in targets.items(): target_node = dependency_nodes[target] target_build_file = gyp.common.BuildFile(target) dependencies = spec.get('dependencies') @@ -1846,14 +1853,14 @@ def VerifyNoGYPFileCircularDependencies(targets): dependency_nodes[build_file] = DependencyGraphNode(build_file) # Set up the dependency links. - for target, spec in targets.iteritems(): + for target, spec in targets.items(): build_file = gyp.common.BuildFile(target) build_file_node = dependency_nodes[build_file] target_dependencies = spec.get('dependencies', []) for dependency in target_dependencies: try: dependency_build_file = gyp.common.BuildFile(dependency) - except GypError, e: + except GypError as e: gyp.common.ExceptionAppend( e, 'while computing dependencies of .gyp file %s' % build_file) raise @@ -2033,7 +2040,7 @@ def MakePathRelative(to_file, fro_file, item): gyp.common.RelativePath(os.path.dirname(fro_file), os.path.dirname(to_file)), item)).replace('\\', '/') - if item[-1] == '/': + if item[-1:] == '/': ret += '/' return ret @@ -2111,7 +2118,7 @@ def is_in_set_or_list(x, s, l): def MergeDicts(to, fro, to_file, fro_file): # I wanted to name the parameter "from" but it's a Python keyword... - for k, v in fro.iteritems(): + for k, v in fro.items(): # It would be nice to do "if not k in to: to[k] = v" but that wouldn't give # copy semantics. Something else may want to merge from the |fro| dict # later, and having the same dict ref pointed to twice in the tree isn't @@ -2246,13 +2253,13 @@ def SetUpConfigurations(target, target_dict): if not 'configurations' in target_dict: target_dict['configurations'] = {'Default': {}} if not 'default_configuration' in target_dict: - concrete = [i for (i, config) in target_dict['configurations'].iteritems() + concrete = [i for (i, config) in target_dict['configurations'].items() if not config.get('abstract')] target_dict['default_configuration'] = sorted(concrete)[0] merged_configurations = {} configs = target_dict['configurations'] - for (configuration, old_configuration_dict) in configs.iteritems(): + for (configuration, old_configuration_dict) in configs.items(): # Skip abstract configurations (saves work only). if old_configuration_dict.get('abstract'): continue @@ -2260,7 +2267,7 @@ def SetUpConfigurations(target, target_dict): # Get the inheritance relationship right by making a copy of the target # dict. new_configuration_dict = {} - for (key, target_val) in target_dict.iteritems(): + for (key, target_val) in target_dict.items(): key_ext = key[-1:] if key_ext in key_suffixes: key_base = key[:-1] @@ -2344,7 +2351,7 @@ def ProcessListFiltersInDict(name, the_dict): lists = [] del_lists = [] - for key, value in the_dict.iteritems(): + for key, value in the_dict.items(): operation = key[-1] if operation != '!' and operation != '/': continue @@ -2392,7 +2399,7 @@ def ProcessListFiltersInDict(name, the_dict): exclude_key = list_key + '!' if exclude_key in the_dict: for exclude_item in the_dict[exclude_key]: - for index in xrange(0, len(the_list)): + for index in range(0, len(the_list)): if exclude_item == the_list[index]: # This item matches the exclude_item, so set its action to 0 # (exclude). @@ -2418,7 +2425,7 @@ def ProcessListFiltersInDict(name, the_dict): raise ValueError('Unrecognized action ' + action + ' in ' + name + \ ' key ' + regex_key) - for index in xrange(0, len(the_list)): + for index in range(0, len(the_list)): list_item = the_list[index] if list_actions[index] == action_value: # Even if the regex matches, nothing will change so continue (regex @@ -2449,7 +2456,7 @@ def ProcessListFiltersInDict(name, the_dict): # the indices of items that haven't been seen yet don't shift. That means # that things need to be prepended to excluded_list to maintain them in the # same order that they existed in the_list. - for index in xrange(len(list_actions) - 1, -1, -1): + for index in range(len(list_actions) - 1, -1, -1): if list_actions[index] == 0: # Dump anything with action 0 (exclude). Keep anything with action 1 # (include) or -1 (no include or exclude seen for the item). @@ -2462,7 +2469,7 @@ def ProcessListFiltersInDict(name, the_dict): the_dict[excluded_key] = excluded_list # Now recurse into subdicts and lists that may contain dicts. - for key, value in the_dict.iteritems(): + for key, value in the_dict.items(): if type(value) is dict: ProcessListFiltersInDict(key, value) elif type(value) is list: @@ -2519,13 +2526,13 @@ def ValidateSourcesInTarget(target, target_dict, build_file, basenames.setdefault(basename, []).append(source) error = '' - for basename, files in basenames.iteritems(): + for basename, files in basenames.items(): if len(files) > 1: error += ' %s: %s\n' % (basename, ' '.join(files)) if error: - print('static library %s has several files with the same basename:\n' % - target + error + 'libtool on Mac cannot handle that. Use ' + print('static library %s has several files with the same basename:\n' % target + + error + 'libtool on Mac cannot handle that. Use ' '--no-duplicate-basename-check to disable this validation.') raise GypError('Duplicate basenames in sources section, see list above') @@ -2639,7 +2646,7 @@ def ValidateActionsInTarget(target, target_dict, build_file): def TurnIntIntoStrInDict(the_dict): """Given dict the_dict, recursively converts all integers into strings. """ - # Use items instead of iteritems because there's no need to try to look at + # Use items instead of items because there's no need to try to look at # reinserted keys and their associated values. for k, v in the_dict.items(): if type(v) is int: @@ -2658,7 +2665,7 @@ def TurnIntIntoStrInDict(the_dict): def TurnIntIntoStrInList(the_list): """Given list the_list, recursively converts all integers into strings. """ - for index in xrange(0, len(the_list)): + for index in range(0, len(the_list)): item = the_list[index] if type(item) is int: the_list[index] = str(item) @@ -2776,7 +2783,7 @@ def Load(build_files, variables, includes, depth, generator_input_info, check, try: LoadTargetBuildFile(build_file, data, aux_data, variables, includes, depth, check, True) - except Exception, e: + except Exception as e: gyp.common.ExceptionAppend(e, 'while trying to load %s' % build_file) raise @@ -2798,7 +2805,7 @@ def Load(build_files, variables, includes, depth, generator_input_info, check, RemoveLinkDependenciesFromNoneTargets(targets) # Apply exclude (!) and regex (/) list filters only for dependency_sections. - for target_name, target_dict in targets.iteritems(): + for target_name, target_dict in targets.items(): tmp_dict = {} for key_base in dependency_sections: for op in ('', '!', '/'): diff --git a/gyp/pylib/gyp/input_test.py b/gyp/pylib/gyp/input_test.py index 4234fbb830..1bc5e3d308 100755 --- a/gyp/pylib/gyp/input_test.py +++ b/gyp/pylib/gyp/input_test.py @@ -22,38 +22,38 @@ def _create_dependency(self, dependent, dependency): dependency.dependents.append(dependent) def test_no_cycle_empty_graph(self): - for label, node in self.nodes.iteritems(): - self.assertEquals([], node.FindCycles()) + for label, node in self.nodes.items(): + self.assertEqual([], node.FindCycles()) def test_no_cycle_line(self): self._create_dependency(self.nodes['a'], self.nodes['b']) self._create_dependency(self.nodes['b'], self.nodes['c']) self._create_dependency(self.nodes['c'], self.nodes['d']) - for label, node in self.nodes.iteritems(): - self.assertEquals([], node.FindCycles()) + for label, node in self.nodes.items(): + self.assertEqual([], node.FindCycles()) def test_no_cycle_dag(self): self._create_dependency(self.nodes['a'], self.nodes['b']) self._create_dependency(self.nodes['a'], self.nodes['c']) self._create_dependency(self.nodes['b'], self.nodes['c']) - for label, node in self.nodes.iteritems(): - self.assertEquals([], node.FindCycles()) + for label, node in self.nodes.items(): + self.assertEqual([], node.FindCycles()) def test_cycle_self_reference(self): self._create_dependency(self.nodes['a'], self.nodes['a']) - self.assertEquals([[self.nodes['a'], self.nodes['a']]], + self.assertEqual([[self.nodes['a'], self.nodes['a']]], self.nodes['a'].FindCycles()) def test_cycle_two_nodes(self): self._create_dependency(self.nodes['a'], self.nodes['b']) self._create_dependency(self.nodes['b'], self.nodes['a']) - self.assertEquals([[self.nodes['a'], self.nodes['b'], self.nodes['a']]], + self.assertEqual([[self.nodes['a'], self.nodes['b'], self.nodes['a']]], self.nodes['a'].FindCycles()) - self.assertEquals([[self.nodes['b'], self.nodes['a'], self.nodes['b']]], + self.assertEqual([[self.nodes['b'], self.nodes['a'], self.nodes['b']]], self.nodes['b'].FindCycles()) def test_two_cycles(self): @@ -68,7 +68,7 @@ def test_two_cycles(self): [self.nodes['a'], self.nodes['b'], self.nodes['a']] in cycles) self.assertTrue( [self.nodes['b'], self.nodes['c'], self.nodes['b']] in cycles) - self.assertEquals(2, len(cycles)) + self.assertEqual(2, len(cycles)) def test_big_cycle(self): self._create_dependency(self.nodes['a'], self.nodes['b']) @@ -77,7 +77,7 @@ def test_big_cycle(self): self._create_dependency(self.nodes['d'], self.nodes['e']) self._create_dependency(self.nodes['e'], self.nodes['a']) - self.assertEquals([[self.nodes['a'], + self.assertEqual([[self.nodes['a'], self.nodes['b'], self.nodes['c'], self.nodes['d'], diff --git a/gyp/pylib/gyp/mac_tool.py b/gyp/pylib/gyp/mac_tool.py index eeeaceb0c7..b8b7344eff 100755 --- a/gyp/pylib/gyp/mac_tool.py +++ b/gyp/pylib/gyp/mac_tool.py @@ -8,6 +8,8 @@ These functions are executed via gyp-mac-tool when using the Makefile generator. """ +from __future__ import print_function + import fcntl import fnmatch import glob @@ -125,7 +127,7 @@ def _DetectInputEncoding(self, file_name): fp = open(file_name, 'rb') try: header = fp.read(3) - except e: + except Exception: fp.close() return None fp.close() @@ -243,7 +245,7 @@ def ExecFilterLibtool(self, *cmd_list): _, err = libtoolout.communicate() for line in err.splitlines(): if not libtool_re.match(line) and not libtool_re5.match(line): - print >>sys.stderr, line + print(line, file=sys.stderr) # Unconditionally touch the output .a file on the command line if present # and the command succeeded. A bit hacky. if not libtoolout.returncode: @@ -324,7 +326,7 @@ def ExecCompileXcassets(self, keys, *inputs): ]) if keys: keys = json.loads(keys) - for key, value in keys.iteritems(): + for key, value in keys.items(): arg_name = '--' + key if isinstance(value, bool): if value: @@ -440,8 +442,7 @@ def _FindProvisioningProfile(self, profile, bundle_identifier): profiles_dir = os.path.join( os.environ['HOME'], 'Library', 'MobileDevice', 'Provisioning Profiles') if not os.path.isdir(profiles_dir): - print >>sys.stderr, ( - 'cannot find mobile provisioning for %s' % bundle_identifier) + print('cannot find mobile provisioning for %s' % (bundle_identifier), file=sys.stderr) sys.exit(1) provisioning_profiles = None if profile: @@ -462,8 +463,7 @@ def _FindProvisioningProfile(self, profile, bundle_identifier): valid_provisioning_profiles[app_id_pattern] = ( profile_path, profile_data, team_identifier) if not valid_provisioning_profiles: - print >>sys.stderr, ( - 'cannot find mobile provisioning for %s' % bundle_identifier) + print('cannot find mobile provisioning for %s' % (bundle_identifier), file=sys.stderr) sys.exit(1) # If the user has multiple provisioning profiles installed that can be # used for ${bundle_identifier}, pick the most specific one (ie. the @@ -487,7 +487,7 @@ def _LoadProvisioningProfile(self, profile_path): def _MergePlist(self, merged_plist, plist): """Merge |plist| into |merged_plist|.""" - for key, value in plist.iteritems(): + for key, value in plist.items(): if isinstance(value, dict): merged_value = merged_plist.get(key, {}) if isinstance(merged_value, dict): @@ -597,7 +597,7 @@ def _ExpandVariables(self, data, substitutions): the key was not found. """ if isinstance(data, str): - for key, value in substitutions.iteritems(): + for key, value in substitutions.items(): data = data.replace('$(%s)' % key, value) return data if isinstance(data, list): diff --git a/gyp/pylib/gyp/msvs_emulation.py b/gyp/pylib/gyp/msvs_emulation.py index ca67b122f0..75da78526c 100644 --- a/gyp/pylib/gyp/msvs_emulation.py +++ b/gyp/pylib/gyp/msvs_emulation.py @@ -205,7 +205,7 @@ def __init__(self, spec, generator_flags): configs = spec['configurations'] for field, default in supported_fields: setattr(self, field, {}) - for configname, config in configs.iteritems(): + for configname, config in configs.items(): getattr(self, field)[configname] = config.get(field, default()) self.msvs_cygwin_dirs = spec.get('msvs_cygwin_dirs', ['.']) @@ -941,7 +941,7 @@ def ExpandMacros(string, expansions): """Expand $(Variable) per expansions dict. See MsvsSettings.GetVSMacroEnv for the canonical way to retrieve a suitable dict.""" if '$' in string: - for old, new in expansions.iteritems(): + for old, new in expansions.items(): assert '$(' not in new, new string = string.replace(old, new) return string @@ -985,7 +985,7 @@ def _FormatAsEnvironmentBlock(envvar_dict): CreateProcess documentation for more details.""" block = '' nul = '\0' - for key, value in envvar_dict.iteritems(): + for key, value in envvar_dict.items(): block += key + '=' + value + nul block += nul return block diff --git a/gyp/pylib/gyp/ordered_dict.py b/gyp/pylib/gyp/ordered_dict.py index a1e89f9199..6fe9c1f6c7 100644 --- a/gyp/pylib/gyp/ordered_dict.py +++ b/gyp/pylib/gyp/ordered_dict.py @@ -161,8 +161,8 @@ def itervalues(self): for k in self: yield self[k] - def iteritems(self): - 'od.iteritems -> an iterator over the (key, value) items in od' + def items(self): + 'od.items -> an iterator over the (key, value) items in od' for k in self: yield (k, self[k]) diff --git a/gyp/pylib/gyp/simple_copy.py b/gyp/pylib/gyp/simple_copy.py index 74c98c5a79..463929386e 100644 --- a/gyp/pylib/gyp/simple_copy.py +++ b/gyp/pylib/gyp/simple_copy.py @@ -38,7 +38,7 @@ def _deepcopy_list(x): def _deepcopy_dict(x): y = {} - for key, value in x.iteritems(): + for key, value in x.items(): y[deepcopy(key)] = deepcopy(value) return y d[dict] = _deepcopy_dict diff --git a/gyp/pylib/gyp/win_tool.py b/gyp/pylib/gyp/win_tool.py index bb6f1ea436..bca0b9e346 100755 --- a/gyp/pylib/gyp/win_tool.py +++ b/gyp/pylib/gyp/win_tool.py @@ -9,6 +9,8 @@ These functions are executed via gyp-win-tool when using the ninja generator. """ +from __future__ import print_function + import os import re import shutil @@ -126,7 +128,7 @@ def ExecLinkWrapper(self, arch, use_separate_mspdbsrv, *args): if (not line.startswith(' Creating library ') and not line.startswith('Generating code') and not line.startswith('Finished generating code')): - print line + print(line) return link.returncode def ExecLinkWithManifests(self, arch, embed_manifest, out, ldcmd, resname, @@ -215,7 +217,7 @@ def ExecManifestWrapper(self, arch, *args): out, _ = popen.communicate() for line in out.splitlines(): if line and 'manifest authoring warning 81010002' not in line: - print line + print(line) return popen.returncode def ExecManifestToRc(self, arch, *args): @@ -255,7 +257,7 @@ def ExecMidlWrapper(self, arch, outdir, tlb, h, dlldata, iid, proxy, idl, for x in lines if x.startswith(prefixes)) for line in lines: if not line.startswith(prefixes) and line not in processing: - print line + print(line) return popen.returncode def ExecAsmWrapper(self, arch, *args): @@ -269,7 +271,7 @@ def ExecAsmWrapper(self, arch, *args): not line.startswith('Microsoft (R) Macro Assembler') and not line.startswith(' Assembling: ') and line): - print line + print(line) return popen.returncode def ExecRcWrapper(self, arch, *args): @@ -283,7 +285,7 @@ def ExecRcWrapper(self, arch, *args): if (not line.startswith('Microsoft (R) Windows (R) Resource Compiler') and not line.startswith('Copyright (C) Microsoft Corporation') and line): - print line + print(line) return popen.returncode def ExecActionWrapper(self, arch, rspfile, *dir): @@ -292,7 +294,7 @@ def ExecActionWrapper(self, arch, rspfile, *dir): env = self._GetEnv(arch) # TODO(scottmg): This is a temporary hack to get some specific variables # through to actions that are set after gyp-time. http://crbug.com/333738. - for k, v in os.environ.iteritems(): + for k, v in os.environ.items(): if k not in env: env[k] = v args = open(rspfile).read() diff --git a/gyp/pylib/gyp/xcode_emulation.py b/gyp/pylib/gyp/xcode_emulation.py index b06bdc4e8b..66f325932a 100644 --- a/gyp/pylib/gyp/xcode_emulation.py +++ b/gyp/pylib/gyp/xcode_emulation.py @@ -7,6 +7,8 @@ other build systems, such as make and ninja. """ +from __future__ import print_function + import copy import gyp.common import os @@ -73,7 +75,7 @@ def _ExpandArchs(self, archs, sdkroot): if arch not in expanded_archs: expanded_archs.append(arch) except KeyError as e: - print 'Warning: Ignoring unsupported variable "%s".' % variable + print('Warning: Ignoring unsupported variable "%s".' % variable) elif arch not in expanded_archs: expanded_archs.append(arch) return expanded_archs @@ -168,7 +170,7 @@ def __init__(self, spec): # the same for all configs are implicitly per-target settings. self.xcode_settings = {} configs = spec['configurations'] - for configname, config in configs.iteritems(): + for configname, config in configs.items(): self.xcode_settings[configname] = config.get('xcode_settings', {}) self._ConvertConditionalKeys(configname) if self.xcode_settings[configname].get('IPHONEOS_DEPLOYMENT_TARGET', @@ -194,8 +196,8 @@ def _ConvertConditionalKeys(self, configname): new_key = key.split("[")[0] settings[new_key] = settings[key] else: - print 'Warning: Conditional keys not implemented, ignoring:', \ - ' '.join(conditional_keys) + print('Warning: Conditional keys not implemented, ignoring:', \ + ' '.join(conditional_keys)) del settings[key] def _Settings(self): @@ -213,7 +215,7 @@ def _Appendf(self, lst, test_key, format_str, default=None): def _WarnUnimplemented(self, test_key): if test_key in self._Settings(): - print 'Warning: Ignoring not yet implemented key "%s".' % test_key + print('Warning: Ignoring not yet implemented key "%s".' % test_key) def IsBinaryOutputFormat(self, configname): default = "binary" if self.isIOS else "xml" @@ -233,6 +235,9 @@ def _IsIosWatchKitExtension(self): def _IsIosWatchApp(self): return int(self.spec.get('ios_watch_app', 0)) != 0 + def _IsXCTest(self): + return int(self.spec.get('mac_xctest_bundle', 0)) != 0 + def GetFrameworkVersion(self): """Returns the framework version of the current target. Only valid for bundles.""" @@ -429,7 +434,7 @@ def _GetSdkVersionInfoItem(self, sdk, infoitem): # Since the CLT has no SDK paths anyway, returning None is the # most sensible route and should still do the right thing. try: - return GetStdout(['xcodebuild', '-version', '-sdk', sdk, infoitem]) + return GetStdoutQuiet(['xcodebuild', '-version', '-sdk', sdk, infoitem]) except: pass @@ -568,6 +573,11 @@ def GetCflags(self, configname, arch=None): cflags += self._Settings().get('WARNING_CFLAGS', []) + if self._IsXCTest(): + platform_root = self._XcodePlatformPath(configname) + if platform_root: + cflags.append('-F' + platform_root + '/Developer/Library/Frameworks/') + if sdk_root: framework_root = sdk_root else: @@ -831,6 +841,11 @@ def GetLdflags(self, configname, product_dir, gyp_to_build_path, arch=None): for directory in framework_dirs: ldflags.append('-F' + directory.replace('$(SDKROOT)', sdk_root)) + if self._IsXCTest(): + platform_root = self._XcodePlatformPath(configname) + if platform_root: + cflags.append('-F' + platform_root + '/Developer/Library/Frameworks/') + is_extension = self._IsIosAppExtension() or self._IsIosWatchKitExtension() if sdk_root and is_extension: # Adds the link flags for extensions. These flags are common for all @@ -876,7 +891,7 @@ def GetPerTargetSettings(self): result = dict(self.xcode_settings[configname]) first_pass = False else: - for key, value in self.xcode_settings[configname].iteritems(): + for key, value in self.xcode_settings[configname].items(): if key not in result: continue elif result[key] != value: @@ -984,8 +999,8 @@ def _GetIOSPostbuilds(self, configname, output_binary): unimpl = ['OTHER_CODE_SIGN_FLAGS'] unimpl = set(unimpl) & set(self.xcode_settings[configname].keys()) if unimpl: - print 'Warning: Some codesign keys not implemented, ignoring: %s' % ( - ', '.join(sorted(unimpl))) + print('Warning: Some codesign keys not implemented, ignoring: %s' % ( + ', '.join(sorted(unimpl)))) return ['%s code-sign-bundle "%s" "%s" "%s" "%s"' % ( os.path.join('${TARGET_BUILD_DIR}', 'gyp-mac-tool'), key, @@ -1251,7 +1266,7 @@ def XcodeVersion(): if XCODE_VERSION_CACHE: return XCODE_VERSION_CACHE try: - version_list = GetStdout(['xcodebuild', '-version']).splitlines() + version_list = GetStdoutQuiet(['xcodebuild', '-version']).splitlines() # In some circumstances xcodebuild exits 0 but doesn't return # the right results; for example, a user on 10.7 or 10.8 with # a bogus path set via xcode-select @@ -1262,7 +1277,7 @@ def XcodeVersion(): except: version = CLTVersion() if version: - version = re.match(r'(\d\.\d\.?\d*)', version).groups()[0] + version = re.match(r'(\d+\.\d+\.?\d*)', version).groups()[0] else: raise GypError("No Xcode or CLT version detected!") # The CLT has no build information, so we return an empty string. @@ -1301,6 +1316,17 @@ def CLTVersion(): continue +def GetStdoutQuiet(cmdlist): + """Returns the content of standard output returned by invoking |cmdlist|. + Ignores the stderr. + Raises |GypError| if the command return with a non-zero return code.""" + job = subprocess.Popen(cmdlist, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + out = job.communicate()[0] + if job.returncode != 0: + raise GypError('Error %d running %s' % (job.returncode, cmdlist[0])) + return out.rstrip('\n') + + def GetStdout(cmdlist): """Returns the content of standard output returned by invoking |cmdlist|. Raises |GypError| if the command return with a non-zero return code.""" @@ -1573,7 +1599,7 @@ def GetEdges(node): order = gyp.common.TopologicallySorted(env.keys(), GetEdges) order.reverse() return order - except gyp.common.CycleError, e: + except gyp.common.CycleError as e: raise GypError( 'Xcode environment variables are cyclically dependent: ' + str(e.nodes)) @@ -1613,7 +1639,7 @@ def _AddIOSDeviceConfigurations(targets): for target_dict in targets.itervalues(): toolset = target_dict['toolset'] configs = target_dict['configurations'] - for config_name, config_dict in dict(configs).iteritems(): + for config_name, config_dict in dict(configs).items(): iphoneos_config_dict = copy.deepcopy(config_dict) configs[config_name + '-iphoneos'] = iphoneos_config_dict configs[config_name + '-iphonesimulator'] = config_dict diff --git a/gyp/pylib/gyp/xcode_ninja.py b/gyp/pylib/gyp/xcode_ninja.py index 3820d6bf04..5acd82e004 100644 --- a/gyp/pylib/gyp/xcode_ninja.py +++ b/gyp/pylib/gyp/xcode_ninja.py @@ -28,7 +28,7 @@ def _WriteWorkspace(main_gyp, sources_gyp, params): workspace_path = os.path.join(options.generator_output, workspace_path) try: os.makedirs(workspace_path) - except OSError, e: + except OSError as e: if e.errno != errno.EEXIST: raise output_string = '\n' + \ @@ -161,7 +161,7 @@ def CreateWrapper(target_list, target_dicts, data, params): params: Dict of global options for gyp. """ orig_gyp = params['build_files'][0] - for gyp_name, gyp_dict in data.iteritems(): + for gyp_name, gyp_dict in data.items(): if gyp_name == orig_gyp: depth = gyp_dict['_DEPTH'] @@ -228,7 +228,7 @@ def CreateWrapper(target_list, target_dicts, data, params): sources_target['configurations'] = {'Default': { 'include_dirs': [ depth ] } } sources = [] - for target, target_dict in target_dicts.iteritems(): + for target, target_dict in target_dicts.items(): base = os.path.dirname(target) files = target_dict.get('sources', []) + \ target_dict.get('mac_bundle_resources', []) diff --git a/gyp/pylib/gyp/xcodeproj_file.py b/gyp/pylib/gyp/xcodeproj_file.py index d08b7f7770..9bd582e477 100644 --- a/gyp/pylib/gyp/xcodeproj_file.py +++ b/gyp/pylib/gyp/xcodeproj_file.py @@ -314,7 +314,7 @@ def Copy(self): """ that = self.__class__(id=self.id, parent=self.parent) - for key, value in self._properties.iteritems(): + for key, value in self._properties.items(): is_strong = self._schema[key][2] if isinstance(value, XCObject): @@ -452,7 +452,7 @@ def _HashUpdate(hash, data): digest_int_count = hash.digest_size / 4 digest_ints = struct.unpack('>' + 'I' * digest_int_count, hash.digest()) id_ints = [0, 0, 0] - for index in xrange(0, digest_int_count): + for index in range(0, digest_int_count): id_ints[index % 3] ^= digest_ints[index] self.id = '%08X%08X%08X' % tuple(id_ints) @@ -475,7 +475,7 @@ def Children(self): """Returns a list of all of this object's owned (strong) children.""" children = [] - for property, attributes in self._schema.iteritems(): + for property, attributes in self._schema.items(): (is_list, property_type, is_strong) = attributes[0:3] if is_strong and property in self._properties: if not is_list: @@ -622,7 +622,7 @@ def _XCPrintableValue(self, tabs, value, flatten_list=False): printable += end_tabs + ')' elif isinstance(value, dict): printable = '{' + sep - for item_key, item_value in sorted(value.iteritems()): + for item_key, item_value in sorted(value.items()): printable += element_tabs + \ self._XCPrintableValue(tabs + 1, item_key, flatten_list) + ' = ' + \ self._XCPrintableValue(tabs + 1, item_value, flatten_list) + ';' + \ @@ -691,7 +691,7 @@ def _XCKVPrint(self, file, tabs, key, value): printable_value[0] == '"' and printable_value[-1] == '"': printable_value = printable_value[1:-1] printable += printable_key + ' = ' + printable_value + ';' + after_kv - except TypeError, e: + except TypeError as e: gyp.common.ExceptionAppend(e, 'while printing key "%s"' % key) raise @@ -730,7 +730,7 @@ def Print(self, file=sys.stdout): self._XCKVPrint(file, 3, 'isa', self.__class__.__name__) # The remaining elements of an object dictionary are sorted alphabetically. - for property, value in sorted(self._properties.iteritems()): + for property, value in sorted(self._properties.items()): self._XCKVPrint(file, 3, property, value) # End the object. @@ -752,7 +752,7 @@ def UpdateProperties(self, properties, do_copy=False): if properties is None: return - for property, value in properties.iteritems(): + for property, value in properties.items(): # Make sure the property is in the schema. if not property in self._schema: raise KeyError(property + ' not in ' + self.__class__.__name__) @@ -865,7 +865,7 @@ def VerifyHasRequiredProperties(self): # TODO(mark): A stronger verification mechanism is needed. Some # subclasses need to perform validation beyond what the schema can enforce. - for property, attributes in self._schema.iteritems(): + for property, attributes in self._schema.items(): (is_list, property_type, is_strong, is_required) = attributes[0:4] if is_required and not property in self._properties: raise KeyError(self.__class__.__name__ + ' requires ' + property) @@ -875,7 +875,7 @@ def _SetDefaultsFromSchema(self): overwrite properties that have already been set.""" defaults = {} - for property, attributes in self._schema.iteritems(): + for property, attributes in self._schema.items(): (is_list, property_type, is_strong, is_required) = attributes[0:4] if is_required and len(attributes) >= 5 and \ not property in self._properties: @@ -1426,7 +1426,7 @@ def PathHashables(self): xche = self while xche != None and isinstance(xche, XCHierarchicalElement): xche_hashables = xche.Hashables() - for index in xrange(0, len(xche_hashables)): + for index in range(0, len(xche_hashables)): hashables.insert(index, xche_hashables[index]) xche = xche.parent return hashables @@ -2401,7 +2401,7 @@ def HeadersPhase(self): # The headers phase should come before the resources, sources, and # frameworks phases, if any. insert_at = len(self._properties['buildPhases']) - for index in xrange(0, len(self._properties['buildPhases'])): + for index in range(0, len(self._properties['buildPhases'])): phase = self._properties['buildPhases'][index] if isinstance(phase, PBXResourcesBuildPhase) or \ isinstance(phase, PBXSourcesBuildPhase) or \ @@ -2422,7 +2422,7 @@ def ResourcesPhase(self): # The resources phase should come before the sources and frameworks # phases, if any. insert_at = len(self._properties['buildPhases']) - for index in xrange(0, len(self._properties['buildPhases'])): + for index in range(0, len(self._properties['buildPhases'])): phase = self._properties['buildPhases'][index] if isinstance(phase, PBXSourcesBuildPhase) or \ isinstance(phase, PBXFrameworksBuildPhase): @@ -2844,7 +2844,7 @@ def CompareProducts(x, y, remote_products): # determine the sort order. return cmp(x_index, y_index) - for other_pbxproject, ref_dict in self._other_pbxprojects.iteritems(): + for other_pbxproject, ref_dict in self._other_pbxprojects.items(): # Build up a list of products in the remote project file, ordered the # same as the targets that produce them. remote_products = [] @@ -2889,7 +2889,7 @@ def Print(self, file=sys.stdout): self._XCPrint(file, 0, '{ ') else: self._XCPrint(file, 0, '{\n') - for property, value in sorted(self._properties.iteritems(), + for property, value in sorted(self._properties.items(), cmp=lambda x, y: cmp(x, y)): if property == 'objects': self._PrintObjects(file) diff --git a/gyp/tools/graphviz.py b/gyp/tools/graphviz.py index 326ae221cf..538b059da4 100755 --- a/gyp/tools/graphviz.py +++ b/gyp/tools/graphviz.py @@ -8,6 +8,8 @@ generate input suitable for graphviz to render a dependency graph of targets.""" +from __future__ import print_function + import collections import json import sys @@ -50,9 +52,9 @@ def WriteGraph(edges): build_file, target_name, toolset = ParseTarget(src) files[build_file].append(src) - print 'digraph D {' - print ' fontsize=8' # Used by subgraphs. - print ' node [fontsize=8]' + print('digraph D {') + print(' fontsize=8') # Used by subgraphs. + print(' node [fontsize=8]') # Output nodes by file. We must first write out each node within # its file grouping before writing out any edges that may refer @@ -63,31 +65,31 @@ def WriteGraph(edges): # the display by making it a box without an internal node. target = targets[0] build_file, target_name, toolset = ParseTarget(target) - print ' "%s" [shape=box, label="%s\\n%s"]' % (target, filename, - target_name) + print(' "%s" [shape=box, label="%s\\n%s"]' % (target, filename, + target_name)) else: # Group multiple nodes together in a subgraph. - print ' subgraph "cluster_%s" {' % filename - print ' label = "%s"' % filename + print(' subgraph "cluster_%s" {' % filename) + print(' label = "%s"' % filename) for target in targets: build_file, target_name, toolset = ParseTarget(target) - print ' "%s" [label="%s"]' % (target, target_name) - print ' }' + print(' "%s" [label="%s"]' % (target, target_name)) + print(' }') # Now that we've placed all the nodes within subgraphs, output all # the edges between nodes. for src, dsts in edges.items(): for dst in dsts: - print ' "%s" -> "%s"' % (src, dst) + print(' "%s" -> "%s"' % (src, dst)) - print '}' + print('}') def main(): if len(sys.argv) < 2: - print >>sys.stderr, __doc__ - print >>sys.stderr - print >>sys.stderr, 'usage: %s target1 target2...' % (sys.argv[0]) + print(__doc__, file=sys.stderr) + print(file=sys.stderr) + print('usage: %s target1 target2...' % (sys.argv[0]), file=sys.stderr) return 1 edges = LoadEdges('dump.json', sys.argv[1:]) diff --git a/gyp/tools/pretty_gyp.py b/gyp/tools/pretty_gyp.py index c51d35872c..d01c692edc 100755 --- a/gyp/tools/pretty_gyp.py +++ b/gyp/tools/pretty_gyp.py @@ -6,6 +6,8 @@ """Pretty-prints the contents of a GYP file.""" +from __future__ import print_function + import sys import re @@ -119,22 +121,22 @@ def prettyprint_input(lines): last_line = "" for line in lines: if COMMENT_RE.match(line): - print line + print(line) else: line = line.strip('\r\n\t ') # Otherwise doesn't strip \r on Unix. if len(line) > 0: (brace_diff, after) = count_braces(line) if brace_diff != 0: if after: - print " " * (basic_offset * indent) + line + print(" " * (basic_offset * indent) + line) indent += brace_diff else: indent += brace_diff - print " " * (basic_offset * indent) + line + print(" " * (basic_offset * indent) + line) else: - print " " * (basic_offset * indent) + line + print(" " * (basic_offset * indent) + line) else: - print "" + print("") last_line = line diff --git a/gyp/tools/pretty_sln.py b/gyp/tools/pretty_sln.py index ca8cf4ad3f..731844caf7 100755 --- a/gyp/tools/pretty_sln.py +++ b/gyp/tools/pretty_sln.py @@ -14,6 +14,8 @@ __author__ = 'nsylvain (Nicolas Sylvain)' +from __future__ import print_function + import os import re import sys @@ -26,7 +28,7 @@ def BuildProject(project, built, projects, deps): for dep in deps[project]: if dep not in built: BuildProject(dep, built, projects, deps) - print project + print(project) built.append(project) def ParseSolution(solution_file): @@ -100,44 +102,44 @@ def ParseSolution(solution_file): return (projects, dependencies) def PrintDependencies(projects, deps): - print "---------------------------------------" - print "Dependencies for all projects" - print "---------------------------------------" - print "-- --" + print("---------------------------------------") + print("Dependencies for all projects") + print("---------------------------------------") + print("-- --") for (project, dep_list) in sorted(deps.items()): - print "Project : %s" % project - print "Path : %s" % projects[project][0] + print("Project : %s" % project) + print("Path : %s" % projects[project][0]) if dep_list: for dep in dep_list: - print " - %s" % dep - print "" + print(" - %s" % dep) + print("") - print "-- --" + print("-- --") def PrintBuildOrder(projects, deps): - print "---------------------------------------" - print "Build order " - print "---------------------------------------" - print "-- --" + print("---------------------------------------") + print("Build order ") + print("---------------------------------------") + print("-- --") built = [] for (project, _) in sorted(deps.items()): if project not in built: BuildProject(project, built, projects, deps) - print "-- --" + print("-- --") def PrintVCProj(projects): for project in projects: - print "-------------------------------------" - print "-------------------------------------" - print project - print project - print project - print "-------------------------------------" - print "-------------------------------------" + print("-------------------------------------") + print("-------------------------------------") + print(project) + print(project) + print(project) + print("-------------------------------------") + print("-------------------------------------") project_path = os.path.abspath(os.path.join(os.path.dirname(sys.argv[1]), projects[project][2])) @@ -153,7 +155,7 @@ def PrintVCProj(projects): def main(): # check if we have exactly 1 parameter. if len(sys.argv) < 2: - print 'Usage: %s "c:\\path\\to\\project.sln"' % sys.argv[0] + print('Usage: %s "c:\\path\\to\\project.sln"' % sys.argv[0]) return 1 (projects, deps) = ParseSolution(sys.argv[1]) diff --git a/gyp/tools/pretty_vcproj.py b/gyp/tools/pretty_vcproj.py index 6099bd7cc4..51cfc4e081 100755 --- a/gyp/tools/pretty_vcproj.py +++ b/gyp/tools/pretty_vcproj.py @@ -11,9 +11,10 @@ It outputs the resulting xml to stdout. """ - __author__ = 'nsylvain (Nicolas Sylvain)' +from __future__ import print_function + import os import sys @@ -61,7 +62,7 @@ def get_string(node): def PrettyPrintNode(node, indent=0): if node.nodeType == Node.TEXT_NODE: if node.data.strip(): - print '%s%s' % (' '*indent, node.data.strip()) + print('%s%s' % (' '*indent, node.data.strip())) return if node.childNodes: @@ -73,23 +74,23 @@ def PrettyPrintNode(node, indent=0): # Print the main tag if attr_count == 0: - print '%s<%s>' % (' '*indent, node.nodeName) + print('%s<%s>' % (' '*indent, node.nodeName)) else: - print '%s<%s' % (' '*indent, node.nodeName) + print('%s<%s' % (' '*indent, node.nodeName)) all_attributes = [] for (name, value) in node.attributes.items(): all_attributes.append((name, value)) all_attributes.sort(CmpTuple()) for (name, value) in all_attributes: - print '%s %s="%s"' % (' '*indent, name, value) - print '%s>' % (' '*indent) + print('%s %s="%s"' % (' '*indent, name, value)) + print('%s>' % (' '*indent)) if node.nodeValue: - print '%s %s' % (' '*indent, node.nodeValue) + print('%s %s' % (' '*indent, node.nodeValue)) for sub_node in node.childNodes: PrettyPrintNode(sub_node, indent=indent+2) - print '%s' % (' '*indent, node.nodeName) + print('%s' % (' '*indent, node.nodeName)) def FlattenFilter(node): @@ -283,8 +284,8 @@ def main(argv): # check if we have exactly 1 parameter. if len(argv) < 2: - print ('Usage: %s "c:\\path\\to\\vcproj.vcproj" [key1=value1] ' - '[key2=value2]' % argv[0]) + print(('Usage: %s "c:\\path\\to\\vcproj.vcproj" [key1=value1] ' + '[key2=value2]' % argv[0])) return 1 # Parse the keys diff --git a/lib/Find-VS2017.cs b/lib/Find-VS2017.cs index 87e0a9c9bb..6e7429b771 100644 --- a/lib/Find-VS2017.cs +++ b/lib/Find-VS2017.cs @@ -232,7 +232,9 @@ private static bool CheckInstance(ISetupInstance2 setupInstance2, ref StringBuil string[] parts = id.Substring(Win10SDKPrefix.Length).Split('.'); if (parts.Length > 1 && parts[1] != "Desktop") continue; - Win10SDKVer = Math.Max(Win10SDKVer, UInt32.Parse(parts[0])); + uint foundSdkVer; + if (UInt32.TryParse(parts[0], out foundSdkVer)) + Win10SDKVer = Math.Max(Win10SDKVer, foundSdkVer); } else if (id == "Microsoft.VisualStudio.Component.Windows81SDK") hasWin8SDK = true; else diff --git a/lib/build.js b/lib/build.js index 0445fb6452..b8389bcd7b 100644 --- a/lib/build.js +++ b/lib/build.js @@ -1,18 +1,12 @@ module.exports = exports = build -/** - * Module dependencies. - */ - var fs = require('graceful-fs') - , rm = require('rimraf') , path = require('path') , glob = require('glob') , log = require('npmlog') , which = require('which') , exec = require('child_process').exec - , processRelease = require('./process-release') , win = process.platform === 'win32' exports.usage = 'Invokes `' + (win ? 'msbuild' : 'make') + '` and builds the module' @@ -23,13 +17,14 @@ function build (gyp, argv, callback) { platformMake = 'gmake' } else if (process.platform.indexOf('bsd') !== -1) { platformMake = 'gmake' + } else if (win && argv.length > 0) { + argv = argv.map(function(target) { + return '/t:' + target + }) } - var release = processRelease(argv, gyp, process.version, process.release) - , makeCommand = gyp.opts.make || process.env.MAKE || platformMake + var makeCommand = gyp.opts.make || process.env.MAKE || platformMake , command = win ? 'msbuild' : makeCommand - , buildDir = path.resolve('build') - , configPath = path.resolve(buildDir, 'config.gypi') , jobs = gyp.opts.jobs || process.env.JOBS , buildType , config @@ -43,6 +38,7 @@ function build (gyp, argv, callback) { */ function loadConfigGypi () { + var configPath = path.resolve('build', 'config.gypi') fs.readFile(configPath, 'utf8', function (err, data) { if (err) { if (err.code == 'ENOENT') { @@ -133,7 +129,7 @@ function build (gyp, argv, callback) { var cmd = 'reg query "HKLM\\Software\\Microsoft\\MSBuild\\ToolsVersions" /s' if (process.arch !== 'ia32') cmd += ' /reg:32' - exec(cmd, function (err, stdout, stderr) { + exec(cmd, function (err, stdout) { if (err) { return callback(new Error(err.message + '\n' + notfoundErr)) } @@ -163,7 +159,7 @@ function build (gyp, argv, callback) { ;(function verifyMsbuild () { if (!msbuilds.length) return callback(new Error(notfoundErr)) msbuildPath = path.resolve(msbuilds.pop().path, 'msbuild.exe') - fs.stat(msbuildPath, function (err, stat) { + fs.stat(msbuildPath, function (err) { if (err) { if (err.code == 'ENOENT') { if (msbuilds.length) { @@ -183,13 +179,11 @@ function build (gyp, argv, callback) { }) } - /** * Actually spawn the process and compile the module. */ function doBuild () { - // Enable Verbose build var verbose = log.levels[log.level] <= log.levels.verbose if (!win && verbose) { @@ -249,10 +243,6 @@ function build (gyp, argv, callback) { proc.on('exit', onExit) } - /** - * Invoked after the make/msbuild command exits. - */ - function onExit (code, signal) { if (code !== 0) { return callback(new Error('`' + command + '` failed with exit code: ' + code)) @@ -262,5 +252,4 @@ function build (gyp, argv, callback) { } callback() } - } diff --git a/lib/clean.js b/lib/clean.js index e69164d45a..8172e1c3d3 100644 --- a/lib/clean.js +++ b/lib/clean.js @@ -3,20 +3,13 @@ module.exports = exports = clean exports.usage = 'Removes any generated build files and the "out" dir' -/** - * Module dependencies. - */ - var rm = require('rimraf') var log = require('npmlog') - function clean (gyp, argv, callback) { - // Remove the 'build' dir var buildDir = 'build' log.verbose('clean', 'removing "%s" directory', buildDir) rm(buildDir, callback) - } diff --git a/lib/configure.js b/lib/configure.js index 1351576d12..756107f3fc 100644 --- a/lib/configure.js +++ b/lib/configure.js @@ -5,10 +5,6 @@ module.exports.test = { findPython: findPython, } -/** - * Module dependencies. - */ - var fs = require('graceful-fs') , path = require('path') , log = require('npmlog') @@ -28,7 +24,6 @@ if (win) exports.usage = 'Generates ' + (win ? 'MSVC project files' : 'a Makefile') + ' for the current module' function configure (gyp, argv, callback) { - var python = gyp.opts.python || process.env.PYTHON || 'python2' , buildDir = path.resolve('build') , configNames = [ 'config.gypi', 'common.gypi' ] @@ -46,7 +41,6 @@ function configure (gyp, argv, callback) { }) function getNodeDir () { - // 'python' should be set by now process.env.PYTHON = python @@ -56,7 +50,6 @@ function configure (gyp, argv, callback) { log.verbose('get node dir', 'compiling against specified --nodedir dev files: %s', nodeDir) createBuildDir() - } else { // if no --nodedir specified, ensure node dependencies are installed if ('v' + release.version !== process.version) { @@ -72,9 +65,11 @@ function configure (gyp, argv, callback) { return callback(new Error('Invalid version number: ' + release.version)) } - // ensure that the target node version's dev files are installed - gyp.opts.ensure = true - gyp.commands.install([ release.version ], function (err, version) { + // If the tarball option is set, always remove and reinstall the headers + // into devdir. Otherwise only install if they're not already there. + gyp.opts.ensure = gyp.opts.tarball ? false : true + + gyp.commands.install([ release.version ], function (err) { if (err) return callback(err) log.verbose('get node dir', 'target node version installed:', release.versionDir) nodeDir = path.resolve(gyp.devDir, release.versionDir) @@ -193,7 +188,7 @@ function configure (gyp, argv, callback) { if (!name) return runGyp() var fullPath = path.resolve(name) log.verbose(name, 'checking for gypi file: %s', fullPath) - fs.stat(fullPath, function (err, stat) { + fs.stat(fullPath, function (err) { if (err) { if (err.code == 'ENOENT') { findConfigs() // check next gypi filename @@ -242,28 +237,35 @@ function configure (gyp, argv, callback) { argv.push('-I', config) }) - // for AIX we need to set up the path to the exp file + // For AIX and z/OS we need to set up the path to the exports file // which contains the symbols needed for linking. - // The file will either be in one of the following - // depending on whether it is an installed or - // development environment: - // - the include/node directory - // - the out/Release directory - // - the out/Debug directory - // - the root directory var node_exp_file = undefined - if (process.platform === 'aix') { + if (process.platform === 'aix' || process.platform === 'os390') { + var ext = process.platform === 'aix' ? 'exp' : 'x' var node_root_dir = findNodeDirectory() - var candidates = ['include/node/node.exp', - 'out/Release/node.exp', - 'out/Debug/node.exp', - 'node.exp'] + var candidates = undefined + if (process.platform === 'aix') { + candidates = ['include/node/node', + 'out/Release/node', + 'out/Debug/node', + 'node' + ].map(function(file) { + return file + '.' + ext + }) + } else { + candidates = ['out/Release/obj.target/libnode', + 'out/Debug/obj.target/libnode', + 'lib/libnode' + ].map(function(file) { + return file + '.' + ext + }) + } var logprefix = 'find exports file' node_exp_file = findAccessibleSync(logprefix, node_root_dir, candidates) if (node_exp_file !== undefined) { log.verbose(logprefix, 'Found exports file: %s', node_exp_file) } else { - var msg = msgFormat('Could not find node.exp file in %s', node_root_dir) + var msg = msgFormat('Could not find node.%s file in %s', ext, node_root_dir) log.error(logprefix, 'Could not find exports file') return callback(new Error(msg)) } @@ -273,7 +275,7 @@ function configure (gyp, argv, callback) { var gyp_script = path.resolve(__dirname, '..', 'gyp', 'gyp_main.py') var addon_gypi = path.resolve(__dirname, '..', 'addon.gypi') var common_gypi = path.resolve(nodeDir, 'include/node/common.gypi') - fs.stat(common_gypi, function (err, stat) { + fs.stat(common_gypi, function (err) { if (err) common_gypi = path.resolve(nodeDir, 'common.gypi') @@ -292,7 +294,7 @@ function configure (gyp, argv, callback) { argv.push('-Dlibrary=shared_library') argv.push('-Dvisibility=default') argv.push('-Dnode_root_dir=' + nodeDir) - if (process.platform === 'aix') { + if (process.platform === 'aix' || process.platform === 'os390') { argv.push('-Dnode_exp_file=' + node_exp_file) } argv.push('-Dnode_gyp_dir=' + nodeGypDir) @@ -327,11 +329,7 @@ function configure (gyp, argv, callback) { }) } - /** - * Called when the `gyp` child process exits. - */ - - function onCpExit (code, signal) { + function onCpExit (code) { if (code !== 0) { callback(new Error('`gyp` failed with exit code: ' + code)) } else { @@ -439,7 +437,7 @@ PythonFinder.prototype = { }, checkPythonVersion: function checkPythonVersion () { - var args = ['-c', 'import platform; print(platform.python_version());'] + var args = ['-c', 'import sys; print "%s.%s.%s" % sys.version_info[:3];'] var env = extend({}, this.env) env.TERM = 'dumb' @@ -451,14 +449,6 @@ PythonFinder.prototype = { '`%s -c "' + args[1] + '"` returned: %j', this.python, stdout) var version = stdout.trim() - if (~version.indexOf('+')) { - this.log.silly('stripping "+" sign(s) from version') - version = version.replace(/\+/g, '') - } - if (~version.indexOf('rc')) { - this.log.silly('stripping "rc" identifier from version') - version = version.replace(/rc(.*)$/ig, '') - } var range = semver.Range('>=2.5.0 <3.0.0') var valid = false try { @@ -502,7 +492,7 @@ PythonFinder.prototype = { } var pythonPath = this.resolve(rootDir, 'Python27', 'python.exe') this.log.verbose('ensuring that file exists:', pythonPath) - this.stat(pythonPath, function (err, stat) { + this.stat(pythonPath, function (err) { if (err) { if (err.code == 'ENOENT') { this.failNoPython() diff --git a/lib/find-node-directory.js b/lib/find-node-directory.js index 3aee8a109a..524909581f 100644 --- a/lib/find-node-directory.js +++ b/lib/find-node-directory.js @@ -1,7 +1,7 @@ var path = require('path') , log = require('npmlog') -function findNodeDirectory(scriptLocation, processObj) { +module.exports = function findNodeDirectory(scriptLocation, processObj) { // set dirname and process if not passed in // this facilitates regression tests if (scriptLocation === undefined) { @@ -57,5 +57,3 @@ function findNodeDirectory(scriptLocation, processObj) { } return node_root_dir } - -module.exports = findNodeDirectory diff --git a/lib/find-vs2017.js b/lib/find-vs2017.js index 8c79e9ec9b..8655d7c931 100644 --- a/lib/find-vs2017.js +++ b/lib/find-vs2017.js @@ -2,13 +2,13 @@ var log = require('npmlog') , execFile = require('child_process').execFile , path = require('path') -function findVS2017(callback) { +module.exports = function findVS2017(callback) { var ps = path.join(process.env.SystemRoot, 'System32', 'WindowsPowerShell', 'v1.0', 'powershell.exe') var csFile = path.join(__dirname, 'Find-VS2017.cs') - var psArgs = ['-ExecutionPolicy', 'Unrestricted', '-Command', - '&{Add-Type -Path \'' + csFile + - '\'; [VisualStudioConfiguration.Main]::Query()}'] + var psArgs = ['-ExecutionPolicy', 'Unrestricted', '-NoProfile', + '-Command', '&{Add-Type -Path \'' + csFile + '\';' + + '[VisualStudioConfiguration.Main]::Query()}'] log.silly('find vs2017', 'Running', ps, psArgs) var child = execFile(ps, psArgs, { encoding: 'utf8' }, @@ -42,5 +42,3 @@ function findVS2017(callback) { child.stdin.end() } - -module.exports = findVS2017 diff --git a/lib/install.js b/lib/install.js index fa2e1c5430..eea7618a01 100644 --- a/lib/install.js +++ b/lib/install.js @@ -1,32 +1,28 @@ +module.exports = exports = function (gyp, argv, callback) { + return install(fs, gyp, argv, callback) +} -module.exports = exports = install - -module.exports.test = { download: download, readCAFile: readCAFile } +module.exports.test = { + download: download, + install: install, + readCAFile: readCAFile, +} exports.usage = 'Install node development files for the specified node version.' -/** - * Module dependencies. - */ - var fs = require('graceful-fs') , osenv = require('osenv') , tar = require('tar') - , rm = require('rimraf') , path = require('path') , crypto = require('crypto') - , zlib = require('zlib') , log = require('npmlog') , semver = require('semver') - , fstream = require('fstream') , request = require('request') - , minimatch = require('minimatch') , mkdir = require('mkdirp') , processRelease = require('./process-release') , win = process.platform == 'win32' -function install (gyp, argv, callback) { - +function install (fs, gyp, argv, callback) { var release = processRelease(argv, gyp, process.version, process.release) // ensure no double-callbacks happen @@ -36,7 +32,7 @@ function install (gyp, argv, callback) { if (err) { log.warn('install', 'got an error, rolling back install') // roll-back the install if anything went wrong - gyp.commands.remove([ release.versionDir ], function (err2) { + gyp.commands.remove([ release.versionDir ], function () { callback(err) }) } else { @@ -78,13 +74,13 @@ function install (gyp, argv, callback) { // check if it is already installed, and only install when needed if (gyp.opts.ensure) { log.verbose('install', '--ensure was passed, so won\'t reinstall if already installed') - fs.stat(devDir, function (err, stat) { + fs.stat(devDir, function (err) { if (err) { if (err.code == 'ENOENT') { log.verbose('install', 'version not already installed, continuing with install', release.version) go() } else if (err.code == 'EACCES') { - eaccesFallback() + eaccesFallback(err) } else { cb(err) } @@ -122,14 +118,13 @@ function install (gyp, argv, callback) { } function go () { - log.verbose('ensuring nodedir is created', devDir) // first create the dir for the node dev files mkdir(devDir, function (err, created) { if (err) { if (err.code == 'EACCES') { - eaccesFallback() + eaccesFallback(err) } else { cb(err) } @@ -144,41 +139,32 @@ function install (gyp, argv, callback) { var tarPath = gyp.opts.tarball var badDownload = false , extractCount = 0 - , gunzip = zlib.createGunzip() - , extracter = tar.Extract({ path: devDir, strip: 1, filter: isValid }) var contentShasums = {} var expectShasums = {} // checks if a file to be extracted from the tarball is valid. // only .h header files and the gyp files get extracted - function isValid () { - var name = this.path.substring(devDir.length + 1) - var isValid = valid(name) - if (name === '' && this.type === 'Directory') { - // the first directory entry is ok - return true - } + function isValid (path) { + var isValid = valid(path) if (isValid) { - log.verbose('extracted file from tarball', name) + log.verbose('extracted file from tarball', path) extractCount++ } else { // invalid - log.silly('ignoring from tarball', name) + log.silly('ignoring from tarball', path) } return isValid } - gunzip.on('error', cb) - extracter.on('error', cb) - extracter.on('end', afterTarball) - - // download the tarball, gunzip and extract! - + // download the tarball and extract! if (tarPath) { - var input = fs.createReadStream(tarPath) - input.pipe(gunzip).pipe(extracter) - return + return tar.extract({ + file: tarPath, + strip: 1, + filter: isValid, + cwd: devDir + }).then(afterTarball, cb) } try { @@ -218,7 +204,11 @@ function install (gyp, argv, callback) { }) // start unzipping and untaring - req.pipe(gunzip).pipe(extracter) + res.pipe(tar.extract({ + strip: 1, + cwd: devDir, + filter: isValid + }).on('close', afterTarball).on('error', cb)) }) // invoked after the tarball has finished being extracted @@ -274,8 +264,6 @@ function install (gyp, argv, callback) { function downloadShasums(done) { log.verbose('check download content checksum, need to download `SHASUMS256.txt`...') - var shasumsPath = path.resolve(devDir, 'SHASUMS256.txt') - log.verbose('checksum url', release.shasumsUrl) try { var req = download(gyp, process.env, release.shasumsUrl) @@ -396,8 +384,8 @@ function install (gyp, argv, callback) { function valid (file) { // header files - return minimatch(file, '*.h', { matchBase: true }) || - minimatch(file, '*.gypi', { matchBase: true }) + var extname = path.extname(file); + return extname === '.h' || extname === '.gypi'; } /** @@ -409,7 +397,9 @@ function install (gyp, argv, callback) { * the compilation will succeed... */ - function eaccesFallback () { + function eaccesFallback (err) { + var noretry = '--node_gyp_internal_noretry' + if (-1 !== argv.indexOf(noretry)) return cb(err) var tmpdir = osenv.tmpdir() gyp.devDir = path.resolve(tmpdir, '.node-gyp') log.warn('EACCES', 'user "%s" does not have permission to access the dev dir "%s"', osenv.user(), devDir) @@ -418,7 +408,7 @@ function install (gyp, argv, callback) { log.verbose('tmpdir == cwd', 'automatically will remove dev files after to save disk space') gyp.todo.push({ name: 'remove', args: argv }) } - gyp.commands.install(argv, cb) + gyp.commands.install([noretry].concat(argv), cb) } } diff --git a/lib/list.js b/lib/list.js index 9d680a56a4..90f6447180 100644 --- a/lib/list.js +++ b/lib/list.js @@ -3,20 +3,13 @@ module.exports = exports = list exports.usage = 'Prints a listing of the currently installed node development files' -/** - * Module dependencies. - */ - var fs = require('graceful-fs') - , path = require('path') , log = require('npmlog') function list (gyp, args, callback) { - var devDir = gyp.devDir log.verbose('list', 'using node-gyp dir:', devDir) - // readdir() the node-gyp dir fs.readdir(devDir, onreaddir) function onreaddir (err, versions) { diff --git a/lib/node-gyp.js b/lib/node-gyp.js index a841161e32..8dd551bf3d 100644 --- a/lib/node-gyp.js +++ b/lib/node-gyp.js @@ -1,16 +1,7 @@ -/** - * Module exports. - */ - module.exports = exports = gyp -/** - * Module dependencies. - */ - -var fs = require('graceful-fs') - , path = require('path') +var path = require('path') , nopt = require('nopt') , log = require('npmlog') , child_process = require('child_process') @@ -35,10 +26,6 @@ var fs = require('graceful-fs') // differentiate node-gyp's logs from npm's log.heading = 'gyp' -/** - * The `gyp` function. - */ - function gyp () { return new Gyp() } @@ -213,4 +200,3 @@ Object.defineProperty(proto, 'version', { } , enumerable: true }) - diff --git a/lib/process-release.js b/lib/process-release.js index 0d177f1c93..4805fcc7d1 100644 --- a/lib/process-release.js +++ b/lib/process-release.js @@ -9,7 +9,7 @@ var semver = require('semver') , bitsreV3 = /\/win-(x86|ia32|x64)\// // io.js v3.x.x shipped with "ia32" but should // have been "x86" -// Captures all the logic required to determine download URLs, local directory and +// Captures all the logic required to determine download URLs, local directory and // file names. Inputs come from command-line switches (--target, --dist-url), // `process.version` and `process.release` where it exists. function processRelease (argv, gyp, defaultVersion, defaultRelease) { @@ -88,35 +88,22 @@ function processRelease (argv, gyp, defaultVersion, defaultRelease) { baseUrl = url.resolve(defaultRelease.headersUrl, './') libUrl32 = resolveLibUrl(name, defaultRelease.libUrl || baseUrl || distBaseUrl, 'x86', versionSemver.major) libUrl64 = resolveLibUrl(name, defaultRelease.libUrl || baseUrl || distBaseUrl, 'x64', versionSemver.major) - - return { - version: version, - semver: versionSemver, - name: name, - baseUrl: baseUrl, - tarballUrl: defaultRelease.headersUrl, - shasumsUrl: url.resolve(baseUrl, 'SHASUMS256.txt'), - versionDir: (name !== 'node' ? name + '-' : '') + version, - libUrl32: libUrl32, - libUrl64: libUrl64, - libPath32: normalizePath(path.relative(url.parse(baseUrl).path, url.parse(libUrl32).path)), - libPath64: normalizePath(path.relative(url.parse(baseUrl).path, url.parse(libUrl64).path)) - } + tarballUrl = defaultRelease.headersUrl + } else { + // older versions without process.release are captured here and we have to make + // a lot of assumptions, additionally if you --target=x.y.z then we can't use the + // current process.release + baseUrl = distBaseUrl + libUrl32 = resolveLibUrl(name, baseUrl, 'x86', versionSemver.major) + libUrl64 = resolveLibUrl(name, baseUrl, 'x64', versionSemver.major) + + // making the bold assumption that anything with a version number >3.0.0 will + // have a *-headers.tar.gz file in its dist location, even some frankenstein + // custom version + canGetHeaders = semver.satisfies(versionSemver, headersTarballRange) + tarballUrl = url.resolve(baseUrl, name + '-v' + version + (canGetHeaders ? '-headers' : '') + '.tar.gz') } - // older versions without process.release are captured here and we have to make - // a lot of assumptions, additionally if you --target=x.y.z then we can't use the - // current process.release - - baseUrl = distBaseUrl - libUrl32 = resolveLibUrl(name, baseUrl, 'x86', versionSemver.major) - libUrl64 = resolveLibUrl(name, baseUrl, 'x64', versionSemver.major) - // making the bold assumption that anything with a version number >3.0.0 will - // have a *-headers.tar.gz file in its dist location, even some frankenstein - // custom version - canGetHeaders = semver.satisfies(versionSemver, headersTarballRange) - tarballUrl = url.resolve(baseUrl, name + '-v' + version + (canGetHeaders ? '-headers' : '') + '.tar.gz') - return { version: version, semver: versionSemver, diff --git a/lib/rebuild.js b/lib/rebuild.js index 4c6f472aa7..dc8025a7b0 100644 --- a/lib/rebuild.js +++ b/lib/rebuild.js @@ -4,7 +4,6 @@ module.exports = exports = rebuild exports.usage = 'Runs "clean", "configure" and "build" all at once' function rebuild (gyp, argv, callback) { - gyp.todo.push( { name: 'clean', args: [] } , { name: 'configure', args: argv } diff --git a/lib/remove.js b/lib/remove.js index eb80981b88..46a1a3d4c2 100644 --- a/lib/remove.js +++ b/lib/remove.js @@ -3,10 +3,6 @@ module.exports = exports = remove exports.usage = 'Removes the node development files for the specified version' -/** - * Module dependencies. - */ - var fs = require('fs') , rm = require('rimraf') , path = require('path') @@ -14,7 +10,6 @@ var fs = require('fs') , semver = require('semver') function remove (gyp, argv, callback) { - var devDir = gyp.devDir log.verbose('remove', 'using node-gyp dir:', devDir) @@ -36,7 +31,7 @@ function remove (gyp, argv, callback) { log.verbose('remove', 'removing development files for version:', version) // first check if its even installed - fs.stat(versionPath, function (err, stat) { + fs.stat(versionPath, function (err) { if (err) { if (err.code == 'ENOENT') { callback(null, 'version was already uninstalled: ' + version) @@ -48,5 +43,4 @@ function remove (gyp, argv, callback) { // Go ahead and delete the dir rm(versionPath, callback) }) - } diff --git a/package.json b/package.json index a66a546889..0839aff48e 100644 --- a/package.json +++ b/package.json @@ -11,7 +11,7 @@ "bindings", "gyp" ], - "version": "3.6.2", + "version": "4.0.0", "installVersion": 9, "author": "Nathan Rajlich (http://tootallnate.net)", "repository": { @@ -22,30 +22,31 @@ "bin": "./bin/node-gyp.js", "main": "./lib/node-gyp.js", "dependencies": { - "fstream": "^1.0.0", "glob": "^7.0.3", "graceful-fs": "^4.1.2", - "minimatch": "^3.0.2", "mkdirp": "^0.5.0", "nopt": "2 || 3", "npmlog": "0 || 1 || 2 || 3 || 4", "osenv": "0", - "request": "2", + "request": "^2.87.0", "rimraf": "2", "semver": "~5.3.0", - "tar": "^2.0.0", + "tar": "^3.1.3", "which": "1" }, "engines": { - "node": ">= 0.8.0" + "node": ">= 4.0.0" }, "devDependencies": { - "tape": "~4.2.0", + "babel-eslint": "^8.2.5", "bindings": "~1.2.1", + "eslint": "^5.0.1", "nan": "^2.0.0", - "require-inject": "~1.3.0" + "require-inject": "~1.3.0", + "tape": "~4.2.0" }, "scripts": { - "test": "tape test/test-*" + "lint": "eslint bin lib test", + "test": "npm run lint && tape test/test-*" } } diff --git a/test/fixtures/test-charmap.py b/test/fixtures/test-charmap.py new file mode 100644 index 0000000000..b752f0bbbf --- /dev/null +++ b/test/fixtures/test-charmap.py @@ -0,0 +1,23 @@ +from __future__ import print_function +import sys +import locale + +reload(sys) + +def main(): + encoding = locale.getdefaultlocale()[1] + if not encoding: + return False + + sys.setdefaultencoding(encoding) + textmap = { + 'cp936': u'\u4e2d\u6587', + 'cp1252': u'Lat\u012Bna', + 'cp932': u'\u306b\u307b\u3093\u3054' + } + if encoding in textmap: + print(textmap[encoding]) + return True + +if __name__ == '__main__': + print(main()) diff --git a/test/process-exec-sync.js b/test/process-exec-sync.js new file mode 100644 index 0000000000..13daf308a3 --- /dev/null +++ b/test/process-exec-sync.js @@ -0,0 +1,138 @@ +'use strict' + +var fs = require('graceful-fs') +var child_process = require('child_process') + +if (!String.prototype.startsWith) { + String.prototype.startsWith = function(search, pos) { + return this.substr(!pos || pos < 0 ? 0 : +pos, search.length) === search + } +} + +function processExecSync(file, args, options) { + var child, error, timeout, tmpdir, command + command = makeCommand(file, args) + + /* + this function emulates child_process.execSync for legacy node <= 0.10.x + derived from https://github.com/gvarsanyi/sync-exec/blob/master/js/sync-exec.js + */ + + options = options || {} + // init timeout + timeout = Date.now() + options.timeout + // init tmpdir + var os_temp_base = '/tmp' + var os = determine_os() + os_temp_base = '/tmp' + + if (process.env.TMP) { + os_temp_base = process.env.TMP + } + + if (os_temp_base[os_temp_base.length - 1] !== '/') { + os_temp_base += '/' + } + + tmpdir = os_temp_base + 'processExecSync.' + Date.now() + Math.random() + fs.mkdirSync(tmpdir) + + // init command + if (os === 'linux') { + command = '(' + command + ' > ' + tmpdir + '/stdout 2> ' + tmpdir + + '/stderr); echo $? > ' + tmpdir + '/status' + } else { + command = '(' + command + ' > ' + tmpdir + '/stdout 2> ' + tmpdir + + '/stderr) | echo %errorlevel% > ' + tmpdir + '/status | exit' + } + + // init child + child = child_process.exec(command, options) + + var maxTry = 100000 // increases the test time by 6 seconds on win-2016-node-0.10 + var tryCount = 0 + while (tryCount < maxTry) { + try { + var x = fs.readFileSync(tmpdir + '/status') + if (x.toString() === '0') { + break + } + } catch (ignore) {} + tryCount++ + if (Date.now() > timeout) { + error = child + break + } + } + + ['stdout', 'stderr', 'status'].forEach(function (file) { + child[file] = fs.readFileSync(tmpdir + '/' + file, options.encoding) + setTimeout(unlinkFile, 500, tmpdir + '/' + file) + }) + + child.status = Number(child.status) + if (child.status !== 0) { + error = child + } + + try { + fs.rmdirSync(tmpdir) + } catch (ignore) {} + if (error) { + throw error + } + return child.stdout +} + +function makeCommand(file, args) { + var command, quote + command = file + if (args.length > 0) { + for(var i in args) { + command = command + ' ' + if (args[i][0] === '-') { + command = command + args[i] + } else { + if (!quote) { + command = command + '\"' + quote = true + } + command = command + args[i] + if (quote) { + if (args.length === (parseInt(i) + 1)) { + command = command + '\"' + } + } + } + } + } + return command +} + +function determine_os() { + var os = '' + var tmpVar = '' + if (process.env.OSTYPE) { + tmpVar = process.env.OSTYPE + } else if (process.env.OS) { + tmpVar = process.env.OS + } else { + //default is linux + tmpVar = 'linux' + } + + if (tmpVar.startsWith('linux')) { + os = 'linux' + } + if (tmpVar.startsWith('win')) { + os = 'win' + } + + return os +} + +function unlinkFile(file) { + fs.unlinkSync(file) +} + +module.exports = processExecSync diff --git a/test/test-addon.js b/test/test-addon.js index c2a71f4498..89350effc4 100644 --- a/test/test-addon.js +++ b/test/test-addon.js @@ -1,10 +1,35 @@ 'use strict' var test = require('tape') -var execFile = require('child_process').execFile var path = require('path') +var fs = require('graceful-fs') +var child_process = require('child_process') var addonPath = path.resolve(__dirname, 'node_modules', 'hello_world') var nodeGyp = path.resolve(__dirname, '..', 'bin', 'node-gyp.js') +var execFileSync = child_process.execFileSync || require('./process-exec-sync') +var execFile = child_process.execFile + +function runHello() { + var testCode = "console.log(require('hello_world').hello())" + return execFileSync(process.execPath, ['-e', testCode], { cwd: __dirname }).toString() +} + +function getEncoding() { + var code = 'import locale;print locale.getdefaultlocale()[1]' + return execFileSync('python', ['-c', code]).toString().trim() +} + +function checkCharmapValid() { + var data + try { + data = execFileSync('python', ['fixtures/test-charmap.py'], + { cwd: __dirname }) + } catch (err) { + return false + } + var lines = data.toString().trim().split('\n') + return lines.pop() === 'True' +} test('build simple addon', function (t) { t.plan(3) @@ -16,12 +41,72 @@ test('build simple addon', function (t) { var lastLine = logLines[logLines.length-1] t.strictEqual(err, null) t.strictEqual(lastLine, 'gyp info ok', 'should end in ok') + t.strictEqual(runHello().trim(), 'world') + }) + proc.stdout.setEncoding('utf-8') + proc.stderr.setEncoding('utf-8') +}) + +test('build simple addon in path with non-ascii characters', function (t) { + t.plan(1) + + if (!checkCharmapValid()) { + return t.skip('python console app can\'t encode non-ascii character.') + } + + var testDirNames = { + 'cp936': '文件夹', + 'cp1252': 'Latīna', + 'cp932': 'フォルダ' + } + // Select non-ascii characters by current encoding + var testDirName = testDirNames[getEncoding()] + // If encoding is UTF-8 or other then no need to test + if (!testDirName) { + return t.skip('no need to test') + } + + t.plan(3) + + var data, configPath = path.join(addonPath, 'build', 'config.gypi') + try { + data = fs.readFileSync(configPath, 'utf8') + } catch (err) { + t.error(err) + return + } + var config = JSON.parse(data.replace(/\#.+\n/, '')) + var nodeDir = config.variables.nodedir + var testNodeDir = path.join(addonPath, testDirName) + // Create symbol link to path with non-ascii characters + try { + fs.symlinkSync(nodeDir, testNodeDir, 'dir') + } catch (err) { + switch (err.code) { + case 'EEXIST': break + case 'EPERM': + t.error(err, 'Please try to running console as an administrator') + return + default: + t.error(err) + return + } + } + + var cmd = [nodeGyp, 'rebuild', '-C', addonPath, + '--loglevel=verbose', '-nodedir=' + testNodeDir] + var proc = execFile(process.execPath, cmd, function (err, stdout, stderr) { try { - var binding = require('hello_world') - t.strictEqual(binding.hello(), 'world') - } catch (error) { - t.error(error, 'load module') + fs.unlink(testNodeDir) + } catch (err) { + t.error(err) } + + var logLines = stderr.toString().trim().split(/\r?\n/) + var lastLine = logLines[logLines.length-1] + t.strictEqual(err, null) + t.strictEqual(lastLine, 'gyp info ok', 'should end in ok') + t.strictEqual(runHello().trim(), 'world') }) proc.stdout.setEncoding('utf-8') proc.stderr.setEncoding('utf-8') diff --git a/test/test-configure-python.js b/test/test-configure-python.js index f235bdbba1..07cdce2b17 100644 --- a/test/test-configure-python.js +++ b/test/test-configure-python.js @@ -6,8 +6,8 @@ var gyp = require('../lib/node-gyp') var requireInject = require('require-inject') var configure = requireInject('../lib/configure', { 'graceful-fs': { - 'openSync': function (file, mode) { return 0; }, - 'closeSync': function (fd) { }, + 'openSync': function () { return 0; }, + 'closeSync': function () { }, 'writeFile': function (file, data, cb) { cb() }, 'stat': function (file, cb) { cb(null, {}) } } diff --git a/test/test-find-accessible-sync.js b/test/test-find-accessible-sync.js index d336243dd0..114403e278 100644 --- a/test/test-find-accessible-sync.js +++ b/test/test-find-accessible-sync.js @@ -5,7 +5,7 @@ var path = require('path') var requireInject = require('require-inject') var configure = requireInject('../lib/configure', { 'graceful-fs': { - 'closeSync': function (fd) { return undefined }, + 'closeSync': function () { return undefined }, 'openSync': function (path) { if (readableFiles.some(function (f) { return f === path} )) { return 0 diff --git a/test/test-find-python.js b/test/test-find-python.js index 2d9f171c57..250e37f4fe 100644 --- a/test/test-find-python.js +++ b/test/test-find-python.js @@ -62,7 +62,7 @@ test('find python - python', function (t) { } f.execFile = function(program, args, opts, cb) { t.strictEqual(program, 'python') - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '2.7.0') } f.checkPython() @@ -83,12 +83,12 @@ test('find python - python too old', function (t) { } f.execFile = function(program, args, opts, cb) { t.strictEqual(program, 'python') - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '2.3.4') } f.checkPython() - function done(err, python) { + function done(err) { t.ok(/is not supported by gyp/.test(err)) } }) @@ -103,12 +103,12 @@ test('find python - python too new', function (t) { } f.execFile = function(program, args, opts, cb) { t.strictEqual(program, 'python') - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '3.0.0') } f.checkPython() - function done(err, python) { + function done(err) { t.ok(/is not supported by gyp/.test(err)) } }) @@ -123,7 +123,7 @@ test('find python - no python', function (t) { } f.checkPython() - function done(err, python) { + function done(err) { t.ok(/Can't find Python executable/.test(err)) } }) @@ -142,7 +142,7 @@ test('find python - no python2', function (t) { } f.execFile = function(program, args, opts, cb) { t.strictEqual(program, 'python') - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '2.7.0') } f.checkPython() @@ -170,7 +170,7 @@ test('find python - no python2, no python, unix', function (t) { } f.checkPython() - function done(err, python) { + function done(err) { t.ok(/Can't find Python executable/.test(err)) } }) @@ -189,7 +189,7 @@ test('find python - no python, use python launcher', function (t) { f.execFile = function(program, args, opts, cb) { f.execFile = function(program, args, opts, cb) { t.strictEqual(program, 'Z:\\snake.exe') - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '2.7.0') } t.strictEqual(program, 'py.exe') @@ -220,7 +220,7 @@ test('find python - python 3, use python launcher', function (t) { f.execFile = function(program, args, opts, cb) { f.execFile = function(program, args, opts, cb) { t.strictEqual(program, 'Z:\\snake.exe') - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '2.7.0') } t.strictEqual(program, 'py.exe') @@ -229,7 +229,7 @@ test('find python - python 3, use python launcher', function (t) { cb(null, 'Z:\\snake.exe') } t.strictEqual(program, 'python') - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '3.0.0') } f.checkPython() @@ -257,7 +257,7 @@ test('find python - python 3, use python launcher, python 2 too old', f.execFile = function(program, args, opts, cb) { f.execFile = function(program, args, opts, cb) { t.strictEqual(program, 'Z:\\snake.exe') - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '2.3.4') } t.strictEqual(program, 'py.exe') @@ -266,12 +266,12 @@ test('find python - python 3, use python launcher, python 2 too old', cb(null, 'Z:\\snake.exe') } t.strictEqual(program, 'python') - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '3.0.0') } f.checkPython() - function done(err, python) { + function done(err) { t.ok(/is not supported by gyp/.test(err)) } }) @@ -291,7 +291,7 @@ test('find python - no python, no python launcher, good guess', function (t) { f.execFile = function(program, args, opts, cb) { f.execFile = function(program, args, opts, cb) { t.ok(re.test(program)) - t.ok(/import platform/.test(args[1])) + t.ok(/import sys/.test(args[1])) cb(null, '2.7.0') } t.strictEqual(program, 'py.exe') @@ -333,7 +333,7 @@ test('find python - no python, no python launcher, bad guess', function (t) { } f.checkPython() - function done(err, python) { + function done(err) { t.ok(/Can't find Python executable/.test(err)) } }) diff --git a/test/test-install.js b/test/test-install.js new file mode 100644 index 0000000000..f647326a7f --- /dev/null +++ b/test/test-install.js @@ -0,0 +1,37 @@ +'use strict' + +var test = require('tape') +var install = require('../lib/install').test.install + +test('EACCES retry once', function (t) { + t.plan(3) + + var fs = {} + fs.stat = function (path, cb) { + var err = new Error() + err.code = 'EACCES' + cb(err) + t.ok(true); + } + + + var gyp = {} + gyp.devDir = __dirname + gyp.opts = {} + gyp.opts.ensure = true + gyp.commands = {} + gyp.commands.install = function (argv, cb) { + install(fs, gyp, argv, cb) + } + gyp.commands.remove = function (argv, cb) { + cb() + } + + gyp.commands.install([], function (err) { + t.ok(true) + if (/"pre" versions of node cannot be installed/.test(err.message)) { + t.ok(true) + t.ok(true) + } + }) +}) diff --git a/tools/gyp/pylib/gyp/generator/compile_commands_json.py b/tools/gyp/pylib/gyp/generator/compile_commands_json.py new file mode 100644 index 0000000000..575db63c4e --- /dev/null +++ b/tools/gyp/pylib/gyp/generator/compile_commands_json.py @@ -0,0 +1,115 @@ +# Copyright (c) 2016 Ben Noordhuis . All rights reserved. +# Use of this source code is governed by a BSD-style license that can be +# found in the LICENSE file. + +import gyp.common +import gyp.xcode_emulation +import json +import os + +generator_additional_non_configuration_keys = [] +generator_additional_path_sections = [] +generator_extra_sources_for_rules = [] +generator_filelist_paths = None +generator_supports_multiple_toolsets = True +generator_wants_sorted_dependencies = False + +# Lifted from make.py. The actual values don't matter much. +generator_default_variables = { + 'CONFIGURATION_NAME': '$(BUILDTYPE)', + 'EXECUTABLE_PREFIX': '', + 'EXECUTABLE_SUFFIX': '', + 'INTERMEDIATE_DIR': '$(obj).$(TOOLSET)/$(TARGET)/geni', + 'PRODUCT_DIR': '$(builddir)', + 'RULE_INPUT_DIRNAME': '%(INPUT_DIRNAME)s', + 'RULE_INPUT_EXT': '$(suffix $<)', + 'RULE_INPUT_NAME': '$(notdir $<)', + 'RULE_INPUT_PATH': '$(abspath $<)', + 'RULE_INPUT_ROOT': '%(INPUT_ROOT)s', + 'SHARED_INTERMEDIATE_DIR': '$(obj)/gen', + 'SHARED_LIB_PREFIX': 'lib', + 'STATIC_LIB_PREFIX': 'lib', + 'STATIC_LIB_SUFFIX': '.a', +} + + +def IsMac(params): + return 'mac' == gyp.common.GetFlavor(params) + + +def CalculateVariables(default_variables, params): + default_variables.setdefault('OS', gyp.common.GetFlavor(params)) + + +def AddCommandsForTarget(cwd, target, params, per_config_commands): + output_dir = params['generator_flags']['output_dir'] + for configuration_name, configuration in target['configurations'].iteritems(): + builddir_name = os.path.join(output_dir, configuration_name) + + if IsMac(params): + xcode_settings = gyp.xcode_emulation.XcodeSettings(target) + cflags = xcode_settings.GetCflags(configuration_name) + cflags_c = xcode_settings.GetCflagsC(configuration_name) + cflags_cc = xcode_settings.GetCflagsCC(configuration_name) + else: + cflags = configuration.get('cflags', []) + cflags_c = configuration.get('cflags_c', []) + cflags_cc = configuration.get('cflags_cc', []) + + cflags_c = cflags + cflags_c + cflags_cc = cflags + cflags_cc + + defines = configuration.get('defines', []) + defines = ['-D' + s for s in defines] + + # TODO(bnoordhuis) Handle generated source files. + sources = target.get('sources', []) + sources = [s for s in sources if s.endswith('.c') or s.endswith('.cc')] + + def resolve(filename): + return os.path.abspath(os.path.join(cwd, filename)) + + # TODO(bnoordhuis) Handle generated header files. + include_dirs = configuration.get('include_dirs', []) + include_dirs = [s for s in include_dirs if not s.startswith('$(obj)')] + includes = ['-I' + resolve(s) for s in include_dirs] + + defines = gyp.common.EncodePOSIXShellList(defines) + includes = gyp.common.EncodePOSIXShellList(includes) + cflags_c = gyp.common.EncodePOSIXShellList(cflags_c) + cflags_cc = gyp.common.EncodePOSIXShellList(cflags_cc) + + commands = per_config_commands.setdefault(configuration_name, []) + for source in sources: + file = resolve(source) + isc = source.endswith('.c') + cc = 'cc' if isc else 'c++' + cflags = cflags_c if isc else cflags_cc + command = ' '.join((cc, defines, includes, cflags, + '-c', gyp.common.EncodePOSIXShellArgument(file))) + commands.append(dict(command=command, directory=output_dir, file=file)) + + +def GenerateOutput(target_list, target_dicts, data, params): + per_config_commands = {} + for qualified_target, target in target_dicts.iteritems(): + build_file, target_name, toolset = ( + gyp.common.ParseQualifiedTarget(qualified_target)) + if IsMac(params): + settings = data[build_file] + gyp.xcode_emulation.MergeGlobalXcodeSettingsToSpec(settings, target) + cwd = os.path.dirname(build_file) + AddCommandsForTarget(cwd, target, params, per_config_commands) + + output_dir = params['generator_flags']['output_dir'] + for configuration_name, commands in per_config_commands.iteritems(): + filename = os.path.join(output_dir, + configuration_name, + 'compile_commands.json') + gyp.common.EnsureDirExists(filename) + fp = open(filename, 'w') + json.dump(commands, fp=fp, indent=0, check_circular=False) + + +def PerformBuild(data, configurations, params): + pass