Skip to content

Latest commit

 

History

History
131 lines (96 loc) · 5.06 KB

CHANGES.md

File metadata and controls

131 lines (96 loc) · 5.06 KB

Change Log

Unreleased

Nothing notable yet.

0.5.0 - 2016-07-20

Added

  • #118: You can now specify --hdfs-download-source (or the equivalent in your config file) to tell Flintrock to download Hadoop from a specific URL when launching your cluster.
  • #125: You can now specify --spark-download-source (or the equivalent in your config file) to tell Flintrock to download Spark from a specific URL when launching your cluster.
  • #112: You can now specify --ec2-security-group to associate additional security groups with your cluster on launch.

Changed

  • #103, #114: Flintrock now opens port 6066 and 7077 so local clients like Apache Zeppelin can connect directly to the Spark master on the cluster.
  • #122: Flintrock now automatically adds executables like spark-submit, pyspark, and hdfs to the default PATH, so they're available to call right when you login to the cluster.

0.4.0 - 2016-03-27

Added

  • #98, #99: You can now specify latest for --spark-git-commit and Flintrock will automatically build Spark on your cluster at the latest commit. This feature is only available for Spark repos hosted on GitHub.
  • #94: Flintrock now supports launching clusters into non-default VPCs.

Changed

  • #86: Flintrock now correctly catches when spot requests fail and bubbles up an appropriate error message.
  • #93, #97: Fixed the ability to build Spark from git. (It was broken for recent commits.)
  • #96, #100: Flintrock launches should now work correctly whether the default Python on the cluster is Python 2.7 or Python 3.4+.

0.3.0 - 2016-02-14

Changed

  • eca59fc, 3cf6ee6: Tweaked a few things so that Flintrock can launch 200+ node clusters without hitting certain limits.

0.2.0 - 2016-02-07

Added

  • b00fd12: Added --assume-yes option to the launch command. Use --assume-yes to tell Flintrock to automatically destroy the cluster if there are problems during launch.

Changed

  • #69: Automatically retry Hadoop download from flaky Apache mirrors.
  • 0df7004: Delete unneeded security group after a cluster is destroyed.
  • 244f734: Default HDFS not to install. Going forward, Spark will be the only service that Flintrock installs by default. Defaults can easily be changed via Flintrock's config file.
  • de33412: Flintrock installs services, not modules. The terminology has been updated accordingly throughout the code and docs. Update your config file to use services instead of modules. Warning: Flintrock will have problems managing existing clusters that were launched with versions of Flintrock from before this change.
  • #73: Major refactoring of Flintrock internals.
  • #74: Flintrock now catches common configuration problems upfront and provides simple error messages, instead of barfing out errors from EC2 or launching broken clusters.
  • bf766ba: Fixed a bug in how Flintrock polls SSH availability from Linux. Cluster launches now work from Linux as intended.

0.1.0 - 2015-12-11

  • Initial release.