Large refactor to support Ansible 2.5

[jackivanov]

The huge update is coming. We need people to get it tested and reviewed. It remains a Work-in-Progress.

Under the hood Ansible fixes

• Upgrade to Ansible 2.5
• Removes unneeded libraries that been merged to the Ansible upstream
• Generate the ssh key with openssl_ native modules
• Organize how facts are gathered and rid of unneeded playbooks
• Use proper boolean variable declarations
• Deprecate tags usage and move to the extra variables schema. Described here (Fixes Move to variables, disable tags #933)
• Rename the main playbook to main.yml and split it into 3 parts: input.yml, cloud.yml and server.yml
• Removes prompts in the algo script and moves them to input.yml and particular roles (Fixes Register inputs in a playbook #942, Fixes Scaleway Server Won't Allocate Unless Typed With Fingers B/C Of a Dumb WLS Bash Bug #833 and Fixes Fails with virtualenvwrapper #762)
• Travis-CI only performs tests via Docker

Cloud provider fixes

• Add a new cloud provider - Vultr (Fixes Potential new VPS providers for the installer #623)
• Enables EC2 volume encryption by default (Fixes EC2 AWS instance volumes aren't encrypted #989)
• Determine available regions for cloud providers at runtime (except Azure) (Fixes DigitalOcean : Region is not available. #941)

Usability fixes

• Adds more compatible server name (Reference Please Make A Guide For Setting Up in Amazon EC2 #987)
• Readme and documentation fixes
• Dump all the config options used into configs/$server_ip/config.yml
• Simplifies the update-users script to only ask for the CA password and the server IP. All the other options are taken from the new configuration dump (Fixes OnDemand rules is lost when using ./algo update-users #885)

Changes to the server

• Remove FreeBSD kernel rebuild since it is not required in FreeBSD 11+ (Fixes Support for *BSD #35)

[jackivanov]

@ookangzheng did you try it from scratch? Did you update the requirements? It worked and the latest travis build says it works

[zoonderkins]

Nope, I reinstall on my current vultr 18.04 ubuntu
I already manually uninstall Wireguard.. still stuck at there

[jackivanov]

@TC1977 I can't reproduce this one. Ensure that the requirements are correctly installed

[digeratus]

I'm seeing a ton of changes the last few days on this branch and I'd like to help test it. Just one question, what's the new way syntax to update users from ansible? In the past/currently, it looks like this:

ansible-playbook users.yml -e "server_ip=$server_ip server_user=$server_user ssh_tunneling_enabled=$ssh_tunneling_enabled IP_subject_alt_name=$IP_subject easyrsa_CA_password=$easyrsa_CA_password" -t update-users --skip-tags common

[TC1977]

@jackivanov I'm not sure what I'm doing wrong. The standard install of core dependencies and remaining dependencies on the readme.md is working every time for me. I'm running High Sierra which comes with python 2.7.10. I can install the master branch without any problems at all.

I upgraded python on my Mac to the latest 2.7.15 from https://www.python.org/downloads/release/python-2715/ and ran the install certificates and update shell profile commands, then made sure I was using 2.7.15 with python --version. I didn't install homebrew to do it. Algo install still fails at the same "no regions found" step. I also tried changing the ansible version in requirements.txt to 2.5.4. Still fails at the same step.

Is it an issue with AWS? I notice that it fails after I enter my access keys, and on the AWS console it doesn't show that the access keys have been used.

Anyone else with a Mac want to try?

[jackivanov]

@TC1977 did you try it on a clean python virtual environment?

[jackivanov]

@davidemyers Yes, definitely, it's covered here and here

[dguido]

How is this coming? I have used this branch a few times and did not encounter any errors.

[digeratus]

@dguido So far, the branch is solid.

The only topic I would love to see more clarity on is related to issue #963 for which the temporary fix is to disable dos_protection. Now, to me, I interpret that to be a possible Strongswan bug for which there might be a fix in the future. And that's fine. However, the fact still remains that the server might still be susceptible to DOS attacks. Additionally, it's not clear if the issue happens only when ondemand_cellular=true AND ondemand_wifi=true. Why do the Algo options matter? Well, if say ondemand_cellular=false is used, perhaps the issue goes away and the server is still "protected" from DOS attacks since no changes were made to charon. See where I'm going with this?

Perhaps, until the Strongswan bug is fixed, ondemand_cellular/wifi should not be used resulting in the worst case scenario that users will have to manually turn on their VPN. Is that better than having a weakened server? That's for the Algo user to decide, but they should have the information to make an educated decision.

Just my opinion.

[TC1977]

@digeratus I think it's somewhat of a moot point, because it doesn't look like disabling dos_protection really solves the problem. See #963.

[msfjarvis]

FWIW I just deployed a vultr machine with this branch and everything went through perfectly

[dguido]

This branch has been in testing for long enough so we're going to go ahead and merge it. I'm fully expecting that we'll encounter a few bugs but I think that merging it into master is the only way it will get the exposure needed to ferret them out.

[Hultner]

Yay proper FreeBSD support in mainline again