Wildcard inside top.sls file for pillar

[jeanpralo]

NOTE: Works for states, this is now an issue for adding this functionality for pillar

Hiy guys,

I am trying to use a wildcard feature documented here :

http://docs.saltstack.com/en/latest/topics/releases/0.9.9.html 7.2.49.1.9. Wildcards for SLS Modules

So I wrote this simple top.sls states inside /srv/pillar:

base:
  'ssh*':
    - users.*

The directory /srv/pillar/user directory contains two files user1.sls and user2.sls

But when trying to get the pillar items on the salt master I am getting an error :

ssh-1:
    ----------
    _errors:
        - Specified SLS 'users.*' in environment '58faaa5-dev' is not available on the salt master

Here is the version of salt I use :

root@salt-master:~/work# salt --version
salt 2015.2.0-742-g58faaa5 (Lithium)

I have asked the question on irc, the guys seemed to think that feature might have been killed.

Thanks for your help.

Cheers,

[basepi]

Have you attempted to use this with the state system instead of the pillar system? From the release notes, it appears this was implemented for the state system, but it may not have been implemented for the pillar system.

[jeanpralo]

Hiya,

Tested with the states works fine. Thought that the state system and pillar were at the same level of functionality, my mistake.

Would be nice to have it for the pillar system too.

Thanks anyway.

[iggy]

Would also be nice to document the differences between state and pillar top files.

[basepi]

Both great points. I've marked this as a feature.

[aboe76]

@basepi is this still on the wish list? I hope it isn't forgotten.... :)

[basepi]

Not forgotten, just busy. =)

[tehmaspc]

+1 - just came across this because i thought i could do the same :)

[pduersteler]

👍 also from our side

[michaelkarrer81]

+1 - really needed - Any possible workaround for now?

[wych42]

+1. need this feature.

[andre-brongniart]

+1 would be an awesome feature. Not sure what workaround to use other than what is recommended to "use a little Jinja and the file.readdir module you can loop over the contents of a directory and dynamically include them."... from https://groups.google.com/forum/#!topic/salt-users/QoYkSwyENiI
Not sure what they meant...

[sjorge]

i was under the impression it was not working in pillar and states. I will need to retest this later.
It would indeed be nice to have!

Edit: works fine for states in 2016.3/develop, just checked.

[mallniya]

👍 just stumbled upon the need of such a feature.

[shellhacks]

+1

[jwyoungpm]

+1

[onlyanegg]

+1

[mooperd]

+1

[danielmorlock]

+1

[pierluca]

+1

[jwyoungpm]

If anyone is interested, I'm using the following scripts to manage this at the deploy time of some of our applications:

https://gist.github.com/jwyoungpm/f74091ac41733deddf7f348f83a020fc

Keep in mind, this is relatively rudimentary and assumes that you have a folder called 'app' under /srv/pillar. This also assumes you have that app folder included in your top.sls:

/srv/pillar
   ->top.sls (has an include for the app folder)
   ->app
      ->init.sls

[nicomz]

+1. Would be very useful

[porunov]

+1

[pepoluan]

My workaround is by creating a custom module cat.py as follows:

#!/usr/bin/env python
# Module name: cat.py
from __future__ import print_function, unicode_literals

import os
import fnmatch


# noinspection PyShadowingBuiltins
def file(name, indent=0):
    ind, ret = ' ' * indent, []
    with open(name, 'rt') as fin:
        for ln in fin:
            ret.append('{0}{1}'.format(ind, ln.rstrip('\r\n')))
    return ret


def directory(name, recurse=False, indent=0, file_filter='*', dir_filter='*'):
    ret = []
    for root, dirs, files in os.walk(name):
        root_base = os.path.basename(root)
        if fnmatch.fnmatch(root_base, dir_filter):
            for fn in files:
                if fnmatch.fnmatch(fn, file_filter):
                    ret.extend(file(os.path.join(root, fn), indent=indent))
        if not recurse:
            break
    return ret

Then, in the proper location in the .sls file, add the following jinja:

{% for ln in salt.cat.directory( ... ) %}
{{ ln }}
{%- endfor %}

EDIT: The latest version will always be available on my Github repo: https://github.com/pepoluan/saltstack-custom-modules

[kapucko]

+1

[gerardjp]

+1

[wych42]

Seems this feature is already supported: https://docs.saltstack.com/en/latest/topics/pillar/

[lorengordon]

@marlowew I'm only seeing info there on globbing the targets, not the pillar paths.

Works (but not what this issue is about):

base:
  'foo*':
    - bar

Not works (what this issue is about):

base:
  'foo':
    - bar*

Did I miss something in that link?

[wych42]

@lorengordon my mistake, I will have a try to solve this issue.

[wych42]

fixed in #41423

[aboe76]

@marlowew nice thanks for this!

[PhilippeAB]

The fix in #41423 don't work for me

As a workaround you can create a folder with all your wildcard wanted file and create and init file with include for all the file in the folder preceded with the full path to the file
example
folder ./users/
with user1.sls, user2.sls
create init.sls file in ./users/ with
include:

• users.user1
• users.user2

[anitakrueger]

Is this really fixed with #41423?
I can't make it work either, with exactly the same use case (users) but using an include:

    _errors:
        - Specified SLS 'common.users.present.*' in environment 'base' is not available on the salt master

This is my include statement:

include:
  - common.users.present.*

[anitakrueger]

@marlowew I've tested this again and it looks like this works from the top file, but not in includes.
So if my top.sls looks like this, the * gets matched correctly:

base:
  '*':
    - common.users.present.*

If my top file just has the pillar file that then has the include, it doesn't match the *.

What would it take to fix this for includes do you think? Why are they different than matches in the top file?

[wych42]

@anitakrueger I haven't use this feature for months, but I will look into this next week.

[amontalban]

I also want to use it in include statements. Did you have time to look in to this @marlowew?

Thanks!

[wych42]

@amontalban Just pushed a PR #45269 , Still working on passing CI, but I have test it with a simple test server and worked.

[amontalban]

Thanks @marlowew, will give it a try!